Compare commits

..
3 Commits
Author SHA1 Message Date
ducoterra b0c7aed613 phase: 165_prose_word_wrap
Build and Push Containers / build-and-push-app (push) Successful in 1m44s
Build and Push Containers / build-and-push-db (push) Successful in 11s
All verification complete. Final report:

**Phase 165 — Prose word wrap (final verification pass)**
- Verified: `PROSE_FORMATS={txt,text}` + `isProseDoc` (single-site, image arm wins); `renderDocument` three-way branch (md → `.doc-md`, prose → `.doc-prose` text node, else `pre.doc-raw` byte-identical); `.doc-prose` CSS = 72rem column, `pre-wrap` + `overflow-wrap: anywhere`, tokens only, zero literals; `[data-view="pages"]` hide list includes `.doc-prose` as one grouped declaration; phase-116 scrollbar rules intact and modal-scoped; no `innerHTML` in the prose path.
- Tests/lint (all 100% green):
  - `uv run pytest` → **3854 passed** (unit + integration)
  - `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90%)
  - `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**
  - `uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov` → **5 passed** (isolation)
  - Isolation regressions: document_viewer 8, image_documents 4, pdf_pages_in_sources 4, pdf_strip_additive_selfheal 5, edit_summaries 4, summary_in_viewer 4, chat_answer_clusters 4, garbage_pdf 3, pdf_support 6, sticky_navbar 3, smoke 3 — all passed
- Completion criteria: image description wraps, full text, both surfaces ✔; `.txt` wraps same way ✔; `py`/`json`/code keep `pre.doc-raw` + horizontal overflow + themed modal scrollbar (E2E negative control + pins) ✔; no innerHTML, AA `--ink`/`--surface` pair ✔; full gates green ✔; commit + phase-dir move are harness-owned (not executed per pipeline rules).
- Documentation: README already updated by task 03 (viewer section + image-docs section); full README/AGENTS.md review found no further inconsistencies — **verified consistent, no changes needed**.
- Notable deviations: `.doc-prose` omits `font-family: inherit` from 00_phase D2's snippet — task 01's authoritative "exact property set" excludes it and `div` inherits fonts anyway (behavior identical). PDF deferral (A4) intact: `format == "pdf"` stays in `.doc-raw`, owner call recorded.
- Next pending phase: `166_hide_empty_answer_bubble`.
2026-10-07 22:57:40 -04:00
ducoterra 8d70aa946c docs(phases): close two gaps in phase 167 - the date-pin tree consumers and drifted line anchors
Audit of the written phase found the depth=1 default was specified without
naming two of its nine consumers, both of which walk the tree without a
depth parameter and assert phase-106 D9 dates:

- tests/integration/test_docs_api_dates.py (reaches a -> a/b -> deeper)
- tests/e2e/test_git_source_dates.py (_find_file recurses by design)

Task 02 gains item 9 (both migrate by requesting ?depth=8 explicitly with
every assertion intact, plus the truncation-proof pin that a truncated
folder's updated_at still equals the deep max), task 06's isolation list
names the E2E suite, and the overview's D2 now carries the full nine-file
consumer list. Also corrected the Context anchors: build_kb_tree is L780
(not 781) and _subtree_max is L760 (not 746).

LLM-Generated: true
2026-10-07 19:49:06 -04:00
ducoterra dcb1062b83 docs(phases): convert TODO L3-L5 into phases 165-167 (prose wrap, no empty answer box, API payload diet)
Three owner-reported defects become executable phases; TODO.md is cleared
once the roadmap is written (Protocol B append, PLAN.md/AGENTS.md untouched).

- 165_prose_word_wrap (TODO L3): image descriptions and .txt bodies render as
  a wrapping .doc-prose block instead of <pre class="doc-raw"> (page + modal via
  the one renderDocument core). Code and code-ish formats keep horizontal scroll
  - phase 116 locked A1 survives; extracted PDF text stays raw and the deferral
  is recorded as an owner call, not an oversight.
- 166_hide_empty_answer_bubble (TODO L4): .msg .bubble:empty { display: none }.
  CSS-only, so the phase-158/161 "reasoning container inserts before the bubble"
  anchor and its unit pins stay untouched; the box returns at the first delta,
  and the shared/restore empty-answer corners take the same path.
- 167_api_payload_diet (TODO L5): selective pure-ASGI gzip (SSE /api/chat skipped
  by exact path), a depth-limited KB tree with server-side totals + truncated
  nodes whose recursive updated_at comes from a new aggregate map, a
  GET /api/docs/tree/node drill route, content-hash weak ETag/304 on the catalogue
  reads (NOT sources_meta - bumping it would mark saved chats stale), the lazy
  client drill, and a /api/* byte-budget log line plus scripts/api_payload_audit.py
  whose measured numbers land in AUDIT.md.

Phase 97's "one fetch, drill client-side" contract is deliberately superseded
(locked A3); phase 116 A1, phase 158/161 anchors and A19/A20 middleware order are
explicitly preserved.

LLM-Generated: true
2026-10-07 19:42:34 -04:00
48 changed files with 2817 additions and 67 deletions
@@ -0,0 +1,63 @@
# Phase 165 — Prose word wrap: image descriptions and plain-text docs stop scrolling horizontally
**Source:** `TODO.md` L3 — "Need word wrap for image explanations and other long text rather than scrolling horizontally"
**Story:** n/a (owner-directed UI defect)
**Context:** `frontend/assets/document.js::renderDocument` (the ONE viewer core, imported by `frontend/assets/document-modal.js:73`), `frontend/assets/styles.css` (`.doc-raw` ~L5218, `.doc-md` ~L4929), phases 10 (viewer), 36 (summary panel), 116 (themed raw scrollbar, **locked A1: code never wraps**), 122 task 04 (image docs: `content` = `summary` = the vision description), 148/163 (page strip + the Text/Pages toggle).
## Objective
An image document's description — and a plain-text (`.txt`) document's body — render as **wrapping prose** inside the 72rem reading column on both viewer surfaces (the `/document.html` page and the in-page modal), so a long sentence is read by scrolling vertically like every other text surface in the app. Code and code-ish content keep their horizontal scroller exactly as phase 116's locked A1 decided.
## Why it scrolls today (traced 2026-10-08)
`renderDocument` has two content branches: `format ∈ {md, markdown}` → `<div class="doc-md">` (wraps, centered 72rem column), **everything else** → `<pre class="doc-raw">`, and `.doc-raw` is `white-space: pre` + `overflow-x: auto`. An image document's `format` is its extension (`png`, `webp`, …) and its `content` is the model's prose description (phase 122 task 03), so the description lands in the `pre` and each generated line becomes one unbreakable line → horizontal scrollbar. A `.txt` document is the same shape.
## Design (shared by all tasks — the executor reads this, not the chat)
**D1 — the prose set is one named constant (task 01, LOCKED A1).** `frontend/assets/document.js`:
```js
const PROSE_FORMATS = new Set(["txt", "text"]);
function isProseDoc(doc) { return doc.is_image === true || PROSE_FORMATS.has(doc.format); }
```
`renderDocument`'s content branch becomes three-way: markdown → `.doc-md` (untouched) → prose → `<div class="doc-prose">` (NEW) → everything else → `<pre class="doc-raw">` (untouched). `is_image` wins regardless of format (an image doc is prose by construction, phase 122 A3). `doc.format` values that stay in `.doc-raw`: `py`, `yaml`, `yml`, `json`, `j2`, the quadlet family, `pdf`, and every unknown/extension-less format.
**D2 — `.doc-prose` is `.doc-md`'s column with prose rhythm (task 01, LOCKED A2).** `frontend/assets/styles.css`, adjacent to `.doc-raw`:
`width: 100%; max-width: var(--chat-column); margin-inline: auto; background: var(--surface); border: 1px solid var(--line); border-radius: var(--radius); box-shadow: var(--shadow); padding: 1.5rem 1.75rem; color: var(--ink); font-family: inherit; white-space: pre-wrap; overflow-wrap: anywhere; line-height: 1.55;`
The content stays a **text node** (`textContent`, never `innerHTML` — the XSS contract is untouched; the description is model prose with literal newlines, so `pre-wrap` keeps paragraph breaks without parsing markdown). Tokens only (the `:root`-drift test parses built-ins from this file — zero literals). No animation (nothing for `prefers-reduced-motion` to still).
**D3 — the code contract stays locked and gets pinned (task 02, LOCKED A3).** Phase 116's locked A1 ("keep horizontal scroll (NO line-wrap) for the code content") survives verbatim: `.doc-raw`, `.bubble pre`, `.md-table-wrap` are byte-identical, and phase 116 task 01's themed scrollbar rules (`.doc-modal .doc-raw`) keep applying to what still scrolls. The unit test asserts the exact prose set, so a future format cannot drift into prose by accident.
**D4 — verification (task 03).** Dedicated Playwright suite `tests/e2e/test_prose_word_wrap.py` (isolated run, deterministic mock LLM, the seeding idiom of `tests/e2e/test_image_documents.py` / `tests/e2e/test_document_viewer.py`): for a seeded image doc AND a seeded `.txt` doc with a > 200-char single line, assert in **both** surfaces (page + modal) that the prose block (a) contains the full text, (b) has `scrollWidth <= clientWidth + 1` (no horizontal overflow — the TODO's literal ask), (c) is taller than one line (proof it wrapped), and that a seeded `.py`/`yaml` doc STILL reports `scrollWidth > clientWidth` (the locked code path).
**Deferred, recorded, not silently dropped:** the same `pre`-scroll applies to **extracted PDF text** (`format == "pdf"` → `.doc-raw`, e.g. the IKEA-manual corpus). Wrapping it means touching `pre.doc-raw` assertions in three completed suites (`test_pdf_pages_in_sources.py`, `test_pdf_strip_additive_selfheal.py`, `test_garbage_pdf.py`) and the phase-163 strip contract — an owner call, one-line change once made (`PROSE_FORMATS.add("pdf")` + those pins). This phase deliberately does NOT change PDF rendering (no behavior change in completed phases without owner sign-off).
## Dependencies
- `122_image_documents` (complete) — image docs: `content` = `summary` = vision description, `is_image` on the content payload, `imageSummaryIsContent` suppression.
- `116_modal_scrollbar_theme` (complete) — **locked A1** (code keeps horizontal scroll) and the themed scrollbar this phase must leave working.
- `163_pdf_strip_additive_selfheal` (complete) — the `.doc-views` wrapper + Text/Pages toggle: the prose block mounts inside the same `slot` the `.doc-raw` path used, so `[data-view="pages"]` hiding must keep working (add `.doc-prose` to the hide list — task 01, it is the additive/PDF case's text view).
## Tasks
1. `01_doc_prose_block.md` — the prose-set constant + `renderDocument`'s third branch + `.doc-prose` CSS (page + modal, strip hide list) + unit pins.
2. `02_raw_scroll_pins.md` — pin the untouched code scrollers (`.doc-raw`, `.bubble pre`, `.md-table-wrap`, phase-116 scrollbar), pin the exact prose set + the XSS/textContent rule.
3. `03_e2e_prose_wrap.md` — `tests/e2e/test_prose_word_wrap.py` + the full gates + the docs pass + the commit.
## Testing & Quality
- Unit (extend, following the house CSS/JS source-assertion style of `tests/unit/test_image_documents.py`, `tests/unit/test_viewer_additive_toggle.py`, `tests/unit/test_document_viewer.py`): `isProseDoc` truth table (image × format, txt/text, py/yaml/json/j2/quadlet/pdf/unknown/extension-less); `renderDocument` output shape for each branch (`className`, `textContent`, no `innerHTML`); the `.doc-prose` rule contains `pre-wrap` + `overflow-wrap: anywhere` + `var(--chat-column)` and zero color literals; the `.doc-raw` rule still contains `white-space: pre` + `overflow-x: auto`.
- E2E: `tests/e2e/test_prose_word_wrap.py` green **in isolation** (`uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov`); `tests/e2e/test_image_documents.py`, `test_document_viewer.py`, `test_pdf_pages_in_sources.py`, `test_pdf_strip_additive_selfheal.py`, `test_edit_summaries.py`, `test_summary_in_viewer.py` stay green (PDF/raw paths unchanged).
- Coverage: **>90%** on `app/` (this phase is frontend+CSS only; the gate must not move).
## Completion Criteria
- [ ] An image document's description renders in `.doc-prose`, wraps at the reading column, and shows the FULL description (no clamp, no horizontal scrollbar) on both `/document.html` and the modal.
- [ ] A `.txt` document wraps the same way; `py`/`yaml`/`json`/`j2`/quadlet/`pdf`/unknown formats still render in `.doc-raw` with the horizontal scroller and the themed modal scrollbar.
- [ ] No `innerHTML` for document content anywhere in the new path; contrast pairs are the existing AA token pairs.
- [ ] `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean; the phase E2E green in isolation.
- [ ] One `--no-gpg-sign` Conventional Commit; the phase dir moves to `complete/`.
## Locked decisions
- **A1 — the prose set is exactly `{is_image: true} ∪ {format ∈ txt, text}`** (owner-confirmed 2026-10-07: "image descriptions + plain-text prose"); code and code-ish formats keep scrolling.
- **A2 — `.doc-prose` is a text-node `pre-wrap` block in the 72rem column** (no markdown parsing of a description — a description is prose, not markup; the phase-122 rule).
- **A3 — phase 116's locked A1 is NOT superseded**: code content keeps `white-space: pre` + `overflow-x: auto`.
- **A4 — extracted PDF text stays in `.doc-raw` for now** (see "Deferred" above) — an owner call, not an executor choice.
## Commit
```bash
git add frontend/ tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(viewer): wrap image descriptions and plain-text docs as prose instead of scrolling horizontally"
```
@@ -0,0 +1,38 @@
# Task 01 — The prose content block: `PROSE_FORMATS`, `renderDocument`'s third branch, `.doc-prose` (phase 165 D1/D2)
**Phase:** `165_prose_word_wrap` · **Source:** `TODO.md:3` — "Need word wrap for image explanations and other long text rather than scrolling horizontally"
**Story:** n/a
## Objective
The viewer renders an image document's description and a `.txt` document's body as a wrapping prose block (`.doc-prose`) instead of `<pre class="doc-raw">`, on both the `/document.html` page and the in-page modal, with the markdown and raw branches untouched.
## Work
1. `frontend/assets/document.js` (module scope, above `renderDocument`) — the ONE prose rule, phase 165 D1:
```js
const PROSE_FORMATS = new Set(["txt", "text"]);
function isProseDoc(doc) {
return doc.is_image === true || PROSE_FORMATS.has(doc.format);
}
```
Docstring: the image arm is unconditional (phase 122 A3 — an image doc's `content` IS the vision description, prose by construction, whatever its extension); `format` comes from the server's `doc_format()` (lowercased suffix). Name the TODO defect in the comment ("the description used to land in the `pre` and each generated line became one unbreakable line").
2. `frontend/assets/document.js` — `renderDocument`'s content branch (today: `if (doc.format === "md" || doc.format === "markdown") { … .doc-md … } else { … .doc-raw … }`): make it three-way and put the prose arm **between** them —
- markdown → the existing `.doc-md` div, byte-identical;
- `isProseDoc(doc)` → `<div class="doc-prose">` with `textContent = doc.content` (a **text node**: the XSS contract is unchanged; `pre-wrap` carries the newlines, no markdown parsing — locked A2);
- else → the existing `<pre class="doc-raw">`, byte-identical.
The prose block appends into the SAME `slot` variable the raw branch uses, so the phase-122 image block (`docImageBlock`) stays FIRST, the phase-36 summary panel (when not suppressed by `imageSummaryIsContent`) stays between them, and the phase-163 additive strip (`docPagesStrip`) still mounts last inside `.doc-views`.
3. `frontend/assets/styles.css` — new `.doc-prose` rule immediately after `.doc-raw` (~L5229), phase 165 D2, exact property set:
`width: 100%; max-width: var(--chat-column); margin-inline: auto; background: var(--surface); border: 1px solid var(--line); border-radius: var(--radius); box-shadow: var(--shadow); padding: 1.5rem 1.75rem; color: var(--ink); white-space: pre-wrap; overflow-wrap: anywhere; line-height: 1.55;`
(`--ink` on `--surface` is the documented 14.5:1 pair; **zero color literals** — the `:root`-drift unit test parses built-ins from this file.) Comment block cites: TODO L3, the `.doc-md` column precedent, `.doc-raw`'s `pre` as the defect, locked A1/A2, "static content — no animation".
4. `frontend/assets/styles.css:5361-5364` — add `.doc-views[data-view="pages"] .doc-prose` to the existing hide list (the four-selector rule that hides the text view behind the page strip). Defensive today (image/`txt` docs are never page-qualifying), load-bearing if the owner later adds `pdf` to `PROSE_FORMATS` — say so in the comment.
5. `tests/unit/test_image_documents.py` (extend; reuse its `renderDocument` harness) — the image-doc content branch now yields `className === "doc-prose"`, `textContent === description`, and NOT a `doc-raw` sibling; the summary-suppression case (`summary === content`) is unchanged.
6. `tests/unit/test_document_viewer.py` (extend) — the format table: `txt` / `text` → `doc-prose`; `md`/`markdown` → `doc-md` (innerHTML through `renderMarkdown`, unchanged); `py`, `yaml`, `json`, `j2`, `conf`, `pdf`, `""`, an unknown suffix → `doc-raw` (proves locked A1/A3 unchanged); the never-`innerHTML` pin for the prose arm (`assert "innerHTML" not in prose_branch`).
7. `tests/unit/test_viewer_additive_toggle.py` (extend) — the `[data-view="pages"]` hide-list test enumerates `.doc-summary`, `.doc-md`, `.doc-raw`, `.doc-image`: add `.doc-prose` to the expected selectors and assert the rule stays ONE grouped declaration.
## Testing & Quality
- Unit: items 5–7 + the CSS pins (`.doc-prose` contains `pre-wrap`, `overflow-wrap: anywhere`, `var(--chat-column)`; `.doc-raw` still contains `white-space: pre` and `overflow-x: auto`).
- Coverage: **>90%** on `app/` unchanged (no `app/` change in this task).
## Completion Criteria
- [ ] `isProseDoc` is the only place the prose rule lives (grep-verified: no second `txt` literal in the content branch).
- [ ] Image docs and `.txt` docs render `.doc-prose` on page + modal (both call `renderDocument`); every other format's markup is byte-identical to pre-task.
- [ ] No `innerHTML` on document content in the new path; full `uv run pytest` green.
@@ -0,0 +1,26 @@
# Task 02 — Pin the untouched scrollers: code keeps horizontal scroll (phase 165 D3, LOCKED A1/A3)
**Phase:** `165_prose_word_wrap` · **Source:** `TODO.md:3` — the "other long text" half of the item, bounded by phase 116's locked A1 ("keep horizontal scroll (NO line-wrap) for the code content … wrapping breaks code readability")
**Story:** n/a
## Objective
The word-wrap fix is provably scoped: every surface that must keep scrolling horizontally still declares it, the prose set is pinned as data, and the chat/table scrollers are asserted byte-identical — so "we fixed wrapping" can never later mean "we wrapped code" or "we wrapped wide tables" by accident.
## Work
1. New `tests/unit/test_prose_word_wrap.py` (the house CSS/JS source-assertion style — read `tests/unit/test_markdown_tables.py` and `tests/unit/test_date_editor.py` first; they parse `frontend/assets/styles.css` and the JS files as text):
- **the prose set is exactly `{txt, text}` + `is_image`**: assert the `PROSE_FORMATS` literal contains exactly `"txt"`, `"text"` (a set-membership assertion over the source line, so adding a format is a deliberate, test-editing act); assert `isProseDoc` reads `doc.is_image` first;
- **code scrollers present and unchanged**: `.doc-raw {` block still contains `white-space: pre` and `overflow-x: auto`; `.bubble pre` still contains `overflow-x: auto` (no `white-space` added); `.md-table-wrap { overflow-x: auto; }` and `.md-table { … min-width: max-content … }` still match the phase-44 declaration;
- **phase 116 A1 stays locked**: the four `.doc-modal .doc-raw` scrollbar rules (`scrollbar-width`, `scrollbar-color`, `::-webkit-scrollbar` ×3) are all still present and still scoped to `.doc-modal`;
- **`.doc-prose` never reaches for the raw scroller**: the `.doc-prose` block contains no `overflow-x` and no `white-space: pre;` (only `pre-wrap`).
2. `tests/unit/test_prose_word_wrap.py` — the contrast/AA pin for the new block: `.doc-prose`'s `color` is `var(--ink)` and its background `var(--surface)` (the documented 14.5:1 pair), and the block contains no hex/`rgb(` literal (the phase-92 zero-literal invariant).
3. Repo hygiene pass: `grep -rn "doc-raw" frontend/` — confirm the only remaining writers of `doc-raw` are the raw branch of `renderDocument` and the phase-116 CSS; if `frontend/doc-edit.html` / `doc-edit.js` / `shared.*` render document content, verify they are NOT a second content renderer that needs the same rule (they are not expected to be — record the finding in the test's docstring).
4. ASSUMPTION: PDF-extracted text (`format == "pdf"`) is deliberately OUT of the prose set — it stays in `.doc-raw` because three completed suites pin `pre.doc-raw` for PDFs and the phase-163 strip contract; the deferral is recorded in `00_phase.md` ("Deferred") and in this test's docstring, so it is an owner decision on record, not an oversight.
## Testing & Quality
- Unit: the new file (all assertions are static-source assertions — no app dependency); run the phase-116/44/163-owning unit files to prove nothing moved: `uv run pytest tests/unit -k "additive_toggle or markdown_tables or date_editor or image_documents or document_viewer" -v`.
- Coverage: **>90%** on `app/` unchanged.
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_prose_word_wrap.py -v` green.
- [ ] The pin set proves: code/raw/tables keep `overflow-x: auto`; `.doc-prose` has none; the prose set is `{txt, text}` + image.
- [ ] Full `uv run pytest` green; ruff + pyright clean; no `app/` change.
@@ -0,0 +1,34 @@
# Task 03 — The wrapping proof: `tests/e2e/test_prose_word_wrap.py`, migrating the two suites that pinned `pre.doc-raw` for prose, final gates (phase 165 D4)
**Phase:** `165_prose_word_wrap` · **Source:** `TODO.md:3` — "Need word wrap for image explanations and other long text rather than scrolling horizontally"
**Story:** n/a
## Objective
A dedicated, isolated Playwright suite proves the TODO's literal claim (no horizontal scrollbar, full text visible, both surfaces, both prose classes) and the two completed suites whose fixtures ARE prose are migrated to the new contract — while the code path that keeps scrolling is still proven to scroll.
## Work
1. New `tests/e2e/test_prose_word_wrap.py` — module-scoped app fixtures + `_reset_db`/seed idiom copied from `tests/e2e/test_document_viewer.py` (read it first; `_seed_long_line_doc` shows the direct-seed pattern) and the image seeding of `tests/e2e/test_image_documents.py`:
- **seed**: one image doc (`is_image`, `content` = `summary` = a ~900-char single-paragraph vision description with `image_path` + a served `image_url`) and one `notes/prose-long.txt` doc whose content is a single ~900-char line plus a second paragraph (proves BOTH intra-line wrap and paragraph breaks); plus one `notes/code-long.py` doc with one ~900-char line (the negative control).
- **assert, per prose doc, on BOTH surfaces** (`/document.html?source=…&path=…` and the same-page modal via the Sources row / chip click):
- `#doc-content .doc-prose` (and `#doc-modal-content .doc-prose`) count == 1 and `to_have_text(<full description>)` — nothing clamped, nothing truncated;
- `el.scrollWidth <= el.clientWidth + 1` — **the TODO's literal defect: no horizontal overflow**;
- `el.getBoundingClientRect().height > 2.2 × line-height` — the text really wrapped;
- the computed `white-space === "pre-wrap"` and the block's max-width resolves to the 72rem column (never wider than the container);
- the image doc: the `<img class="doc-image-img">` still renders ABOVE the prose block (phase 122 ordering) and the summary panel stays suppressed (`summary === content`).
- **negative control**: `notes/code-long.py` renders `pre.doc-raw` with `scrollWidth > clientWidth` (phase 116 A1 alive).
- `--no-cov` isolation run per AGENTS.md rule 9.
2. `tests/e2e/test_document_viewer.py` — MIGRATE the phase-116 scrollbar proof: `_seed_long_line_doc()` seeds `notes/long-line.txt`, which is now prose. Change the seed to `notes/long-line.json` (same long line, a code-ish format), keep every assertion (`pre.doc-raw`, real `scrollWidth > clientWidth`, the themed `scrollbar-width`/`scrollbar-color` reading, the screenshots) and update the docstring + the file's outline lines 27/42 so they say the raw block is a CODE doc and that prose wraps (cross-reference phase 165). Do NOT weaken the overflow assertion.
3. `tests/e2e/test_image_documents.py:390,410` — MIGRATE the viewer assertion: the image doc's description now lives in `.doc-prose` (keep `to_have_text(seeded["description"])`), and the docstring line that says "the plain-content path (`pre.doc-raw`)" becomes the prose path; the rest of the suite (thumbnail, alt, chips, figure) is untouched.
4. `tests/unit/test_image_documents.py:1042` — update the comment/assert pair that names "the existing plain-content path — doc-raw via textContent" to the prose path (task 01 already added the shape assertion; here just remove the stale claim).
5. Docs pass + gates: `README.md` — the document-viewer section gains one sentence (prose formats wrap, code keeps the horizontal scroller; the phase-116 themed scrollbar applies to what still scrolls). `AGENTS.md` — verify the §7 UI list needs no change (the 72rem reading column already covers this). Then the full gate set: `uv run pytest`, `uv run pytest --cov=app --cov-report=term-missing` (> 90%), `uv run ruff check . && uv run pyright`, the phase E2E in isolation, plus the regression set that owns the surfaces touched: `tests/e2e/test_document_viewer.py`, `tests/e2e/test_image_documents.py`, `tests/e2e/test_pdf_pages_in_sources.py`, `tests/e2e/test_pdf_strip_additive_selfheal.py`, `tests/e2e/test_edit_summaries.py`, `tests/e2e/test_summary_in_viewer.py`, `tests/e2e/test_chat_answer_clusters.py` (each in isolation).
6. Move `.agents/phases/todo/165_prose_word_wrap/` → `.agents/phases/complete/` and commit with the `00_phase.md` message (AGENTS.md rule 8, `--no-gpg-sign`).
## Testing & Quality
- E2E: the new suite green in isolation + the seven regression suites green in isolation (the migration in items 2–3 must keep each suite's ORIGINAL claim true, just on the right element).
- Coverage: **>90%** on `app/` (no `app/` change; the gate must not move).
## Completion Criteria
- [ ] `uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov` green — image description and `.txt` body wrap with zero horizontal overflow on page + modal, full text present.
- [ ] The `code-long.py` control still overflows horizontally (locked A1 pinned end-to-end, not just in CSS).
- [ ] `test_document_viewer.py` and `test_image_documents.py` pass with their phase-116/122 claims intact on the new elements — no assertion deleted, only re-pointed.
- [ ] `uv run pytest` green, coverage > 90%, ruff + pyright clean, one atomic `--no-gpg-sign` commit, phase dir in `complete/`.
@@ -0,0 +1,57 @@
# Phase 166 — No empty answer box before the first token
**Source:** `TODO.md` L4 — "There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly." (the owner attached `image.png`: mid-turn the chat shows `Thinking` ▸, `Tool calls (2)` ▸, an expanded `Thinking` run with text, and BELOW it a bordered, padded, empty rounded box ≈ 40px tall)
**Story:** n/a (owner-directed UI defect)
**Context:** `frontend/assets/styles.css` (`.bubble` ~L501, `.bubble typing`, `.msg-body` flex gap ~L489), `frontend/assets/app.js` (`addMessage` L1386, `addTyping` L1433, `ensureReasoningWrap` L1610-1625, the `if (!wrap) wrap = addMessage("brain", "")` sites at ~4063 thinking / ~4164 tool / ~4269 first delta, `finalizeFailedTurn` ~3841, the restore path ~2864), `frontend/assets/shared.js:113` (the same `.msg/.msg-body/.bubble` structure), phases 06 (loading feedback), 17/117/158/161 (the per-run timeline that inserts BEFORE `.bubble`), 120/164 (failed-turn bubbles).
## Objective
While a turn is in flight, nothing paints an empty answer bubble: the box appears **at the first answer token**, and every visible-feedback affordance the app already owns (the typing indicator, the reasoning container with its live status, `#send-status`, the elapsed clock) keeps working exactly as before. Same rule covers the shared page's empty persisted answer, which renders the same empty box today.
## Why the box exists (traced 2026-10-08)
The first `thinking`/`tool` frame of a turn calls `addMessage("brain", "")` (`app.js:4063` / `:4164`) — an EMPTY bubble is created on purpose, because `ensureReasoningWrap` uses it as the DOM anchor (`body.insertBefore(cont, body.querySelector(".bubble"))`). `.bubble` carries `padding: 0.7rem 1rem` + `border` + `background` + `box-shadow`, so an empty one paints a box; the `.msg-body` flex gap adds the visible separation. `renderMarkdown("")` returns `""` (verified), so the restore path produces the same empty element — and the shared page (`shared.js:113`) too.
## Design (shared by all tasks — the executor reads this, not the chat)
**D1 — hide it in CSS, do not restructure the JS (task 01, LOCKED A1).** `frontend/assets/styles.css`, in the Messages block right after `.bubble p`:
```css
.msg .bubble:empty { display: none; }
```
`:empty` matches only an element with NO child nodes, which is exactly what `addMessage("brain", "")` produces (`<div class="bubble"></div>`); the first delta writes `innerHTML` and the box returns on the same frame. `display: none` also drops the bubble from the `.msg-body` flex layout, so the gap collapses (no orphan whitespace). The typing indicator is `.bubble typing` WITH three `<span>` children, so it is never `:empty` and its animation is untouched. Scoped to `.msg .bubble` — the viewer's `.doc-*` blocks and the composer are unaffected.
**D2 — the JS anchor stays (task 02).** Creating the empty wrap and anchoring the reasoning container before it is the phase-158/161 contract, pinned by `tests/unit/test_frontend_tool_states.py:61`, `tests/unit/test_thinking_timeline.py:475,522`, `tests/unit/test_frontend_edit_message.py:971,1072,1177`. Changing it (create-the-bubble-at-first-delta) would re-own those anchors for zero user-visible gain over D1 — so `app.js` is **not** modified by this phase; task 02's job is to prove the feedback contract holds with the box hidden.
**D3 — §7.4 feedback contract, verified not assumed (task 02).** With the box hidden, a live turn still shows, in order: `#typing-indicator` (role=status, `TYPING_LABEL()`, elapsed-seconds suffix from `thinkingClock`) → the open `details.reasoning` container with its live status span ("Thinking…" / "Calling <name>…" / "N steps") → the answer bubble at the first delta. Nothing else in the app depends on the empty bubble being painted: `scrollReveal` measures the `.msg` wrap (not the bubble), the phase-161 window pins measure `.reasoning-body`, the phase-18 follow-bottom logic is user-intent-gated, and `#messages` remains the single `aria-live="polite"` region (an `aria-live` announcement fires when content lands — at which point the bubble is visible again).
**D4 — verification (task 03).** Dedicated suite `tests/e2e/test_no_empty_box.py`: stub `POST /api/chat` with a controllable SSE stream (`thinking` → `tool` → a pause → `delta`), assert `page.locator(".msg.brain .bubble").first` is `not_to_be_visible()` (or has zero box) during the pre-delta window while `#typing-indicator` / `details.reasoning[open]` / `#send-label` ARE in their expected states, and that after the first `delta` the same locator becomes visible and holds the token. Plus a restore case (a saved chat whose last brain record has empty text renders no empty box) and the shared-page case.
## Dependencies
- `161_reasoning_bubble` + `158_thinking_timeline` (complete) — the container-before-the-bubble anchor this phase keeps, and the unit pins that must stay green untouched.
- `06_story_loading_feedback` + `109_turn_progress_loader` + `120_failed_turn_retry` (complete) — the visible-feedback states that must not regress when the box stops painting.
- `130_chat_answer_clusters` (complete) — `.msg-meta` rows are siblings of `.bubble`, never children: hiding an empty bubble never hides a chip/action row (pin it).
## Tasks
1. `01_hide_empty_bubble.md` — the CSS rule + its provenance comment + the unit pins (styles.css source assertions + the typing-indicator non-emptiness pin).
2. `02_feedback_contract_check.md` — prove §7.4 endures: the existing state-machine pins stay green, the shared/restore empty-answer corners are covered, and nothing measures the hidden box.
3. `03_e2e_no_empty_box.md` — `tests/e2e/test_no_empty_box.py` + the regression set + the full gates + the commit.
## Testing & Quality
- Unit: new `tests/unit/test_no_empty_answer_bubble.py` (the house "read the source file, assert on the rules" style — see `tests/unit/test_thinking_scroll.py`): the `.msg .bubble:empty { display: none; }` declaration exists exactly once and sits in the Messages block; `.bubble` itself keeps `padding`/`border`/`background`/`box-shadow` (the rule hides, it does not restyle); `addTyping`'s markup still contains the three `<span>` dots (⇒ never `:empty`); `app.js` still contains the three `addMessage("brain", "")` anchors (D2 — the rule is CSS-only).
- E2E: `tests/e2e/test_no_empty_box.py` green **in isolation**; the owning regression set green in isolation: `test_thinking_timeline.py`, `test_turn_progress_loader.py`, `test_chat_history.py`, `test_share_chat.py`, `test_save_share_ux.py`, `test_failed_turn_retry.py`, `test_chat_answer_clusters.py`, `test_chat_error_honesty.py`.
- Coverage: **>90%** on `app/` (no `app/` change; the gate must not move).
## Completion Criteria
- [ ] Mid-turn (thinking and/or tool frames, zero deltas) NO empty answer box paints — the reasoning container and the typing indicator are the only brain-side boxes.
- [ ] The answer bubble appears with the first delta, in the same DOM position, with the source chips / actions / figures below it exactly as phase 130 arranged them.
- [ ] A restored or shared answer whose persisted text is empty paints no empty box either; every non-empty bubble is byte-identical.
- [ ] No `app.js` change; the phase-158/161 anchor unit pins pass untouched; `uv run pytest` green, coverage > 90%, ruff + pyright clean.
- [ ] One `--no-gpg-sign` Conventional Commit; the phase dir moves to `complete/`.
## Locked decisions
- **A1 — CSS-only suppression** (`.msg .bubble:empty { display: none }`); the JS continues to create the empty bubble as the reasoning anchor (owner-confirmed 2026-10-07: "hide until the first token", not "restructure the anchor").
- **A2 — nothing replaces the box**: no skeleton, no shimmer — the existing loading feedback (typing indicator → reasoning container → status label) is the turn's visible state (PLAN §7.4), owner-confirmed 2026-10-07.
- **A3 — the rule extends to the shared page and the restore path** (same `.msg .bubble` structure; an empty persisted answer is the same defect), owner-confirmed 2026-10-07.
## Commit
```bash
git add frontend/ tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(chat): stop painting an empty answer bubble before the first token"
```
@@ -0,0 +1,44 @@
# Task 01 — `.msg .bubble:empty { display: none }`: the empty answer bubble stops painting (phase 166 D1)
**Phase:** `166_hide_empty_answer_bubble` · **Source:** `TODO.md:4` — "There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly." (attached `image.png`: Thinking ▸ / Tool calls (2) / an expanded Thinking run, then an empty bordered box)
**Story:** n/a
## Objective
One CSS rule suppresses the empty answer bubble during a turn (and for an empty persisted answer), with the box returning the instant the first token lands — no JS change, no restyle of a non-empty bubble, no change to any loading affordance.
## Work
1. `frontend/assets/styles.css` — in the `---------- Messages ----------` block, directly after `.bubble p { margin: 0.2rem 0; }` (~L509), add:
```css
/* Phase 166 (TODO L4): the answer bubble an in-flight turn creates is
EMPTY by construction — app.js:4063/:4164 call addMessage("brain","")
so ensureReasoningWrap has an anchor to insert the per-run timeline
BEFORE. With .bubble's own padding + border + background + shadow an
empty one paints a ~40px box, which is the "empty box before the LLM
responds" the owner reported. :empty matches only a child-less
element — exactly innerHTML === "" (renderMarkdown("") is "", so the
restore and the shared page take the same path) — and the first
delta writes content, so the box appears AT the first token.
display:none also drops the flex item, so .msg-body's gap collapses.
The typing indicator is never :empty (three dot <span>s) — its
animation is untouched. Scoped to .msg .bubble: the viewer's .doc-*
blocks and the composer never match. */
.msg .bubble:empty { display: none; }
```
No new tokens, no animation (nothing for `prefers-reduced-motion` to still).
2. `tests/unit/test_no_empty_answer_bubble.py` (new — static source-assertion style, copy the harness of `tests/unit/test_thinking_scroll.py` / `tests/unit/test_markdown_tables.py`):
- the declaration appears **exactly once**, spelled `.msg .bubble:empty { display: none; }`, and sits after `.bubble {` and before `.bubble pre {` (provenance ordering, the `test_date_editor.py:411` `css.find(...)` idiom);
- the `.bubble {` rule still declares `padding`, `border-radius`, `background`, `border`, `box-shadow` (the fix HIDES, it never restyles);
- `.bubble typing` / the typing-dot rules are still present and unchanged (the indicator stays visible);
- `frontend/assets/app.js` still contains all three `addMessage("brain", "")` call sites and `ensureReasoningWrap` still does `body.insertBefore(cont, body.querySelector(".bubble"))` (D2: the anchor survives — this is the pin that stops a future "cleanup" from moving the fix into JS and breaking the timeline);
- `frontend/assets/markdown.js` — `renderMarkdown("")` and `renderMarkdown(" ")` both return `""` (execute the file the way the existing markdown unit tests do), which is WHY the restore/shared paths are covered by the same rule;
- no CDN / no `url(http` introduced (the AGENTS.md rule-6 idiom these files already assert).
3. `frontend/assets/app.js` — **no change** (explicitly). If the executor believes a JS change is needed, stop and flag it: LOCKED A1 (the reasoning-container anchor and the phase-158/161 pins are the reason this is CSS-only).
## Testing & Quality
- Unit: the new file; then `uv run pytest tests/unit -k "thinking_timeline or tool_states or edit_message or frontend" -v` — the phase-158/161/159 anchor pins pass untouched.
- Coverage: **>90%** on `app/` unchanged (no `app/` change).
## Completion Criteria
- [ ] The rule exists exactly once, scoped, with the provenance comment naming TODO L4, `image.png`, and the anchor it preserves.
- [ ] `addTyping()`'s bubble is provably non-empty (unit-pinned) so the loading state never disappears.
- [ ] `git diff` for this task touches `frontend/assets/styles.css` + one new unit test file only.
@@ -0,0 +1,32 @@
# Task 02 — §7.4 audit: the turn is never feedback-less with the box hidden (phase 166 D3)
**Phase:** `166_hide_empty_answer_bubble` · **Source:** `TODO.md:4` — hiding the empty box must not remove the owner's only mid-turn signal; PLAN §7.4 ("never stale": every state has a defined UI) and AGENTS.md rule 10
**Story:** n/a
## Objective
Prove — with tests, not prose — that every mid-turn state still has a visible affordance with the empty bubble suppressed, that no code path measures the hidden box, and that the restore and shared-page corners render no empty box either.
## Work
1. `tests/unit/test_no_empty_answer_bubble.py` (extend) — the **feedback inventory** pins, each naming the state it guarantees:
- `addTyping()` markup contains `role="status"`, `aria-label="${TYPING_LABEL()}"` and the three `<span></span>` dots (the pre-thinking state);
- `thinkingClock` still writes the `Ns` `.typing-elapsed` suffix into `#typing-indicator .bubble` (the long-tool-call state) — and note in the comment that this bubble is never `:empty`, so the clock keeps painting;
- `setReasoningStatus("Thinking…")` / `"Calling "` / the settle `"N steps"` literals are still wired at their three call sites (the thinking/tool state), and `ensureReasoningWrap` sets `cont.open = true` (the container is visible while live);
- `#send-status` / `#send-label` state copy is untouched (the composer-side signal), and `#messages` keeps `aria-live="polite"` (`frontend/index.html:167`) — the ONE conversation live region; an announcement fires when content lands, at which point the bubble is visible again (comment this reasoning).
2. `tests/unit/test_no_empty_answer_bubble.py` (extend) — the **"nothing measures the hidden box"** audit, executed as source assertions with the finding recorded per site:
- `scrollReveal(wrap)` receives the `.msg` WRAP (not the bubble) — pin the call site `if (scroll) scrollReveal(wrap);`;
- the phase-161 window pins operate on `.reasoning-body` (`isReasoningPinned` / `scrollReasoningToBottom`) — pin that neither takes a `.bubble`;
- `.bubble` is only ever read for CONTENT (`bubble.textContent`, `bubble.innerHTML`, `bubble.querySelector(".msg-image")`, `bubble.classList`) in `app.js`/`shared.js` — grep-assert there is no `getBoundingClientRect`/`offsetHeight`/`clientHeight` on a `.bubble` selector;
- `.msg-meta` (chips, actions, figures, `.failed-note`, `.stopped-note`) are appended to `.msg-body` — pin one append site each so a future refactor cannot move them INTO the bubble (where hiding an empty one would hide them).
3. `tests/unit/test_no_empty_answer_bubble.py` (extend) — the **restore / shared corners**:
- `app.js` restore renders brain bubbles through `addMessage("brain", renderMarkdown(m.text), true)` — with `m.text === ""` the bubble is child-less ⇒ `:empty` ⇒ hidden (assert the call shape, plus the `renderMarkdown("")` fact already pinned in task 01);
- `shared.js` builds the identical `<div class="bubble">${html}</div>` structure (its line ~113), so the same `.msg .bubble:empty` scope covers the shared page — assert the rule's scope prefix `.msg ` matches both (the shared page uses the same `.msg`/`.msg-body` classes).
4. Findings note: append a short `AUDIT NOTE:` block to the test module docstring listing the four mid-turn states and the affordance that carries each after this phase (idle/typing → typing dots + clock; thinking/tool → reasoning container + status span; first token → the bubble appears; terminal → answer/deflection/stopped/failed copy). This is the record the next agent reads instead of re-deriving it.
## Testing & Quality
- Unit: the extended file + `uv run pytest tests/unit -k "frontend or reasoning or thinking or tool" -v` green; no `app/` change, coverage gate unmoved.
## Completion Criteria
- [ ] Every mid-turn state has a pinned, visible affordance with the empty bubble suppressed (the four-state inventory is in the test docstring).
- [ ] No code path measures a `.bubble` box (grep-pinned), so `display:none` cannot shift a scroll or a pin.
- [ ] The restore and shared paths are covered by the same scope (pinned), with no `shared.js` change needed.
- [ ] Full `uv run pytest` green.
@@ -0,0 +1,34 @@
# Task 03 — `tests/e2e/test_no_empty_box.py`: the mid-turn proof, a paced mock turn, and the final gates (phase 166 D4)
**Phase:** `166_hide_empty_answer_bubble` · **Source:** `TODO.md:4` — "There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly."
**Story:** n/a
## Objective
An isolated Playwright suite proves the defect is gone with a race-free timeline: during the pre-delta window (thinking frames + a tool call, no answer token) the brain message paints NO answer box, the reasoning container and the typing/loading affordances DO paint, and the box appears exactly with the first delta. Plus the deterministic mock turn that makes that window long enough to assert, the regression set, and the phase gates.
## Work
1. `tests/e2e/mock_llm.py` — a new dedicated paced trigger (the `TURN_PROGRESS_TRIGGER` precedent, its block documents the convention): `EMPTY_BUBBLE_TRIGGER = "paint the pre-answer window"` (verify the phrase appears in NO existing E2E question and NO fixture file — the phase-109 suite's disjointness check, grep before committing), streaming only, with baked-in delays so the windows are ≥ 1 s:
- request 1 (tools offered, no tool results): `TURN_*`-style local constants (defined next to the trigger, never inline magic): a ~1.2 s pre-delay, then 3 `reasoning_content` chunks at ~0.4 s (≈ 1.2 s of thinking with **no** `delta.content`), then one synthetic `ls` tool call (`call_0`, `finish_reason: "tool_calls"`) — i.e. request 1 NEVER emits a content delta;
- request 2 (a `tool`-role result present): a ~2.0 s gap, then 3 short `delta.content` chunks (0.05 s apart) + `finish_reason: "stop"`;
- the module docstring's trigger inventory gains the entry; every other suite's classification is untouched (stateless discrimination from the messages, like the phase-109 flow).
2. New `tests/e2e/test_no_empty_box.py` (module-scoped `app_server`/`app_url` + `_reset_db(mock_llm, seed=True)` + `login()` copied from `tests/e2e/test_reasoning_bubble.py`, which is the closest live-stream suite — read its `_submit`/`_wait_answer` helpers first):
- **the defect window**: submit `EMPTY_BUBBLE_TRIGGER` question; while the thinking/tool frames stream (`expect(page.locator("details.reasoning")).to_be_visible()` first, as the sync point) assert on `.msg.brain .bubble`: `be_not_visible()` AND a zero-height box (`evaluate("el => el.getBoundingClientRect().height") === 0`) — the box does not paint; assert `details.reasoning[open]` is visible, its status span carries the live literal (`Thinking…` / `Calling `), and `#send-label` reads the in-flight state;
- **the settle**: after `done`, the SAME locator is visible, `height > 0`, and contains the answer text; the container folds (`details.reasoning:not([open])`), the chips/actions rows are present below the bubble (phase 130 ordering survives);
- **no-thinking control**: a plain answer question (no thinking, no tools) shows `#typing-indicator` before the settle (the typing bubble has dots ⇒ visible) and no empty box at any point — assert `#typing-indicator .bubble` height > 0 while in flight;
- **restore**: save the answered chat, reload `/?chat=<id>` — the answered turn renders its bubble normally (non-empty), and NO `.bubble:empty` exists in the DOM (`page.locator(".msg .bubble:empty").count() === 0`);
- **shared page**: share the chat, open `/shared/<token>` — the same zero-`:empty` assertion (LOCKED A3);
- screenshot of the mid-turn state (the owner's `image.png` before/after record) into the suite's screenshot dir the way `test_document_viewer.py` does.
3. Regression run (each **in isolation**, `--no-cov`): `tests/e2e/test_thinking_timeline.py`, `test_reasoning_bubble.py`, `test_turn_progress_loader.py`, `test_chat_history.py`, `test_share_chat.py`, `test_save_share_ux.py`, `test_failed_turn_retry.py`, `test_chat_answer_clusters.py`, `test_chat_error_honesty.py`, `test_markdown_tables.py` (the `.bubble pre` scroller), `test_wide_desktop_column.py` (the chat column metrics).
4. Docs + gates: `README.md` — one sentence in the chat-turn/loading section (the answer bubble appears at the first token; mid-turn feedback is the typing indicator + the reasoning container). Then `uv run pytest`, `uv run pytest --cov=app --cov-report=term-missing` (> 90%), `uv run ruff check . && uv run pyright`, the phase E2E in isolation.
5. Move `.agents/phases/todo/166_hide_empty_answer_bubble/` → `.agents/phases/complete/` and commit with the `00_phase.md` message (`--no-gpg-sign`).
## Testing & Quality
- E2E: the new suite green in isolation (its windows are ≥ 1 s by construction — no `sleep`-style flake), plus the ten regression suites above.
- Mock: the new trigger changes NO other suite (grep-proven disjoint phrase; the full E2E smoke `tests/e2e/test_smoke.py` green).
- Coverage: **>90%** on `app/` (mock + tests only here; no `app/` change in the phase).
## Completion Criteria
- [ ] Mid-turn: `height === 0` / not visible on `.msg.brain .bubble` while thinking + tool frames stream; the reasoning container, the status literal, and the composer state are all visible (screenshotted).
- [ ] The box appears with the first delta and holds the answer; chips/actions/figures keep their positions below it.
- [ ] Zero `.bubble:empty` elements after a restore and on the shared page.
- [ ] `uv run pytest` green, coverage > 90%, ruff + pyright clean, all ten regression suites green in isolation, one atomic `--no-gpg-sign` commit, phase dir in `complete/`.
@@ -0,0 +1,82 @@
# Phase 167 — API payload diet: gzip, a lazily-drilled KB tree, revalidation, and a measured byte budget
**Source:** `TODO.md` L5 — "GET <https://brain.reeseapps.com/api/docs/tree> loads 2 MB in production which causes long loading times. Fix that any any other endpoints that could load a ton of data."
**Story:** n/a (owner-directed performance defect)
**Context:** `app/api/docs.py` (`GET /api/docs/tree` L919, the pure builder `build_kb_tree` L780 / `_level_children` L671 / `_folder_counts` L626 / `_subtree_max` L760, `GET /api/docs` L129), `app/schemas.py` (`KbTree` L576, `KbTreeFolder` L501, `KbTreeSource` L539, `KbTreeFile` L447 + the `_drop_image_fields` omission precedent), `app/main.py` (middleware stack: `SessionMiddleware` → routers → `configure_caching` → `SecurityHeadersMiddleware` outermost), `app/core/caching.py` (the `/api/*` pass-through contract), `app/core/logging.py` (§9 log style), `frontend/assets/sources.js` (`loadTree` L1625, `renderLevel` L1888, `treeStats` L1703, `folderExistsIn` L1667, `resetVanishedLocation` L1646), `app/models.py` (`SourcesMeta` — deliberately NOT the ETag source, see A3), phases 94/97/98/99/106/122 (the tree contract), A15 (SSE), A19 (caching), A20 (security headers).
## Objective
`GET /api/docs/tree` stops shipping 2 MB: the first load carries what the RAG view actually renders (sources + one level, with server-computed totals), deeper levels load on drill, every response the client already has revalidates to a 304, and **every** non-SSE response ships gzipped (~8-12× on this JSON). Then the same lens is pointed at the rest of the API with a byte-budget log line and a measured audit, so "any other endpoints that could load a ton of data" becomes a recorded finding with fixes for the offenders instead of a guess.
## Why it is 2 MB (traced 2026-10-08)
`list_kb_tree` returns the FULL recursive tree in one body (the deliberate phase-97 "the RAG view drills CLIENT-side, zero per-level requests" contract): every indexed document as a file node (`path`, `title`, `chunks`, `created_at`, `indexed_at`, + `is_image`/`image_url`/`summary` on image nodes) AND **every stored `folder_summaries` row inlined on its folder/source node**, with no compression anywhere in the stack (the app has no GZip middleware — `app/main.py` adds only Session / caching / security-headers; nothing sets `Content-Encoding`). At the live corpus (≈1 k documents, hundreds of folders with AI descriptions, image docs whose `summary` is a full vision description) that is ~2 MB of uncompressed JSON per Sources-page load, and the browser re-downloads it after every sync and on every shell boot.
## Design (shared by all tasks — the executor reads this, not the chat)
**D1 — selective gzip, SSE never touched (task 01, LOCKED A1).** New `app/core/compression.py`, a **pure-ASGI** middleware (`BaseHTTPMiddleware` is forbidden here — it buffers):
```python
class SelectiveGZipMiddleware:
def __init__(self, app, *, minimum_size: int, skip_paths: tuple[str, ...] = ("/api/chat",)) -> None:
self.app = app
self.gzip = GZipMiddleware(app, minimum_size=minimum_size) # starlette, reused, not forked
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or scope["path"] in self.skip_paths:
await self.app(scope, receive, send); return
await self.gzip(scope, receive, send)
```
Exact-path skip (never a prefix — `/api/chat-images` must still compress). Starlette's `GZipMiddleware` already: requires `Accept-Encoding: gzip`, respects `minimum_size`, skips a response that already sets `content-encoding`, appends `Vary: Accept-Encoding`, and streams with `compressobj` + a final flush. `app/main.py`: new `Settings.compress_min_bytes: int = 860` (`BOR_COMPRESS_MIN_BYTES`, `0` = the feature OFF (skip the middleware entirely), negative → fail startup loudly — the `agent_max_rounds` pattern, PLAN §13), wired by `app.add_middleware(...)` **before** the `SecurityHeadersMiddleware` line so SecurityHeaders stays outermost (A20) and gzip sits outside the caching layer (an already-rewritten HTML page compresses; `/api/*` still passes the caching middleware byte-identically). No production nginx change is required; if nginx also gzips, it will not double-encode an already-encoded body.
**D2 — a depth-limited tree + server totals (task 02, LOCKED A2).** `app/api/docs.py`:
- `build_kb_tree(..., depth: int | None = None)` and `_level_children(..., depth)` — `None` = unlimited (the legacy behavior every existing unit test keeps, byte-identical); a positive budget stops the recursion. A folder node whose own children were NOT built carries `truncated=True`, `children=[]`, its recursive `documents` from `counts[sub]` (unchanged) and — the correctness trap — its `updated_at` from a NEW recursive max-date map computed in `_folder_counts` (because `_subtree_max(children)` over an empty child list would be `None`, which would silently break the phase-106 D9 column). Its `summary` still rides the node (the level row shows a subfolder's description).
- `GET /api/docs/tree` gains `depth: int = Query(1, ge=1, le=8)`. **The default is what breaks the existing pins:** the full consumer list is exactly nine files (`grep -rl "docs/tree" tests app` → `tests/integration/test_docs_api.py`, `tests/integration/test_docs_api_dates.py`, `tests/unit/test_kb_tree_builder.py`, `tests/unit/test_kb_tree_ui.py`, `tests/unit/test_frontend_router.py`, `tests/e2e/test_kb_tree.py`, `tests/e2e/test_navbar_refresh.py`, `tests/e2e/test_document_dates.py`, `tests/e2e/test_git_source_dates.py`), and the TWO DATE SUITES walk straight through the tree with no depth param (`test_docs_api_dates.py` reaches `a` → `a/b` → deeper; `test_git_source_dates.py::_find_file` recurses by design) — they assert phase-106 D9 dates, not payload shape, so they migrate by requesting `?depth=8` explicitly with every assertion kept (task 02 item 9; the E2E one is named in task 06 item 4).
- `KbTree` gains **`totals: KbTreeTotals`** (`documents`, `chunks`, `last_indexed: str | None`) computed in the endpoint from the SAME catalogue rows (no extra query) — `sources.js::treeStats()` cannot sum a truncated tree client-side (it walks children) and the three stat cards must stay honest.
- `truncated: bool = False` on `KbTreeFolder`/`KbTreeSource` with a `@model_serializer(mode="wrap")` omission when `False` (the `_drop_image_fields` precedent): an untruncated node's wire shape stays byte-identical, and only the depth-limited nodes add the key.
**D3 — the subtree route + revalidation (task 03, LOCKED A3/A4).**
- `GET /api/docs/tree/node?source=<name>&path=<folder>&depth=1` (admin-gated like `/api/docs/tree`, `path=""` = the source root): one node in the `KbTreeFolder`/`KbTreeSource` shape (its direct children + `truncated` flags), 404 `{"detail": "unknown node"}` for an unregistered+unindexed source or a folder that fails the phase-94 existence rule. The RAG view's drill target; the catalogue query is the same one `GET /api/docs/tree` runs (documented: one query per drill, no N+1).
- Revalidation, both tree routes (and `GET /api/docs`): build → serialize once (`model_dump_json()` / json) → `ETag: W/"<sha256(body)[:32]>"` + `Cache-Control: private, must-revalidate`; `If-None-Match` match (weak/strong and `*` tolerant) → `Response(304)` with the same two headers. Helpers live in a new `app/core/http_cache.py` (`weak_etag(body)`, `etag_matches(header, tag)`) so the pattern is one function, not a copy. **Why not `sources_meta.version`** (LOCKED A3): bumping that generation marks every saved chat `stale` (phase 53) — the wrong tool — and the admin PATCHes (`/api/documents/summary`, `/api/documents/date`, `/api/folders/summary`) change the tree WITHOUT bumping it, so a version-derived tag would serve stale trees; a content hash is exact, needs no migration, and the tree is built anyway. This is a deliberate, recorded extension of A19 (which governs HTML pages + `/assets/*` + the caching pass-through) — `/api/*` gains a validator on two catalogue routes only; the SSE stream and every mutating route are untouched.
**D4 — the client drills lazily (task 04).** `frontend/assets/sources.js`: `loadTree()` → `fetch("/api/docs/tree?depth=1")` (still ONE first load), a `levelCache` (Map keyed `source + "/" + folder`), and a drill that fetches `GET /api/docs/tree/node?…` **only** when the target node reports `truncated === true` and is not cached — merged into the cached node before `renderLevel()`. Never-stale (§7.4): the drill's in-flight state disables the row's link + shows one "Loading…" row (the `.kb-*` empty/loading idiom), a failure shows a `role="alert"` line with a Retry affordance and leaves the current level rendered, and the race token (`loadSeq`) extends to the node fetch (a stale drill response may never paint). `folderExistsIn`/`resetVanishedLocation` stop walking a full tree: the node route's 404 IS the "the folder vanished" signal → reset to the top level. `treeStats()` reads `kbTree.totals` (fallback to the existing walk when `totals` is absent — the defensive no-crash path).
**D5 — the byte budget and the audit (task 05).** New `app/core/response_budget.py`, pure ASGI, one log line per `/api/*` response in the §9 single-line style:
`api method=GET path=/api/docs/tree status=200 bytes=204511 enc=gzip duration_ms=41`
It counts `http.response.body` chunk bytes as they pass (zero buffering, SSE-safe: `duration_ms` for a stream is the stream's life, documented), logs at INFO when `bytes >= settings.log_large_response_bytes` (`BOR_LOG_LARGE_RESPONSE_BYTES`, default 65536, `0` = log every `/api/*` response, negative fails startup) and at DEBUG otherwise. `GET /api/chat` gets a `stream=1` variant of the line. New `scripts/api_payload_audit.py` (`python -m scripts.api_payload_audit --base-url … --user … --password …`): logs in, hits the candidate endpoints (`/api/docs/tree` at depth 1 and unlimited, `/api/docs`, `/api/documents/content` for the largest doc, `/api/chats`, `/api/chats/{id}` for the largest chat, `/api/git-sources`, `/api/suggestions`, `/api/sync/status`, `/api/tokens`, `/api/config`), and prints uncompressed AND gzip bytes + duration per endpoint with a stated budget (256 KB uncompressed = "look at it", 64 KB = "fine"). Findings land in this directory as `AUDIT.md` (endpoint, before/after bytes, the remedy applied or the accepted-why) — the TODO's "any other endpoints" answered with numbers, not vibes. The audit runs against the local 1,000-doc live-replica snapshot (the `restore-test-db` skill) so the numbers are the production order of magnitude, not the fixture KB's.
**D6 — verification (task 06).** New `tests/e2e/test_kb_tree_lazy_load.py`: the Sources page's FIRST `/api/docs/tree` request carries `depth=1` + `totals` and no deep children (route-level byte assertion + `via response.json()` node walk), drilling two levels deep issues the node fetch (exactly once per level, cached the second time), the stat cards match `totals` at every level, a 404 node (folder pruned under us — a `page.route` stub) resets to the top level with no stale breadcrumb, `Accept-Encoding: gzip` yields `Content-Encoding: gzip` + a `Vary` header, and the second visit revalidates to a 304 (`page.route` observes the 304 + the body unchanged from cache). Plus the migrated pins: `tests/unit/test_kb_tree_ui.py` (asserts exactly ONE `fetch("/api/docs/tree")` occurrence), `tests/unit/test_frontend_router.py:591`, `tests/e2e/test_navbar_refresh.py:115` (the `#nav-sources` → `/api/docs/tree` expectation), and the phase-97/99 integration shape tests.
## Dependencies
- `97_kb_tree_catalog` + `99_kb_tree_table_and_back_nav` (complete) — the one-fetch/drill-client-side contract this phase **deliberately supersedes** (owner-confirmed 2026-10-07, TODO L5); `106_document_dates` (D9: a truncated node's `updated_at` must stay the recursive max — D2's trap), `98_sync_summary_visibility` (D3 `summary_pending` on every node, including truncated ones), `122_image_documents` (task 04's image map, which stays a bounded select), `156_table_ux_polish` (the bounded scroll panel + sticky header the drill re-renders into), `82_security_headers` / A19 / A20 (the middleware order this phase must not disturb), A15 (SSE).
- No migration: nothing in the DB changes.
## Tasks
1. `01_gzip_middleware.md` — `app/core/compression.py` + `Settings.compress_min_bytes` + wiring + the compression/SSE/passthrough tests.
2. `02_tree_depth_and_totals.md` — `KbTreeTotals`, the `truncated` omission serializer, `build_kb_tree(depth=…)`, the recursive max-date map, `depth` on `/api/docs/tree`, builder + integration tests.
3. `03_tree_node_endpoint_etag.md` — `app/core/http_cache.py`, `GET /api/docs/tree/node`, ETag/304 on the two tree routes + `/api/docs`, integration matrix.
4. `04_client_lazy_drill.md` — `sources.js` level cache + per-level fetch + §7.4 drill states + `totals` stat cards + the migrated unit pins.
5. `05_response_budget_audit.md` — `app/core/response_budget.py` + `scripts/api_payload_audit.py` + `AUDIT.md` with the measured numbers + the remedies for the offenders (and the accepted-why for the rest) + README/env docs.
6. `06_e2e_payload_diet.md` — `tests/e2e/test_kb_tree_lazy_load.py`, the migrated E2E pins, the full gate set, the commit.
## Testing & Quality
- Unit: the builder depth/truncation/max-date matrix (`tests/unit/test_kb_tree_builder.py`), the schema omission wire-shape pins, `tests/unit/test_caching.py` extended (the `/api/*` pass-through survives gzip coexistence), config validator pins, `tests/unit/test_kb_tree_ui.py` extended for the drill (cache, single fetch per level, race token, loading/alert states, `totals` stat cards).
- Integration: depth + truncated + totals + node-route matrix (200/404/403), the migrated `tests/integration/test_docs_api_dates.py` (explicit `?depth=8`, every D9 date assertion intact), the ETag matrix (no header → 200 + `ETag`; match → 304 no body; mismatch → 200; `*`; weak/strong equivalence; a folder-summary PATCH changes the tag), gzip (`Content-Encoding: gzip`, `Vary`, under-`minimum_size` identity, `Accept-Encoding: identity` identity, the SSE stream NOT compressed and arriving frame-by-frame), and the response-budget log line (caplog: INFO over the threshold, DEBUG under, `stream=1` for `/api/chat`).
- E2E: `tests/e2e/test_kb_tree_lazy_load.py` green **in isolation**; the owning regression set green in isolation: `test_kb_tree.py` (migrated), `test_kb_tree_nav.py`, `test_navbar_refresh.py`, `test_document_dates.py`, `test_git_source_dates.py` (its `_find_file` recurses — it needs the explicit `?depth=8`), `test_sync_summary_visibility.py`, `test_table_ux_polish.py`, `test_sources_midstream_bug.py`, `test_ls_tree_drilldown.py` (the agent's `ls` view must stay byte-identical), `test_smoke.py` (the no-CDN integration test stays green).
- Coverage **>90%** on `app/` (the new `compression.py` / `http_cache.py` / `response_budget.py` are small and fully pinned).
## Completion Criteria
- [ ] `GET /api/docs/tree` (default `depth=1`) + `Content-Encoding: gzip` transfers < 40 KB on the audited live-replica corpus (measured by `scripts/api_payload_audit.py`, recorded in `AUDIT.md`) — from ~2 MB — and a repeat load answers **304** with an empty body.
- [ ] Drilling two folders deep renders correct rows (counts, dates, descriptions, thumbnails, pending markers) with exactly one node fetch per uncached level; a vanished folder resets honestly.
- [ ] The stat cards are right at every level (server `totals`), and the tree's per-node `updated_at` is still the recursive max even when the node's children are truncated.
- [ ] `POST /api/chat` streams uncompressed and frame-by-frame (unit + E2E proof); HTML pages still serve `no-cache` with the `?v=` rewrite; every response still carries the A20 headers.
- [ ] `AUDIT.md` lists every candidate endpoint with before/after bytes and its verdict; offenders over the stated budget are fixed or explicitly accepted with a reason.
- [ ] `uv run pytest` green, coverage > 90%, `uv run ruff check . && uv run pyright` clean, the phase E2E green in isolation, one `--no-gpg-sign` commit, phase dir in `complete/`.
## Locked decisions
- **A1 — gzip is selective and pure-ASGI**: `POST /api/chat` (SSE, A15) is skipped by EXACT path; no `BaseHTTPMiddleware`; `BOR_COMPRESS_MIN_BYTES` (`0` = off, negative = fail loudly) — owner-confirmed 2026-10-07 ("gzip + ETag + lazy drill").
- **A2 — `depth` (default 1, max 8) + `totals` on the tree, `truncated` omitted when false**: the depth-limited response keeps every per-node field the view renders (counts, `updated_at`, `summary`, `summary_pending`, image keys) and only omits un-rendered DEPTH; the recursive max-date map is mandatory so phase 106 D9 does not silently regress — owner-confirmed 2026-10-07.
- **A3 — the phase-97 "ONE fetch, drill client-side" contract is superseded** by one initial load + one fetch per drilled level (the deliberate cost of a small first payload; the drill UX, breadcrumb, and scroll-panel behavior stay as phases 97/99/156 built them) — owner-confirmed 2026-10-07.
- **A4 — the ETag is a weak hash of the response body, NOT `sources_meta.version`**: bumping the generation would mark every saved chat `stale` (phase 53), and the admin summary/date/folder PATCHes mutate the tree without bumping it. Recorded as a scoped extension of A19 (validators on two read-only catalogue routes; SSE and mutations untouched) — owner-confirmed 2026-10-07.
- **A5 — the byte budget is a log line + a script, not a hard 413**: no endpoint starts rejecting large payloads this phase (the app's own documents are legitimately large); the audit's 256 KB "look at it" threshold drives follow-up work, recorded in `AUDIT.md`.
## Commit
```bash
git add app/ frontend/ scripts/ tests/ README.md .env.example .agents/phases/ && git commit --no-gpg-sign -m "perf(api): gzip non-SSE responses, a depth-limited KB tree with per-level drill and ETag revalidation, and a measured response byte budget"
```
@@ -0,0 +1,33 @@
# Task 01 — Selective gzip: `app/core/compression.py`, `BOR_COMPRESS_MIN_BYTES`, SSE untouched (phase 167 D1, LOCKED A1)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "GET <https://brain.reeseapps.com/api/docs/tree> loads 2 MB in production which causes long loading times."
**Story:** n/a
## Objective
Every HTTP response except the `POST /api/chat` SSE stream compresses with gzip, with a config kill switch, the existing middleware order and contracts (A19 caching pass-through, A20 security headers outermost) untouched, and a test that proves the chat stream still arrives frame-by-frame and uncompressed.
## Work
1. New `app/core/compression.py` — the middleware of phase 167 D1, **pure ASGI** (never `BaseHTTPMiddleware`, which buffers and would break streaming responses):
- `class SelectiveGZipMiddleware` holding `self.app` and a composed `starlette.middleware.gzip.GZipMiddleware(self.app, minimum_size=minimum_size)`; `__call__` delegates to the gzip path unless `scope["type"] != "http"` or `scope["path"] in skip_paths` (EXACT match — `/api/chat-images` must still compress), in which case it calls `self.app` directly;
- `DEFAULT_SKIP_PATHS = ("/api/chat",)` module constant, docstring names A15 (SSE) and the buffering reason; do NOT re-implement compression math — Starlette's `GZipMiddleware` owns `Accept-Encoding` negotiation, `minimum_size`, the already-`content-encoding` skip, the `Vary: Accept-Encoding` append, and the `compressobj` streaming + final flush;
- module docstring: the TODO's 2 MB defect, the ~8-12× JSON expectation, the SSE exclusion, and "nginx may also gzip; an already-encoded body is never double-encoded".
2. `app/config.py` — `compress_min_bytes: int = 860` (`BOR_COMPRESS_MIN_BYTES`) beside the other ints, with the `agent_max_rounds` validator pattern (`_compress_min_bytes_non_negative`: negative → `ValueError("compress_min_bytes must be >= 0 (0 = compression disabled)")`) and the docstring stating `0` = the feature is OFF (no middleware), `> 0` = the Starlette minimum-size floor (860 = the classic nginx default, below which compression can inflate).
3. `app/main.py` — wire it, added **before** the `app.add_middleware(SecurityHeadersMiddleware)` line so SecurityHeaders stays outermost (A20, the existing comment says "last add_middleware"): construct `SelectiveGZipMiddleware` only when `settings.compress_min_bytes > 0` (the `0` = OFF path adds nothing at all), with a comment naming TODO L5, the SSE skip, and the position (outside `configure_caching`, inside SecurityHeaders → the rewritten HTML pages compress too, `/api/*` still passes the caching layer byte-identically before compression).
4. `.env.example` — `BOR_COMPRESS_MIN_BYTES=860` with the three-value comment (0 off, negative refuses startup, the 860 rationale).
5. New `tests/unit/test_compression.py`:
- a tiny app with a ~50 KB JSON route + a ~50 byte route, wrapped in the middleware: `Accept-Encoding: gzip` → `Content-Encoding: gzip` + a body that `gzip.decompress`es to the original + `Vary` mentions `accept-encoding`; `Accept-Encoding: identity` → no `Content-Encoding`; the small route → identity (under the floor); a response that already set `content-encoding` → untouched;
- `skip_paths`: a `/api/chat`-named route in the test app is NOT compressed and streams incrementally (assert the received chunks arrive before the response completes — the `list(stream)` idiom the existing SSE tests use), while a `/api/chat-images/x` route IS compressed (the exact-match pin);
- `scope["type"] == "lifespan"`/`"websocket"` pass-through;
- config: `compress_min_bytes = 0` → `create_app` adds no gzip layer (assert via `app.user_middleware`'s classes), `-1` → the validator raises naming the field.
6. Integration pin `tests/integration/test_docs_api.py` (or the middleware-owning integration file — locate via `SecurityHeadersMiddleware` in `tests/`): the real app's `GET /api/health`-class small response stays identity; a large `/api/docs/tree` (seed ≥ 50 docs) returns `Content-Encoding: gzip` and decompresses to the documented JSON; the A20 headers (`CSP`, `X-Content-Type-Options`, `X-Frame-Options`/`frame-ancestors`) are STILL present on a gzipped response (the outer-layer invariant); the HTML page contract (A19: `no-cache`, `?v=` rewrite, no validators) survives decompression.
7. ASSUMPTION: `Settings.compress_min_bytes` (name + default 860) — the TODO named no knob; the house kill-switch convention (`0` off, negative loud) is what makes it owner-controllable without a code change.
## Testing & Quality
- Unit + integration as above; `uv run pytest tests/integration/test_chat_api.py -v` (or the suite that owns the SSE contract) stays green — the stream is not compressed and not buffered.
- Coverage: **>90%** on `app/core/compression.py` (every branch listed above is pinned).
## Completion Criteria
- [ ] `curl -H 'Accept-Encoding: gzip' …/api/docs/tree | wc -c` beats the identity size by an order of magnitude on any corpus above the floor; `Vary: Accept-Encoding` is present.
- [ ] `POST /api/chat` responses carry NO `Content-Encoding` and stream frame-by-frame (test-proven), and `/api/chat-images` still compresses.
- [ ] `BOR_COMPRESS_MIN_BYTES=0` removes the middleware entirely; `-1` refuses startup naming the field.
- [ ] A20 headers, A19 page/asset caching, and the `/api/*` pass-through are unchanged; `uv run pytest` green; ruff + pyright clean.
@@ -0,0 +1,31 @@
# Task 02 — Depth-limited tree + server totals: `truncated`, the recursive max-date map, `KbTreeTotals` (phase 167 D2, LOCKED A2)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "GET …/api/docs/tree loads 2 MB in production which causes long loading times."
**Story:** n/a
## Objective
The pure builder can stop at a depth budget: a node whose children were not built says so (`truncated: true`) while keeping every field the Sources view renders (recursive count, recursive `updated_at`, its own `summary`, `summary_pending`, the image affordance), the response gains the `totals` the stat cards need, and `GET /api/docs/tree` defaults to `depth=1`.
## Work
1. `app/schemas.py` — `KbTreeTotals(BaseModel)`: `documents: int = Field(ge=0)`, `chunks: int = Field(ge=0)`, `last_indexed: str | None = None` (docstring: the catalogue-wide totals the RAG view's three stat cards read, because a depth-limited tree cannot be summed client-side — the phase-97 `treeStats()` walk needed the whole tree).
2. `app/schemas.py` — `KbTree` gains `totals: KbTreeTotals = KbTreeTotals()` (docstring: additive top-level key, TODO L5) and `KbTreeFolder` / `KbTreeSource` gain `truncated: bool = False` with an OMIT-WHEN-FALSE serializer. Follow the file's existing omission idiom (`_drop_image_fields` + `@model_serializer(mode="wrap")`): add the `truncated` key only when `True`, so an untruncated node's wire shape is byte-identical to today (every existing wire pin stays green). `KbTreeFile` never truncates (a file has no children).
3. `app/api/docs.py::_folder_counts` — extend the return to `(folders, counts, max_dates)` where `max_dates[folder]` is the folder's RECURSIVE MAX document `created_at` (the same ISO-8601 strings, so the lexicographic max is the chronological one — the phase-106 D9 rule), computed from the rows in the same pass. Rationale in the docstring: a truncated node has NO built children, so `_subtree_max(children)` would read `None` and silently lose the D9 column; the aggregate map is the truncation-proof source. Keep `_subtree_max` for the expanded path UNCHANGED and add a unit test asserting the two agree for every expanded folder (one concept, two derivations, cross-pinned).
4. `app/api/docs.py::_level_children` — add `depth: int | None`; per subfolder: when `depth is None` or `depth > 1`, build the node as today with the recursion at `depth - 1`; when the budget is exhausted, build `KbTreeFolder(path=sub, documents=counts[sub], updated_at=max_dates[sub], summary=summaries.get((source, sub)), summary_pending=<unchanged rule>, children=[], truncated=True)`. Update the docstring with the depth rule + the truncated-node field set (the level row renders a subfolder's name, count, date and description WITHOUT its contents).
5. `app/api/docs.py::build_kb_tree` — thread `depth` (`None` = unlimited: every existing unit call and every existing expectation is byte-identical); `_source_node` passes the budget down and never marks the SOURCE node itself truncated at `depth ≥ 1` (the source's direct children are always built — the top level of the RAG view is source → level 1).
6. `app/api/docs.py::list_kb_tree` — `depth: int = Query(1, ge=1, le=8)` (docstring: 1 = sources + one level, the view's first paint; 8 = the practical KB ceiling; the endpoint computes `totals` from the catalogue rows it already fetched: `documents = len(rows)`, `chunks = sum(chunks)`, `last_indexed = max(indexed_at)` → `None` for an empty KB) and passes it to the builder. `GET /api/docs` and the agent's `ls` path are untouched (pin it: `app/rag/agent.py` imports are unchanged).
7. `tests/unit/test_kb_tree_builder.py` (extend): depth `None` vs `1` vs `2` vs deep on the multi-level fixture — the truncated node's exact field set (count recursive, `updated_at` = the deep max, summary present, `summary_pending` present, `children == []`, `truncated is True`), the totals math incl. the empty KB (`last_indexed is None`), the `_subtree_max`/`max_dates` cross-check on every expanded folder, and the phase-94/98 invariants (existence rule, `missing_folder_summaries` cross-check) at each depth.
8. `tests/integration/test_docs_api.py` (extend): `GET /api/docs/tree` default → depth 1 (deep children absent, `truncated` present on a subfolder that has them, `totals` correct vs `GET /api/docs`), `?depth=3` → deeper, `?depth=0` / `?depth=9` / `?depth=abc` → 422, anonymous → 403 unchanged, and the wire-shape pin: an UNtruncated node's JSON keys are exactly the pre-phase set (no `truncated` key).
9. Migrate the TWO date suites that consume the tree with no depth param (they break on the new default — the phase changes the default, so it owns the repair; every assertion stays, only the URL gains `?depth=8`):
- `tests/integration/test_docs_api_dates.py` (~L157): `admin_client.get("/api/docs/tree")` → `"/api/docs/tree?depth=8"`, keeping the whole `a` → `a/b` → deeper D9 walk (deep date beats shallow at every level) and the 0-document-source `updated_at is None` case; add a one-line comment naming phase 167 (the default is the view's first paint, `depth=8` is the whole tree).
- `tests/e2e/test_git_source_dates.py` (~L419): the `page.request.get(f"{app_url}/api/docs/tree")` call gains `?depth=8` so `_find_file` (which recurses by design) still reaches `old/…` and `recent/…`; the folder `updated_at` date assertions stay verbatim.
- Add the truncation-proof counterpart so the guarantee is tested BOTH ways: at the DEFAULT depth, a truncated folder node's `updated_at` still equals the deep max (this is exactly what the task-02 `max_dates` map buys — the date contract survives the payload diet), pinned in `test_docs_api_dates.py` and in `tests/unit/test_kb_tree_builder.py`.
## Testing & Quality
- Unit + integration above; `uv run pytest tests/unit/test_kb_tree_builder.py tests/integration/test_docs_api.py -v` green.
- Coverage **>90%** on the modified `app/api/docs.py` / `app/schemas.py` lines.
## Completion Criteria
- [ ] Default `/api/docs/tree` returns sources + ONE level with correct `totals`, and every node that has unloaded children carries `truncated: true`.
- [ ] A truncated node still carries its recursive count, its recursive `updated_at`, its `summary`, and `summary_pending` — phase 98/106 semantics intact at every depth (unit + integration).
- [ ] `depth=None` (builder default) reproduces today's tree byte-for-byte; untruncated nodes' wire shape is unchanged.
- [ ] `uv run pytest` green; coverage > 90%; ruff + pyright clean.
@@ -0,0 +1,35 @@
# Task 03 — The subtree route and revalidation: `GET /api/docs/tree/node`, `app/core/http_cache.py`, ETag/304 (phase 167 D3, LOCKED A3/A4)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "Fix that any any other endpoints that could load a ton of data."
**Story:** n/a
## Objective
One small admin-gated route serves a single tree node's direct children (the drill's fetch target), and the catalogue read routes (`/api/docs/tree`, `/api/docs/tree/node`, `/api/docs`) carry a content-exact weak `ETag` + `Cache-Control: private, must-revalidate` so a repeat load costs a 304 with no body — without touching `sources_meta` (which would mark saved chats stale) or the SSE/mutating surface.
## Work
1. New `app/core/http_cache.py` (small, total, no I/O):
- `weak_etag(body: str | bytes) -> str` → `W/"<sha256 hexdigest[:32]>"` (docstring: content-exact by construction; 32 hex chars is the file's truncation convention and collisions are not a correctness risk for a revalidated cache — a false match only ever costs one stale repaint on a 2 MB namespace, and the tag is over the EXACT served bytes);
- `etag_matches(header: str | None, tag: str) -> bool` → tolerant of `*`, a comma-separated list, and weak/strong equivalence (`W/"x"` matches `"x"` both directions), whitespace-tolerant, `None`/empty → `False`;
- `revalidate_headers(etag: str) -> dict[str, str]` → `{"ETag": etag, "Cache-Control": "private, must-revalidate"}` (ONE place for the pair; docstring names A19's scope — HTML pages stay `no-cache` with no validators, `/assets/*` stay immutable, this is the read-only API arm).
2. `app/api/docs.py` — `GET /api/docs/tree/node`:
- signature: `source: str = Query(...)`, `path: str = Query("")` (source-relative; `""` = the source root node), `depth: int = Query(1, ge=1, le=8)`, `db=Depends(get_db)`, `_admin=Depends(require_admin)` (`403 {"detail": "admin only"}` — the same gate as `/api/docs/tree`), `request: Request`;
- composition: the SAME catalogue query as `list_kb_tree` (one query, no N+1 — the file's docstring paragraph says so), the same `folder_summaries` + image-docs bounded selects, then build the single node through the pure helpers: resolve the source (registry name or indexed source), `_folder_counts(rows)` for that source, `_level_children(source, path, folders, counts, rows, summaries, images, depth=depth)`; the node is a `KbTreeSource` when `path == ""` else a `KbTreeFolder` with `documents=counts[path]`, `updated_at=max_dates[path]`, `summary`, `summary_pending` (the phase-98 D3 rule), `truncated` per the depth budget;
- 404 `{"detail": "unknown node"}` when: the source is neither registered nor indexed, OR the folder fails the phase-94 existence rule (`path` not in `folders` and `path != ""`). Never a filesystem path built from `path` (docstring: the row-only rule, the phase-10 traversal argument reused);
- `response_model=None` on the decorator with the union return annotated, and the body returned as a raw `Response` (below) so the serializer is not run twice.
3. `app/api/docs.py` — ETag wiring on the three read routes (`/api/docs/tree`, `/api/docs/tree/node`, `/api/docs`): build the model → `body = model.model_dump_json()` (for `/api/docs`: `DocList.model_dump_json()`) → `etag = weak_etag(body)` → `if etag_matches(request.headers.get("if-none-match"), etag): return Response(status_code=304, headers=revalidate_headers(etag))` → else `Response(content=body, media_type="application/json", headers=revalidate_headers(etag))`. Docstring per route: the client is `fetch` (a 304 is served from its cache transparently), and `must-revalidate` guarantees a sync or an admin edit is seen on the next load — the tag changes because the body changes. Keep `response_model=KbTree`/`DocList` on the decorators for the OpenAPI schema (a returned `Response` short-circuits serialization — note that in the docstring).
- ASSUMPTION: `GET /api/documents/content` is NOT given a validator this phase (one document's body, on demand, already compressed; the audit in task 05 decides whether it needs more) — recorded so the scope is explicit.
4. `tests/integration/test_docs_api.py` (extend):
- the node route: 200 for a source root (`path=""`, the `KbTreeSource` shape), 200 for a nested folder (correct `documents`/`updated_at`/`summary`/`summary_pending`/`children` at `depth=1` with `truncated` on the deeper subfolders), `?depth=2` deeper, 404 `unknown node` for an unknown source / a nonexistent folder / a `path` that is a FILE's path, 403 anonymous, and a cross-check that the node's children equal the same level of the unlimited `/api/docs/tree?depth=8` response (ONE concept, two routes, pinned equal);
- ETag matrix on all three routes: first GET → 200 + `ETag` + `Cache-Control: private, must-revalidate`; re-send with `If-None-Match` → 304 + empty body + both headers; a wrong tag → 200; a `W/`-vs-strong mismatch pair → 304; `*` → 304; after `PATCH /api/folders/summary` (and after `PATCH /api/documents/date`) the tag CHANGES and the next GET is a 200 with the new tree (the A4 proof — a version-derived tag would have failed exactly here);
- gzip coexistence: a gzipped 200 still carries the `ETag` (the header survives the compression layer) and a conditional request returns a body-less 304.
5. `tests/unit/test_http_cache.py` (new): `weak_etag` stability/length/prefix (`W/"…"`), bytes-vs-str equality, `etag_matches` for `None`, empty, `*`, single, comma-list, weak↔strong both directions, whitespace, and a tag that merely shares a prefix → `False`.
## Testing & Quality
- Integration + unit above; `uv run pytest tests/integration/test_docs_api.py tests/unit/test_http_cache.py -v` green; `tests/integration/test_caching.py` stays green (the `/api/*` pass-through and the HTML rewrite are untouched by a route-level header).
- Coverage **>90%** on `app/core/http_cache.py` and the new route lines.
## Completion Criteria
- [ ] `GET /api/docs/tree/node?source=X&path=Y` returns exactly that folder's direct children with the same per-node fields the full tree produces (equality pinned against `depth=8`).
- [ ] A repeat request with `If-None-Match` answers **304, no body**; a folder-summary or document-date edit changes the tag and forces the next 200.
- [ ] Anonymous gets `403 {"detail":"admin only"}`; an unknown source/folder gets `404 {"detail":"unknown node"}`; nothing builds a filesystem path from `path`.
- [ ] `uv run pytest` green; coverage > 90%; ruff + pyright clean.
@@ -0,0 +1,35 @@
# Task 04 — The client drills lazily: level cache, per-level fetch, §7.4 drill states, `totals` stat cards (phase 167 D4, LOCKED A3)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "GET …/api/docs/tree loads 2 MB in production which causes long loading times."
**Story:** n/a
## Objective
The RAG view loads one small tree, fetches a folder's contents the first time it is opened, caches it, keeps the breadcrumb/scroll-panel/editing behavior identical, and never shows a stale or half-updated level — the UX cost of A3 paid in full so the user cannot tell the tree is no longer one payload.
## Work
1. `frontend/assets/sources.js::loadTree()` (~L1625) — `fetch("/api/docs/tree?depth=1")`; on success: store the tree, CLEAR `levelCache` (a refresh invalidates every drilled level — the sync/refresh call sites at ~L1258, ~L1386, ~L2192, ~L2213 keep working unchanged), then `resetVanishedLocation()` + `renderLevel()` as today. Keep the `loadSeq` race token and the `renderEmpty()` failure path.
2. `frontend/assets/sources.js` — new module state + the drill:
- `const levelCache = new Map();` keyed `source + "/" + folder` (a node's `path`), populated ONLY by a node fetch;
- `function needsLevelFetch(node) { return node.truncated === true && !levelCache.has(key); }`;
- `enterFolder(source, folder)` (the folder-row click path, today a pure `current = …; renderLevel()`): when the target needs a fetch → set the in-flight state (`drillPending = {source, folder}`, the row link gets `aria-busy="true"` + `aria-disabled="true"` so a double click cannot start a second fetch), `await fetch("/api/docs/tree/node?source=…&path=…")`, merge the result (`Object.assign(targetNode, node)` + `levelCache.set(key, true)`), clear the flags, then `current = …; renderLevel()`. A stale response (token changed) never paints (the existing `my === loadSeq` convention extended to the drill).
- In-flight rendering: render the level the user is LEAVING unchanged and show one loading row in the folders panel using the file's existing status/empty idiom (a `role="status"` `.kb-level-note`-style line "Loading <folder>…" — reuse an existing class if one fits, otherwise add ONE `.kb-level-loading` rule to `styles.css` with tokens only, no animation beyond the existing house spinner if one exists);
- failure (non-OK or a network throw): `role="alert"` line "Couldn't load <folder> — Retry." with a real `<button type="button">Retry</button>` that re-runs the same fetch, and the CURRENT level stays rendered (never blank, never a stale breadcrumb — PLAN §7.4 "every failure path re-enables its controls").
3. `frontend/assets/sources.js` — the existence rules stop walking a full tree:
- `folderExistsIn(src, folderPath)` (~L1667) is replaced by `folderKnownIn(src, folderPath)`: true when the path's whole ancestor chain is present in what the client holds (the loaded tree + cached levels), so `resetVanishedLocation()` can still answer after a re-fetch WITHOUT assuming a full tree; when a drill fetch answers `404` the folder is gone → `current = { source: null, folder: null }` + `renderLevel()` (the honest reset, the phase-97 "never stale" contract carried over), and the failing `levelCache` entry is not cached;
- `currentLevelNode()` (~L1680) keeps walking, but through cached nodes (unchanged semantics for loaded levels).
4. `frontend/assets/sources.js::treeStats()` (~L1703) — read `kbTree.totals` (`documents`, `chunks`, `last_indexed`) when present, and keep the existing recursive walk ONLY as the defensive fallback when `totals` is absent (a tree served by a pre-phase process); the three stat cards (`statDocs`, `statChunks`, `statLast`) render identically at every level (pin the exact formatting call — `fmtDate(st.last)` — unchanged).
5. `frontend/assets/sources.js` docstring (the module header, ~L55-103) — rewrite the "ONE fetch of GET /api/docs/tree returns the whole tree" paragraph for the depth-1 + per-level model, citing TODO L5 and phase 167, and keep the phase-79 race-token, phase-90 upload and phase-98 pending narratives accurate.
6. `tests/unit/test_kb_tree_ui.py` (MIGRATE — it currently asserts `js.count('fetch("/api/docs/tree")') == 1` at ~L250 and the "ONE fetch" contract at ~L9/L27): the fetch assertion becomes `fetch("/api/docs/tree?depth=1")` exactly once + `fetch("/api/docs/tree/node?` exactly once, and the "drills client-side" docstring assertions become the level-cache assertions: `levelCache` declared once, cleared inside `loadTree`, consulted by `needsLevelFetch`, written only after a 200; the 404 → reset branch exists; `drillPending` guards a second click; the Retry button is a real `<button>`; `treeStats` prefers `totals`. Preserve every OTHER phase-97/99 pin (breadcrumb, sticky header, scroll panel, editor reset, empty state).
7. `tests/unit/test_frontend_router.py:591` — the shell-route pin (`'fetch("/api/docs/tree")' in body`) moves to the `?depth=1` URL, keeping its original intent (the Sources view loads the catalogue from the tree endpoint, not the flat one).
## Testing & Quality
- Unit: the migrated + new pins above; run the owning set for collateral: `uv run pytest tests/unit -k "kb_tree or frontend_router or table_ux or source" -v`.
- No `app/` change → coverage gate unmoved (> 90% on `app/` stays green).
- Frontend house rules hold: `createElement` + `textContent` only (no HTML strings), `aria-busy`/`aria-disabled` on the in-flight link, `role="status"`/`role="alert"` for the two states, `:focus-visible` on the new Retry button via the global rule, tokens-only CSS if a new class is added.
## Completion Criteria
- [ ] First paint = ONE `depth=1` request; the second visit to a folder is free (cache hit, zero network — unit-pinned).
- [ ] A drilled level shows correct rows (counts, dates, descriptions, pending markers, image thumbnails) inside the phase-156 bounded panel with the sticky header intact.
- [ ] In-flight = `aria-busy` + one status line; failure = `role="alert"` + working Retry with the current level still rendered; a 404 resets to the top level with no stale breadcrumb; a stale response never paints.
- [ ] The three stat cards match the server `totals` at every level, and the pre-phase fallback path exists.
- [ ] Full `uv run pytest` green; ruff + pyright clean.
@@ -0,0 +1,36 @@
# Task 05 — The byte budget: `/api/*` response-size log line + `scripts/api_payload_audit.py` + `AUDIT.md` (phase 167 D5, LOCKED A5)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "Fix that **any any other endpoints that could load a ton of data**."
**Story:** n/a
## Objective
The app can answer "which endpoint is heavy, right now, in production?" from its own logs, and the owner's "any other endpoints" gets a measured, written answer: every candidate endpoint's uncompressed and gzipped size against a stated budget, with each offender either fixed or explicitly accepted.
## Work
1. New `app/core/response_budget.py` — **pure ASGI** (no buffering, SSE-safe):
- wraps `send`, counting the bytes of every `http.response.body` message's `body` chunk, and `receive`-independent timing (`time.monotonic()` at entry → at `more_body=False` completion);
- logs ONE §9-style line per `/api/*` response on the `http.response.body` completion (for a stream that is when the stream ends — say so in the docstring):
`api method=GET path=/api/docs/tree status=200 bytes=204511 enc=gzip duration_ms=41`
(`enc` = the response's `content-encoding` or `identity`; add ` stream=1` for `media_type == "text/event-stream"` responses, where `bytes` is the streamed total and `duration_ms` is the stream's life);
- level: INFO when `bytes >= settings.log_large_response_bytes`, DEBUG otherwise; non-`/api/` paths and non-HTTP scopes pass through silently; an exception in the wrapped app still logs (finally-style, `bytes` as counted) and re-raises unchanged;
- `304`s log `bytes=0` (the win this phase bought is visible in the log, not hidden).
2. `app/config.py` — `log_large_response_bytes: int = 65536` (`BOR_LOG_LARGE_RESPONSE_BYTES`) with the `agent_max_rounds` validator (negative → `ValueError` naming the field; `0` = log EVERY `/api/*` response at INFO). `.env.example` entry with the three-value comment.
3. `app/main.py` — `app.add_middleware(ResponseBudgetMiddleware)` **before** the gzip `add_middleware` call, so the stack is SecurityHeaders (outermost) → gzip → response-budget → caching → session → routes. Consequence to state in the comment: `bytes=` is the bytes the ROUTE produced (uncompressed) while `enc=` reports what went on the wire — the two together answer "how fat is it, and how fat did the client see it".
4. New `scripts/api_payload_audit.py` (`python -m scripts.api_payload_audit`), stdlib + `httpx` only, `app.core.debugging` conditional import preserved if it touches boot (AGENTS.md rule 7 — it does not import the app factory, so just follow the other scripts' shape, see `scripts/eval_retrieval.py`):
- args: `--base-url` (default `http://localhost:8000`), `--username`/`--password` (or `BOR_ADMIN_*` env reads, never hardcoded), `--budget` (default `262144`), `--path-tree` etc. only if needed;
- logs in (the real `/api/login` session cookie), then for each candidate: `GET /api/docs/tree?depth=1`, `/api/docs/tree?depth=8`, `/api/docs/tree/node?source=<first>&path=<first folder>`, `GET /api/docs`, `GET /api/documents/content` for the LARGEST document (ask `/api/docs` then pick by chunks) — its `content` length is the size, `GET /api/chats`, `GET /api/chats/{id}` for the largest chat, `/api/git-sources`, `/api/suggestions`, `/api/sync/status`, `/api/tokens`, `/api/config`, `/api/whoami` — measuring BOTH `identity` and `gzip` request variants (explicit `Accept-Encoding`) via `response.content` length + wall time, plus one conditional re-request for the ETag routes (expect 304/0 bytes);
- prints a table (endpoint, identity bytes, gzip bytes, ratio, ms, verdict) where the verdict is `ok` / `LOOK (> budget)` / `304`, exits 0 (it is a report, not a gate) — and prints the corpus it measured (document count from `/api/docs`) so numbers are interpretable.
5. `.agents/phases/todo/167_api_payload_diet/AUDIT.md` (new, committed with the phase) — the measured record: how it was run (the local DB restored to the **1,000-document live-replica snapshot** via the `restore-test-db` skill, so the order of magnitude matches production), the full table before (uncompressed, no middleware) and after (depth-1 + gzip + ETag), and one verdict line per endpoint: **fixed** (tree: 2 MB → X KB, second load 304) / **accepted with why** (`/api/documents/content` — one document on demand, compressed; `/api/chats/{id}` — the owner opening one saved chat; `/api/sync/status` — tiny but polled, so the log line is what matters) / **follow-up** (anything still over `--budget`, with the proposed remedy named). Do not hand-write numbers: paste the script's output.
6. Remedies in scope if the audit shows an offender (apply, don't just note): add the task-03 ETag pair to any other read route that is large and re-fetched (e.g. `/api/git-sources` if it grows past the budget); raise nothing, reject nothing (LOCKED A5).
7. `README.md` — an "Operational notes" subsection: the log line format + how to read it (`grep ' :: api ' `), `BOR_LOG_LARGE_RESPONSE_BYTES`, running the audit script against a restored replica, and the deploy note that the app gzips itself (so nginx `gzip on` is optional, and `proxy_set_header Accept-Encoding` passthrough is fine).
## Testing & Quality
- Unit (`tests/unit/test_response_budget.py`, new): the line's exact shape for a large 200 (INFO), a small 200 (DEBUG), a 304 (`bytes=0`), a non-`/api/` path (nothing logged), a streaming response (`stream=1`, bytes = sum of chunks, logs on completion), the `enc=gzip` vs `enc=identity` reading, an app exception (logged + re-raised), and the config validator (`0` → INFO for everything, negative → refuses startup).
- Script smoke: `uv run python -m scripts.api_payload_audit --base-url http://localhost:8000` against a running dev app (document count + at least the tree rows printed; the script must fail loudly with a clear message when the app is unreachable or the login is rejected).
- Coverage **>90%** on `app/core/response_budget.py`.
## Completion Criteria
- [ ] One parseable `api method=… path=… status=… bytes=… enc=… duration_ms=…` line per `/api/*` response at the configured level, with the SSE stream logged once at stream end and never buffered.
- [ ] `AUDIT.md` exists in this phase directory with real numbers (before/after), the corpus it measured, and a verdict per candidate endpoint.
- [ ] Every endpoint over the stated budget is either fixed in this phase or carries a named follow-up with its reason.
- [ ] README documents the log line, the env var, the audit script, and the self-gzip deploy note; `uv run pytest` green; ruff + pyright clean.
@@ -0,0 +1,39 @@
# Task 06 — `tests/e2e/test_kb_tree_lazy_load.py`, the migrated pins, the full gates (phase 167 D6)
**Phase:** `167_api_payload_diet` · **Source:** `TODO.md:5` — "GET <https://brain.reeseapps.com/api/docs/tree> loads 2 MB in production which causes long loading times. Fix that any any other endpoints that could load a ton of data."
**Story:** n/a
## Objective
An isolated Playwright suite proves the owner-visible outcome — a small first Sources load, correct content when drilling, no refetch, an honest reset when a folder disappears, gzip on the wire, a 304 on the second visit — and every pin the superseded phase-97 contract owned (D3/A3) is migrated rather than deleted. Then the phase gates close it out.
## Work
1. New `tests/e2e/test_kb_tree_lazy_load.py` (module-scoped `app_server`/`app_url`; seed a multi-level KB directly through the DB the way `tests/e2e/test_kb_tree.py` does — ≥ 3 sources, a 3-deep folder chain, one folder with a stored `folder_summaries` row, one pending folder, and one image doc if `BOR_IMAGES=true` is cheap in this suite's env; read `tests/e2e/test_kb_tree.py` and `tests/e2e/test_kb_tree_nav.py` for the seeding + drill helpers first):
- **the first load is small**: collect the network (`page.on("response")`) for the Sources view boot → exactly one `/api/docs/tree?depth=1` request; its JSON has `totals` with the expected `documents`/`chunks`, its top-level source nodes carry `documents` counts that include the deep files (recursive), and NO node deeper than level 1 exists while level-1 subfolders report `truncated === true` (assert on the parsed body, not on bytes);
- **the stat cards** match `totals` at the top level AND after drilling two levels (the numbers never change — they are catalogue-wide);
- **the drill**: click into a level-1 folder → exactly one `/api/docs/tree/node?source=…&path=…` request; rows render (subfolders sorted, then files in catalog order) with the stored description in the level block, the pending marker on the pending folder, and the phase-156 sticky header/scroll panel intact; click into a deeper folder → a second node fetch; go BACK to the first folder → **zero** new requests (cache);
- **the revalidation**: reload the view → the browser re-sends `If-None-Match` (observe the request header) and the response is `304` with an empty body while the table still renders (the `page.route`/`expect_response` idiom);
- **gzip**: assert the tree response's `Content-Encoding` is `gzip` and `Vary` includes `accept-encoding` (Chromium sends `Accept-Encoding: gzip, deflate, br` by default — pin what actually arrives, and if the harness negotiates identity, assert the server side in the integration suite instead and say so in the test docstring);
- **the vanished folder** (§7.4): a `page.route` stub that answers the node fetch with `404 {"detail":"unknown node"}` → the alert line + Retry button appear, the level the user was on stays rendered; click a parent crumb → the level is fine; then let the real route answer and Retry → the level loads;
- **in-flight**: a `page.route` handler that delays the node response → the clicked link is `aria-busy`/`aria-disabled` and the status line shows, and a second click does NOT issue a second request.
2. `tests/e2e/test_kb_tree.py` (MIGRATE, do not delete): the suite's "ONE fetch of `/api/docs/tree`, drill is client-side" assertions become the depth-1 + per-level assertions (the file's outline comment block first, then each assertion); every OTHER claim it owns (breadcrumb text, back navigation, table columns, dates, summary editor round-trip, pending markers, empty state, the anonymous gate) stays, re-aimed at the level the user is on. Each edit is a re-point, never a removal — state that in the commit body.
3. `tests/e2e/test_navbar_refresh.py:115` — the `("#nav-sources", "/sources.html", "/api/docs/tree")` expectation matches the new URL (keep the substring `/api/docs/tree` if the assertion is a substring test — verify, then pin precisely); re-run that suite.
4. `tests/e2e/test_kb_tree_nav.py`, `test_document_dates.py`, `test_git_source_dates.py`, `test_sync_summary_visibility.py`, `test_table_ux_polish.py`, `test_sources_midstream_bug.py`, `test_ls_tree_drilldown.py` (the agent's `ls` must be untouched) — run in isolation, repair ONLY where a condition assumed the full tree loaded (e.g. a test that expects a deep folder's rows before drilling — add the drill step; a test that walks the tree JSON — give it `?depth=8`, the repair task 02 item 9 defines), never by weakening an assertion.
5. Final gates + docs, in one pass:
- `uv run pytest` (unit + integration) green;
- `uv run pytest --cov=app --cov-report=term-missing` → `app/` > 90% (report the number in the commit body);
- `uv run ruff check . && uv run pyright` clean;
- `uv run pytest tests/e2e/test_kb_tree_lazy_load.py -v --no-cov` green in isolation + the six regression suites in isolation + `tests/e2e/test_smoke.py`;
- `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` unchanged (retrieval is untouched — record it);
- `AUDIT.md` (task 05) reflects the FINAL state after these fixes; `README.md` + `AGENTS.md` reviewed in full once and every inconsistency from this phase fixed (the tree endpoint table, the env var list, the deploy note about self-gzip);
- UI Structure Check (AGENTS.md rule 5) for the drill: the level stays inside the 72rem container, the bounded scroll panel + sticky header survive a lazy render, the loading/alert lines are `role="status"`/`role="alert"`, the Retry button meets 44px + `:focus-visible`, contrast pairs are the existing AA tokens.
6. Move `.agents/phases/todo/167_api_payload_diet/` → `.agents/phases/complete/` (with `AUDIT.md`) and commit with the `00_phase.md` message, `--no-gpg-sign`, one atomic commit (AGENTS.md rule 8).
## Testing & Quality
- E2E: the new suite green in isolation + the six migrated/regression suites + `test_smoke.py` in isolation.
- Coverage **>90%** on `app/` including `compression.py`, `http_cache.py`, `response_budget.py` and the new route/builder lines.
## Completion Criteria
- [ ] Boot = one `depth=1` request whose body has `totals` and no deep children; drill = one node fetch per uncached level, zero on a cache hit; reload = 304.
- [ ] Every level renders the same data the pre-phase full tree rendered (counts, dates, descriptions, pending markers, thumbnails) — proven by the migrated `test_kb_tree.py` claims, none dropped.
- [ ] Failure paths are honest and recoverable (alert + Retry, level preserved, vanished folder resets the breadcrumb), and the agent's `ls` output is unchanged.
- [ ] The measured numbers are in `AUDIT.md`, the coverage number is in the commit body, `uv run pytest` green, ruff + pyright clean, one atomic `--no-gpg-sign` commit, phase dir in `complete/`.
@@ -0,0 +1,14 @@
All verification complete. Final report:
**Phase 165 — Prose word wrap (final verification pass)**
- Verified: `PROSE_FORMATS={txt,text}` + `isProseDoc` (single-site, image arm wins); `renderDocument` three-way branch (md → `.doc-md`, prose → `.doc-prose` text node, else `pre.doc-raw` byte-identical); `.doc-prose` CSS = 72rem column, `pre-wrap` + `overflow-wrap: anywhere`, tokens only, zero literals; `[data-view="pages"]` hide list includes `.doc-prose` as one grouped declaration; phase-116 scrollbar rules intact and modal-scoped; no `innerHTML` in the prose path.
- Tests/lint (all 100% green):
- `uv run pytest` → **3854 passed** (unit + integration)
- `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90%)
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**
- `uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov` → **5 passed** (isolation)
- Isolation regressions: document_viewer 8, image_documents 4, pdf_pages_in_sources 4, pdf_strip_additive_selfheal 5, edit_summaries 4, summary_in_viewer 4, chat_answer_clusters 4, garbage_pdf 3, pdf_support 6, sticky_navbar 3, smoke 3 — all passed
- Completion criteria: image description wraps, full text, both surfaces ✔; `.txt` wraps same way ✔; `py`/`json`/code keep `pre.doc-raw` + horizontal overflow + themed modal scrollbar (E2E negative control + pins) ✔; no innerHTML, AA `--ink`/`--surface` pair ✔; full gates green ✔; commit + phase-dir move are harness-owned (not executed per pipeline rules).
- Documentation: README already updated by task 03 (viewer section + image-docs section); full README/AGENTS.md review found no further inconsistencies — **verified consistent, no changes needed**.
- Notable deviations: `.doc-prose` omits `font-family: inherit` from 00_phase D2's snippet — task 01's authoritative "exact property set" excludes it and `div` inherits fonts anyway (behavior identical). PDF deferral (A4) intact: `format == "pdf"` stays in `.doc-raw`, owner call recorded.
- Next pending phase: `166_hide_empty_answer_bubble`.
@@ -0,0 +1,125 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
...................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 136 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 487 0 100%
--------------------------------------------------
TOTAL 5883 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,16 @@
**Task 01 complete — all gates green.**
**Implemented**
- `frontend/assets/document.js`: `PROSE_FORMATS = new Set(["txt","text"])` + `isProseDoc()` (the ONE prose rule, locked A1; image arm unconditional per phase 122 A3); `renderDocument`'s content branch is now three-way — md → `.doc-md` (byte-identical), prose → `<div class="doc-prose">` via `textContent` (never innerHTML), else → `<pre class="doc-raw">` (byte-identical); same `slot`, so image block/summary/additive strip order is preserved.
- `frontend/assets/styles.css`: new `.doc-prose` right after `.doc-raw` (72rem `var(--chat-column)` column, `pre-wrap` + `overflow-wrap: anywhere`, tokens only, zero literals, no animation); `.doc-prose` joined the `[data-view="pages"]` hide list (one grouped declaration, defensive-today comment); stale `.doc-raw`/header comments updated (also `document-modal.js`).
**Tests / gates (exact)**
- `uv run pytest` → **3844 passed** (incl. new: isProseDoc 26-case node truth table, renderDocument format table `txt/text→doc-prose`, `md→doc-md`, `py/yaml/yml/json/j2/conf/pdf/quadlet/""/unknown→doc-raw`, image→`["doc-image","doc-prose"]`, prose-arm never-innerHTML pin, `.doc-prose`/`.doc-raw` CSS pins, hide-list + wide-column 7→8 updates).
- `uv run pytest --cov=app` → **TOTAL 99%** (>90, unchanged); `uv run ruff check . && uv run pyright` → clean.
- Isolated E2E (all 25 viewer-touching suites + reasoning/timeline/answer_images): all passed, incl. re-run `test_document_viewer` (8), `test_image_documents` (4), `test_pdf_support` (6), `test_chat_answer_clusters` (4).
**Pre-existing defects fixed** (all fail identically on clean HEAD): image-modal selectors `pre.doc-raw`→`.doc-prose` (3 suites); phase-116 scroll fixture re-seeded `.txt`→`.json` per this phase's task-03 migration (code proof stays on the locked raw path); `test_chat_answer_clusters` msg-body pins re-targeted to the phase-161 reasoning container (+ tool record inside); `test_sticky_navbar` re-targeted to phase 156's bounded catalogue card (test-only, deep-scroll proof named as test 2).
**Docs**: README updated (viewer content-by-kind contract; image description wraps). AGENTS.md: no change needed.
**Next pending task**: `165_prose_word_wrap/02_raw_scroll_pins.md`.
@@ -0,0 +1,125 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 136 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 487 0 100%
--------------------------------------------------
TOTAL 5883 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,19 @@
All gates green. Final state:
**Implemented** (task `165_prose_word_wrap/02_raw_scroll_pins.md`)
- New `tests/unit/test_prose_word_wrap.py` — 10 static-source pins (no app/browser): prose set is exactly `{"txt","text"}` (parsed from the `PROSE_FORMATS` literal) + `isProseDoc` reads `doc.is_image === true` **first**; `.doc-raw` keeps `white-space: pre` + `overflow-x: auto`; `.bubble pre` keeps `overflow-x: auto` with no `white-space` added; `.md-table-wrap` byte-identical (`{ overflow-x: auto; }`) and `.md-table` keeps `min-width: max-content`; all four phase-116 `.doc-modal .doc-raw` scrollbar rules present **and** scope-pinned (every scrollbar-themed rule in the file starts with `.doc-modal .doc-raw`); `.doc-prose` has no `overflow-x` and no bare `pre`; AA pin `var(--ink)` on `var(--surface)` with zero hex/rgb/hsl/color-mix literals; the `[data-view="pages"]` hide list still holds `.doc-prose` in one grouped declaration; and the `doc-raw` writer scan (comments stripped) proving `document.js`'s raw arm + `styles.css` are the only writers.
- Hygiene finding recorded in the module docstring (step 3): `doc-edit.js` fills a `<textarea>` (body never becomes markup) and `shared.js` renders chat bubbles via `renderMarkdown` into `.bubble` — neither is a second content renderer, so the prose rule correctly stays in `isProseDoc` + `.doc-prose`. PDF (`format == "pdf"`) deferral (A4) documented as an owner decision on record.
- Verified by mutation: prose-set drift, `.doc-raw` → `pre-wrap`, `.bubble pre` gaining `white-space`, `overflow-x` on `.doc-prose`, scrollbar rule escaping `.doc-modal`, hide list losing `.doc-prose`, and a hex literal each fail exactly one named test (files restored byte-identical after each).
**Gates**
- `uv run pytest tests/unit/test_prose_word_wrap.py -v --no-cov` → 10 passed
- `uv run pytest tests/unit -k "additive_toggle or markdown_tables or date_editor or image_documents or document_viewer" -v` → 149 passed
- `uv run pytest` → **3854 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90, `app/` untouched)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
- Viewer-touching E2E in isolation: `test_document_viewer` (8), `test_image_documents` (4), `test_pdf_strip_additive_selfheal`, `test_pdf_pages_in_sources`, `test_edit_summaries`, `test_summary_in_viewer`, `test_garbage_pdf` (15) — all passed
**Docs**: no doc impact (test-only task; README already carries the phase-165 content-by-kind contract from task 01; AGENTS.md unchanged).
**Notable**: pins use comment-stripped source so provenance comments naming `doc-raw`/`scrollbar-color` can't create false positives; the scope pin asserts *exactly 4* themed-scrollbar rules, so a future global restyle breaks it loudly.
**Next pending task**: `165_prose_word_wrap/03_e2e_prose_wrap.md`.
@@ -0,0 +1,125 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
...................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 136 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 487 0 100%
--------------------------------------------------
TOTAL 5883 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,17 @@
All gates green. Final report:
**Implemented (task 03, phase 165):**
- New `tests/e2e/test_prose_word_wrap.py` (5 tests): module seed (image doc w/ served PNG + ~860-char description, `notes/prose-long.txt` w/ 858-char line + 2nd paragraph, `notes/code-long.py` 906-char control) asserting on **page + modal**: one `.doc-prose` w/ full raw text, `scrollWidth ≤ clientWidth+1`, height > 2.2×line-height, `pre-wrap`, `max-width: 1152px` (72rem), `<img>` above prose, summary suppressed; code control keeps `pre.doc-raw` + real horizontal overflow (locked A1/A3).
- Task items 2–4 (migrate `test_document_viewer.py` `.json` fixture, `test_image_documents.py` `.doc-prose`, unit comment) were already carried by tasks 01/02 — verified claims intact, no assertion weakened.
- **Defect fixed:** `test_sticky_navbar.py::test_short_page_stretch…` flaked (~1-in-8, reproduced 2×): read `#composer` computed `position` during phase-88's pre-boot static window; added the house `_wait_chat_booted` sentinel wait before measuring (6/6 green after).
- README viewer sentence extended (themed phase-116 scrollbar applies to whatever still scrolls); AGENTS.md §7 verified — 72rem reading-column item already covers `.doc-prose`, no change.
**Gates (exact):**
- `uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov` → 5 passed
- Regression suites, each isolated `--no-cov`: test_document_viewer 8✓, test_image_documents 4✓, test_pdf_pages_in_sources 4✓, test_pdf_strip_additive_selfheal 5✓, test_edit_summaries 4✓, test_summary_in_viewer 4✓, test_chat_answer_clusters 4✓ (+ test_pdf_support 6✓, test_sticky_navbar 3✓, test_smoke 3✓)
- `uv run pytest` → 3854 passed; `--cov=app` → **TOTAL 99%** (>90)
- `uv run ruff check .` clean; `uv run pyright` → 0 errors
**Notes/deviations:** innerText returns source (not soft-wrapped) lines in pre-wrap — paragraph-break proof re-pointed to raw `\n\n` + rendered blank line; wrapping itself is pinned by the scrollWidth/height bars. Per harness rules I did **not** move the phase dir to `complete/` or commit — the harness owns staging/commit (`feat(viewer): wrap image descriptions and plain-text docs as prose instead of scrolling horizontally`).
**Next pending task:** none in phase 165 (task 03 was the last; phase completes on the harness commit).
@@ -0,0 +1,125 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
...................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 136 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 487 0 100%
--------------------------------------------------
TOTAL 5883 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 43 KiB

+11 -2
View File
@@ -135,7 +135,14 @@ restores on a refresh — but a shared chat never shows it.
### RAG / Knowledge base (`/sources.html`) ### RAG / Knowledge base (`/sources.html`)
The indexed document list. The *Path* column opens each document in the same The indexed document list. The *Path* column opens each document in the same
**almost-fullscreen modal**. Document rows carry a *Created* date and folder **almost-fullscreen modal** (and `/document.html` renders it through the one
shared core). Content renders by kind (phase 165): markdown becomes a
rendered document, an image's description or a plain-text (`.txt`) body
renders as **wrapping prose** in the 72rem reading column, and code and
code-ish formats keep the monospace block with a horizontal scroller
(long lines never wrap in code — locked; the themed modal scrollbar of
phase 116 keeps applying to whatever still scrolls). Document rows carry
a *Created* date and folder
rows an *Updated* date (phase 106) — sourced at sync time (git commit dates rows an *Updated* date (phase 106) — sourced at sync time (git commit dates
for git sources, restored metadata for archive uploads, file mtime otherwise; for git sources, restored metadata for archive uploads, file mtime otherwise;
unresolvable or future dates fall back to today) — and the admin can correct a unresolvable or future dates fall back to today) — and the admin can correct a
@@ -705,7 +712,9 @@ first-class document:
- **Retrieved & displayed.** Image docs appear in the Sources catalog - **Retrieved & displayed.** Image docs appear in the Sources catalog
with a small lazy-loaded thumbnail (progressive enhancement — a fetch with a small lazy-loaded thumbnail (progressive enhancement — a fetch
failure falls back to the doc icon), the document viewer renders the failure falls back to the doc icon), the document viewer renders the
image with the description below it, and a cited image doc renders image with the description below it as **wrapping prose** (phase 165 —
the full description reads by scrolling down, never sideways), and a
cited image doc renders
**in the chat's sources images row** (phase 130: its own **in the chat's sources images row** (phase 130: its own
`.msg-meta.msg-images` cluster below the chips row — no chip for an `.msg-meta.msg-images` cluster below the chips row — no chip for an
image doc, the figure is the citation; alt text = the description; image doc, the figure is the citation; alt text = the description;
-3
View File
@@ -1,4 +1 @@
# TODO # TODO
- [ ] Need word wrap for image explanations and other long text rather than scrolling horizontally
- [ ] There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly. ![example image](image.png)
+3 -1
View File
@@ -17,7 +17,9 @@
* stateless endpoint the /document.html page uses — identical * stateless endpoint the /document.html page uses — identical
* percent-encoding), and renders through document.js's shared * percent-encoding), and renders through document.js's shared
* renderDocument: md/markdown → .doc-md via the escape-first * renderDocument: md/markdown → .doc-md via the escape-first
* renderer, other formats → <pre class="doc-raw"> via textContent. * renderer, the prose set (phase 165 — image descriptions,
* txt/text) → <div class="doc-prose">, other formats →
* <pre class="doc-raw"> via textContent.
* A !ok / network failure renders a short "document not found" line * A !ok / network failure renders a short "document not found" line
* in the content area. * in the content area.
* • closeDocumentModal() — hides the overlay, removes the Escape/Tab * • closeDocumentModal() — hides the overlay, removes the Escape/Tab
+40 -5
View File
@@ -9,10 +9,14 @@
* so the two surfaces can never drift: * so the two surfaces can never drift:
* *
* • md / markdown → the shared escape-first renderer (markdown.js) * • md / markdown → the shared escape-first renderer (markdown.js)
* in a ≤46rem centered column; * in the 72rem centered reading column;
* • prose (phase 165, LOCKED A1 — isProseDoc: an image doc's
* description or a txt/text doc) → the content
* as a text node inside <div class="doc-prose">,
* pre-wrap: it WRAPS at the reading column;
* • any other → the raw content as a text node inside * • any other → the raw content as a text node inside
* <pre class="doc-raw"> (mono, horizontal * <pre class="doc-raw"> (mono, horizontal
* scroll); * scroll — locked A3: code never wraps);
* • doc.summary non-empty (phase 36 — phase 30 summaries exist * • doc.summary non-empty (phase 36 — phase 30 summaries exist
* only on non-markdown docs) → a labeled * only on non-markdown docs) → a labeled
* .doc-summary section ABOVE the content; * .doc-summary section ABOVE the content;
@@ -111,7 +115,8 @@
* the summary, the WCAG alt contract; a NULL summary falls back to * the summary, the WCAG alt contract; a NULL summary falls back to
* the title), and the description (doc.content — the ONLY readable * the title), and the description (doc.content — the ONLY readable
* text of the doc) follows in the EXISTING plain-content slot below * text of the doc) follows in the EXISTING plain-content slot below
* (the .doc-raw path; a description is prose, not markdown). The * (phase 165: the .doc-prose path — a description IS prose, so it
* wraps; it was never markdown, and never belonged in the pre). The
* labeled Summary panel is suppressed for the verbatim-description * labeled Summary panel is suppressed for the verbatim-description
* case (summary === content — the importer invariant; the panel * case (summary === content — the importer invariant; the panel
* would duplicate the text right below the image); an admin-edited * would duplicate the text right below the image); an admin-edited
@@ -313,6 +318,26 @@ function copyPathButton() {
return b; return b;
} }
/* ---------- prose set (phase 165, task 01 — LOCKED A1) ----------
* The ONE place the prose rule lives (TODO.md L3: "Need word wrap for
* image explanations and other long text rather than scrolling
* horizontally"). The defect: the description used to land in the
* `pre` and each generated line became one unbreakable line — a
* horizontal scrollbar where every other text surface wraps.
*
* The image arm is UNCONDITIONAL (phase 122 A3: an image doc's
* `content` IS the vision description — prose by construction,
* whatever its extension). The `format` arm covers plain-text docs;
* `doc.format` comes from the server's doc_format() (the lowercased
* suffix). Code and code-ish formats (py, yaml, json, j2, the quadlet
* family, pdf — locked A1/A3: phase 116's scroll contract stands)
* never enter this set. */
const PROSE_FORMATS = new Set(["txt", "text"]);
function isProseDoc(doc) {
return doc.is_image === true || PROSE_FORMATS.has(doc.format);
}
/* ---------- shared renderer (phase 26, task 02) ---------- /* ---------- shared renderer (phase 26, task 02) ----------
* Populates the three elements every render surface provides: a title, * Populates the three elements every render surface provides: a title,
* a .doc-meta badge row (source · format · mono path · created · * a .doc-meta badge row (source · format · mono path · created ·
@@ -321,7 +346,8 @@ function copyPathButton() {
* section first (phase 36: only when doc.summary is non-empty — * section first (phase 36: only when doc.summary is non-empty —
* markdown docs and fail-soft rows carry none, so they render exactly * markdown docs and fail-soft rows carry none, so they render exactly
* as before), then .doc-md for md/markdown (the shared escape-first * as before), then .doc-md for md/markdown (the shared escape-first
* renderer), <pre class="doc-raw"> otherwise. The XSS contract: * renderer), <div class="doc-prose"> for the prose set (phase 165 —
* isProseDoc), <pre class="doc-raw"> otherwise. The XSS contract:
* innerHTML only through renderMarkdown; every document-derived * innerHTML only through renderMarkdown; every document-derived
* string (summary text included) is a text node. */ * string (summary text included) is a text node. */
export function renderDocument(doc, { titleEl, metaEl, contentEl }) { export function renderDocument(doc, { titleEl, metaEl, contentEl }) {
@@ -385,7 +411,7 @@ export function renderDocument(doc, { titleEl, metaEl, contentEl }) {
// document) is the ONE branch that replaces the content slot (phase // document) is the ONE branch that replaces the content slot (phase
// 163: the PRIMARY class only — an "additive" doc falls through to // 163: the PRIMARY class only — an "additive" doc falls through to
// the text rendering with the strip behind a toggle, D4/A1): the // the text rendering with the strip behind a toggle, D4/A1): the
// markdown/.doc-raw content block AND the labeled Summary panel // markdown/.doc-prose/.doc-raw content block AND the labeled Summary panel
// (including its admin edit wiring — moot without the panel) are // (including its admin edit wiring — moot without the panel) are
// NOT built. The per-page descriptions stay in the DB for Q&A (the // NOT built. The per-page descriptions stay in the DB for Q&A (the
// importer's content/embeddings/summary are untouched); they are // importer's content/embeddings/summary are untouched); they are
@@ -467,6 +493,15 @@ export function renderDocument(doc, { titleEl, metaEl, contentEl }) {
wrap.className = "doc-md"; wrap.className = "doc-md";
wrap.innerHTML = renderMarkdown(doc.content); // escape-first: XSS-safe wrap.innerHTML = renderMarkdown(doc.content); // escape-first: XSS-safe
slot.appendChild(wrap); slot.appendChild(wrap);
} else if (isProseDoc(doc)) {
// Phase 165 (task 01, LOCKED A2): the prose arm — a TEXT NODE in
// the wrapping .doc-prose block (pre-wrap carries the newlines;
// no markdown parsing of a description). The XSS contract is
// untouched: the content is never parsed as HTML.
const prose = document.createElement("div");
prose.className = "doc-prose";
prose.textContent = doc.content; // text node: never parsed as HTML
slot.appendChild(prose);
} else { } else {
const pre = document.createElement("pre"); const pre = document.createElement("pre");
pre.className = "doc-raw"; pre.className = "doc-raw";
+35 -1
View File
@@ -5214,7 +5214,9 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
} }
.doc-date-status:empty, .doc-date-error:empty { display: none; } /* an empty live line takes no space */ .doc-date-status:empty, .doc-date-error:empty { display: none; } /* an empty live line takes no space */
/* Raw (non-markdown) formats: full-width mono pre, horizontal scroll. */ /* Raw (non-markdown, non-prose) formats: full-width mono pre,
horizontal scroll (locked A3, phase 165 — the prose set renders in
the wrapping .doc-prose block below, never here). */
.doc-raw { .doc-raw {
width: 100%; width: 100%;
margin: 0; margin: 0;
@@ -5230,6 +5232,33 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
line-height: 1.5; line-height: 1.5;
} }
/* Prose (phase 165, task 01 — TODO.md L3, LOCKED A1/A2): the image
description and plain-text (txt/text) docs as WRAPPING prose — the
.doc-md column geometry (72rem reading column, the surface card)
with pre-wrap rhythm. The defect: these docs landed in the .doc-raw
pre above (white-space: pre), so every generated line was one
unbreakable line and a long sentence scrolled sideways. The content
stays a text node (pre-wrap carries the paragraph breaks — no
markdown parsing: a description is prose, not markup). Tokens only
(no new hue — the monochrome invariant; --ink on --surface is the
documented 14.5:1 AA pair). Static content — no animation (nothing
for prefers-reduced-motion to still). Code keeps the .doc-raw
scroller (phase 116 locked A1, here locked A3). */
.doc-prose {
width: 100%;
max-width: var(--chat-column);
margin-inline: auto;
background: var(--surface);
border: 1px solid var(--line);
border-radius: var(--radius);
box-shadow: var(--shadow);
padding: 1.5rem 1.75rem;
color: var(--ink);
white-space: pre-wrap;
overflow-wrap: anywhere;
line-height: 1.55;
}
/* Image documents (phase 122, task 04): the viewer's image block — /* Image documents (phase 122, task 04): the viewer's image block —
the document's persistent bytes (the /api/documents/{id}/image the document's persistent bytes (the /api/documents/{id}/image
route) above its description (the .doc-raw content slot below). The route) above its description (the .doc-raw content slot below). The
@@ -5358,9 +5387,14 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
font-weight: 700; /* pressed = weight + fill, never color alone (B5) */ font-weight: 700; /* pressed = weight + fill, never color alone (B5) */
} }
.doc-views .doc-pages { display: none; } /* Text default — hidden lazy imgs never fetch */ .doc-views .doc-pages { display: none; } /* Text default — hidden lazy imgs never fetch */
/* The text views hide behind the strip on Pages. .doc-prose (phase
165, task 01) joins the list DEFENSIVELY today — an image/txt doc
never page-qualifies — and becomes load-bearing the day the owner
adds "pdf" to PROSE_FORMATS (phase 165 D1 + phase 163 A2). */
.doc-views[data-view="pages"] .doc-summary, .doc-views[data-view="pages"] .doc-summary,
.doc-views[data-view="pages"] .doc-md, .doc-views[data-view="pages"] .doc-md,
.doc-views[data-view="pages"] .doc-raw, .doc-views[data-view="pages"] .doc-raw,
.doc-views[data-view="pages"] .doc-prose,
.doc-views[data-view="pages"] .doc-image { display: none; } .doc-views[data-view="pages"] .doc-image { display: none; }
.doc-views[data-view="pages"] .doc-pages { display: grid; } .doc-views[data-view="pages"] .doc-pages { display: grid; }
+32 -12
View File
@@ -52,8 +52,11 @@ Test → observable mapping (Playwright Mapping Rule):
BOTH docs — the read text doc first (read order), then the BOTH docs — the read text doc first (read order), then the
auto-cited image ref (``image_url`` present; the text ref's key is auto-cited image ref (``image_url`` present; the text ref's key is
ABSENT); the last brain bubble's ``.msg-body`` direct children are ABSENT); the last brain bubble's ``.msg-body`` direct children are
exactly ``[tool-calls-disclosure, bubble, msg-meta, msg-meta exactly ``[reasoning, bubble, msg-meta, msg-meta msg-images,
msg-images, msg-meta msg-actions]`` — the chips row (EXACTLY the msg-meta msg-actions]`` — the ONE phase-161 reasoning container
ABOVE the answer (this turn's tool record lives inside its
``.reasoning-body`` as a ``.tool-calls-disclosure`` block, the
phase-130 timeline re-targeted by phase 161 D1) then the chips row (EXACTLY the
text chip — NO chip for the image doc, phase 130 task 01) before text chip — NO chip for the image doc, phase 130 task 01) before
the images row (the figure, ``<img>`` loaded from the bytes route, the images row (the figure, ``<img>`` loaded from the bytes route,
alt + caption settling to the canned description), no related row alt + caption settling to the canned description), no related row
@@ -205,11 +208,14 @@ assert all(len(q) <= 120 for q in (Q_ORDER, Q_FIGURE, Q_RESTORE, Q_NONADMIN)), (
) )
#: The .msg-body child sequence of a completed scripted-read bubble #: The .msg-body child sequence of a completed scripted-read bubble
#: (phase 130's fix, DOM-asserted): the tool record (the scripted read), #: (phase 130's fix, DOM-asserted; re-targeted by phase 161 D1): the
#: the answer bubble, the chips row (the bare .msg-meta), the images #: ONE reasoning container (the scripted read's tool record rides
#: row, the actions row LAST. The live and restore paths share it. #: INSIDE it — .reasoning .tool-calls-disclosure, asserted beside the
#: sequence), the answer bubble, the chips row (the bare .msg-meta),
#: the images row, the actions row LAST. The live and restore paths
#: share it.
EXPECTED_CHILD_CLASSES = [ EXPECTED_CHILD_CLASSES = [
"tool-calls-disclosure", "reasoning",
"bubble", "bubble",
"msg-meta", "msg-meta",
"msg-meta msg-images", "msg-meta msg-images",
@@ -759,13 +765,20 @@ def test_answer_clusters_render_in_order(
expect(bubble).to_contain_text(TEXT_TITLE) expect(bubble).to_contain_text(TEXT_TITLE)
expect(bubble).to_contain_text(READ_BODY_QUOTE) expect(bubble).to_contain_text(READ_BODY_QUOTE)
# The DOM ORDER (the fix): tool record, answer, chips row, images # The DOM ORDER (the fix): reasoning container (tool record
# row, actions row LAST — asserted exactly (this sequence fails # inside), answer, chips row, images row, actions row LAST —
# against the pre-phase soup div, where the buttons shared the # asserted exactly (this sequence fails against the pre-phase
# chips row and the figure interleaved with it). # soup div, where the buttons shared the chips row and the figure
# interleaved with it).
assert _body_child_classes(page) == EXPECTED_CHILD_CLASSES, ( assert _body_child_classes(page) == EXPECTED_CHILD_CLASSES, (
_body_child_classes(page) _body_child_classes(page)
) )
# The tool record lives INSIDE the ONE reasoning container above
# the answer (phase 161 D1/D5 re-target of the phase-130
# timeline) — the read line carries the read document's path.
tool_record = _last_brain(page).locator(".reasoning .tool-calls-disclosure")
expect(tool_record).to_have_count(1)
expect(tool_record.locator(".tool-call").last).to_contain_text(TEXT_SP)
# The clusters + the actions row's contents (all three buttons — # The clusters + the actions row's contents (all three buttons —
# admin, docs repo configured) + nothing after the actions row. # admin, docs repo configured) + nothing after the actions row.
@@ -817,7 +830,9 @@ def test_image_without_chip_keeps_its_citation_via_figure(
img = page.locator("#doc-modal .doc-image-img") img = page.locator("#doc-modal .doc-image-img")
expect(img).to_have_count(1) expect(img).to_have_count(1)
expect(img).to_have_attribute("src", seeded["image_url"]) expect(img).to_have_attribute("src", seeded["image_url"])
expect(page.locator("#doc-modal pre.doc-raw")).to_have_text( # Phase 165: the description renders in the wrapping .doc-prose
# block (an image description is prose — no scrolling pre).
expect(page.locator("#doc-modal .doc-prose")).to_have_text(
seeded["description"] seeded["description"]
) )
page.keyboard.press("Escape") # the phase-26 close affordance page.keyboard.press("Escape") # the phase-26 close affordance
@@ -877,10 +892,15 @@ def test_restore_renders_the_same_clusters(
expect(restored).to_contain_text(READ_ANSWER_PREFIX, timeout=SETTLE_TIMEOUT_MS) expect(restored).to_contain_text(READ_ANSWER_PREFIX, timeout=SETTLE_TIMEOUT_MS)
# The IDENTICAL structure (the restore renders through the same # The IDENTICAL structure (the restore renders through the same
# appenders — the child sequence is the live one, exactly). # appenders — the child sequence is the live one, exactly) — the
# segments path re-renders the reasoning container with the tool
# record inside (phase 161 D4).
assert _body_child_classes(page) == EXPECTED_CHILD_CLASSES == live_children, ( assert _body_child_classes(page) == EXPECTED_CHILD_CLASSES == live_children, (
_body_child_classes(page) _body_child_classes(page)
) )
tool_record = _last_brain(page).locator(".reasoning .tool-calls-disclosure")
expect(tool_record).to_have_count(1)
expect(tool_record.locator(".tool-call").last).to_contain_text(TEXT_SP)
_assert_cluster_rows( _assert_cluster_rows(
page, seeded, page, seeded,
action_buttons=["tune-btn", "retry-btn", "save-as-doc-btn"], action_buttons=["tune-btn", "retry-btn", "save-as-doc-btn"],
+17 -8
View File
@@ -38,10 +38,13 @@ Test → story mapping (Playwright Mapping Rule):
inner content). inner content).
7. ``test_modal_theme_and_no_cdn`` — dark page background, the panel on 7. ``test_modal_theme_and_no_cdn`` — dark page background, the panel on
the Phase-08 surface colour, every asset same-origin or ``data:``. the Phase-08 surface colour, every asset same-origin or ``data:``.
8. ``test_raw_code_block_themed_scrollbar`` — phase 116: the raw code 8. ``test_raw_code_block_themed_scrollbar`` — phase 116: the raw CODE
block (``pre.doc-raw``) overflows horizontally, a scroll action moves block (``pre.doc-raw`` — a seeded ``.json`` doc since phase 165;
it, the themed scrollbar tokens are in effect via computed style, and prose — image descriptions and ``txt``/``text`` — now WRAPS in
a screenshot records the themed (not native light-gray) bar. ``.doc-prose`` and never lands here) overflows horizontally, a
scroll action moves it, the themed scrollbar tokens are in effect
via computed style, and a screenshot records the themed (not native
light-gray) bar.
""" """
from __future__ import annotations from __future__ import annotations
@@ -153,14 +156,20 @@ def _seed_long_line_doc() -> None:
a non-markdown doc whose one long line is deliberately longer than a non-markdown doc whose one long line is deliberately longer than
the modal width — the shape of the TODO's quest .pl ``quest::say`` the modal width — the shape of the TODO's quest .pl ``quest::say``
line. Direct DB seed, the ``test_modal_xss_safe`` pattern (the line. Direct DB seed, the ``test_modal_xss_safe`` pattern (the
viewer is database-only; no chunks needed).""" viewer is database-only; no chunks needed). Phase 165: the path is
``.json`` — txt/text joined the PROSE set (wrapped in .doc-prose),
so the code-scroll fixture is a CODE doc, keeping the locked A1
proof on the branch that still scrolls."""
long_line = "quest::say(\"" + ("the quick brown fox jumps over the lazy dog " * 20) + "\")" long_line = "quest::say(\"" + ("the quick brown fox jumps over the lazy dog " * 20) + "\")"
with SessionLocal() as db: with SessionLocal() as db:
db.add( db.add(
Document( Document(
source="docs", source="docs",
path="notes/long-line.txt", # Phase 165: .json, NOT .txt — txt/text joined the PROSE
full_path="/tmp/long-line.txt", # set (wrapped in .doc-prose); the phase-116 scroll
# contract binds CODE, and json stays code-ish (D1).
path="notes/long-line.json",
full_path="/tmp/long-line.json",
title="Long Line Fixture", title="Long Line Fixture",
content=( content=(
"# long-line-fixture: the line below is deliberately\n" "# long-line-fixture: the line below is deliberately\n"
@@ -551,7 +560,7 @@ def test_raw_code_block_themed_scrollbar(
login(page, app_url) login(page, app_url)
_drill(page, "docs", "notes") _drill(page, "docs", "notes")
row = page.locator("#docs-tbody tr", has_text="long-line.txt") row = page.locator("#docs-tbody tr", has_text="long-line.json")
expect(row).to_have_count(1) expect(row).to_have_count(1)
row.locator("td:nth-child(2) a.doc-link").click() row.locator("td:nth-child(2) a.doc-link").click()
+5 -3
View File
@@ -387,7 +387,8 @@ def test_document_viewer_renders_image_and_description(
(phase 26): the ``is_image`` content block renders the PERSISTENT (phase 26): the ``is_image`` content block renders the PERSISTENT
bytes first (``.doc-image-img`` from the bytes route, ``alt`` = bytes first (``.doc-image-img`` from the bytes route, ``alt`` =
the summary), and the description follows in the normal content the summary), and the description follows in the normal content
slot (``pre.doc-raw`` — the plain-content path). The labeled slot (``.doc-prose`` — the phase-165 wrapping prose arm: an image
description is prose, it WRAPS, never a scrolling pre). The labeled
Summary panel is SUPPRESSED for the verbatim-description case Summary panel is SUPPRESSED for the verbatim-description case
(summary === content — the importer invariant; the panel would (summary === content — the importer invariant; the panel would
duplicate the text right below the image).""" duplicate the text right below the image)."""
@@ -406,8 +407,9 @@ def test_document_viewer_renders_image_and_description(
expect(img).to_have_attribute("alt", seeded["description"]) expect(img).to_have_attribute("alt", seeded["description"])
# The description below the image (the doc's readable content IS # The description below the image (the doc's readable content IS
# the vision description). # the vision description) — prose (phase 165), never the pre.
expect(modal.locator("pre.doc-raw")).to_have_text(seeded["description"]) expect(modal.locator(".doc-prose")).to_have_text(seeded["description"])
expect(modal.locator("pre.doc-raw")).to_have_count(0)
# The verbatim case: no duplicate Summary panel. # The verbatim case: no duplicate Summary panel.
expect(modal.locator(".doc-summary")).to_have_count(0) expect(modal.locator(".doc-summary")).to_have_count(0)
# No "Image unavailable" note — the copy is intact. # No "Image unavailable" note — the copy is intact.
+4 -2
View File
@@ -546,7 +546,8 @@ def test_document_viewer_renders_the_embedded_image_doc(
"""The embedded image's row (the synthetic ``#page=2#img=0.png`` """The embedded image's row (the synthetic ``#page=2#img=0.png``
path) opens the modal: the PERSISTENT copy renders first (the bytes path) opens the modal: the PERSISTENT copy renders first (the bytes
route, alt = the description), the description follows in the route, alt = the description), the description follows in the
content slot, and the labeled Summary panel is SUPPRESSED for the WRAPPING prose block (phase 165: .doc-prose — an image description
is prose, never a scrolling pre), and the labeled Summary panel is SUPPRESSED for the
verbatim-description case (summary === content — the importer verbatim-description case (summary === content — the importer
invariant; the panel would duplicate the text right below the invariant; the panel would duplicate the text right below the
image).""" image)."""
@@ -563,7 +564,8 @@ def test_document_viewer_renders_the_embedded_image_doc(
expect(img).to_have_attribute("src", seeded["image_url"]) expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("alt", seeded["description"]) expect(img).to_have_attribute("alt", seeded["description"])
expect(modal.locator("pre.doc-raw")).to_have_text(seeded["description"]) expect(modal.locator(".doc-prose")).to_have_text(seeded["description"])
expect(modal.locator("pre.doc-raw")).to_have_count(0)
# The verbatim case: no duplicate Summary panel. # The verbatim case: no duplicate Summary panel.
expect(modal.locator(".doc-summary")).to_have_count(0) expect(modal.locator(".doc-summary")).to_have_count(0)
# No "Image unavailable" note — the copy is intact. # No "Image unavailable" note — the copy is intact.
+494
View File
@@ -0,0 +1,494 @@
"""Phase 165 E2E (Playwright): prose word wrap — the TODO.md L3 claim
("Need word wrap for image explanations and other long text rather than
scrolling horizontally") proven end-to-end, with ZERO horizontal
overflow, on BOTH viewer surfaces: the ``/document.html`` page and the
in-page modal opened from a Sources row.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_prose_word_wrap.py -v --no-cov
Seeding is the direct-DB idiom of ``tests/e2e/test_document_viewer.py``
(``_seed_long_line_doc`` — the viewer is database-only: no chunks, no
fixture import, no mock LLM turn) + the image-document shape of
``tests/e2e/test_image_documents.py``: an ``is_image`` row whose
``content`` AND ``summary`` are the vision description (the importer
invariant, phase 122 A3) and whose ``image_path`` points at a real PNG
written into the module's scratch dir — so the ``image_url`` bytes
route serves REAL pixels (probed over HTTP at seed time and again
through the rendered ``<img>.naturalWidth``).
Three seeded docs, one per rendering arm (phase 165 D1):
* ``notes/landscape.png`` — image doc (prose by construction, whatever
its extension): a ~860-char SINGLE-PARAGRAPH vision description;
* ``notes/prose-long.txt`` — the ``txt`` prose arm: one ~860-char line
PLUS a second paragraph (proves BOTH intra-line wrap and the
paragraph break — ``white-space: pre-wrap`` carries the ``\\n\\n`` as
a blank rendered line);
* ``notes/code-long.py`` — the NEGATIVE control (locked A1/A3): one
~900-char line that MUST keep overflowing horizontally in
``pre.doc-raw`` — the wrapping fix must never wrap code.
Per prose doc the suite asserts, on BOTH surfaces: the prose block
exists exactly once and carries the FULL text (nothing clamped),
``scrollWidth <= clientWidth + 1`` (the TODO's literal defect — no
horizontal scrollbar), the block is taller than 2.2 line-heights (the
text really wrapped), the computed ``white-space`` is ``pre-wrap``, and
the block rides the 72rem reading column (``max-width: 1152px``, never
wider than its content host). The image doc additionally keeps the
phase-122 ordering (the ``<img>`` ABOVE the prose) and the suppressed
Summary panel (``summary === content``).
Test → observable mapping (Playwright Mapping Rule):
1. ``test_image_doc_description_wraps_on_the_page`` — the standalone
viewer page: image above the wrapping ``.doc-prose``, full text,
zero horizontal overflow, summary suppressed, no ``pre.doc-raw``.
2. ``test_image_doc_description_wraps_in_the_modal`` — the same doc
through the Sources-row modal: identical contract on
``#doc-modal-content``.
3. ``test_txt_doc_wraps_on_the_page`` — the ``.txt`` body on the page:
full raw text, the ``\\n\\n`` preserved AND rendered as a blank
line, no horizontal overflow, 72rem column.
4. ``test_txt_doc_wraps_in_the_modal`` — the same through the modal.
5. ``test_code_doc_still_scrolls_horizontally`` — the locked code path
on both surfaces: ``pre.doc-raw``, ``white-space: pre``,
``scrollWidth > clientWidth`` (phase 116 A1 alive E2E, not just in
the CSS pins).
"""
from __future__ import annotations
import base64
from collections.abc import Iterator
from datetime import UTC, datetime
from urllib.parse import quote
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.db import SessionLocal
from app.models import Document
from e2e.auth_helpers import login
SOURCE = "docs"
IMAGE_PATH = "notes/landscape.png"
TXT_PATH = "notes/prose-long.txt"
CODE_PATH = "notes/code-long.py"
#: The mock-free stand-in for a vision description: ONE single-line
#: paragraph (no ``\\n`` at all — exactly the shape that used to become
#: one unbreakable line in the old ``pre.doc-raw``), long enough that
#: rendering it unwrapped would overflow any viewport.
DESCRIPTION = (
"A wide screenshot of the homelab dashboard on the wall display: the top row shows four "
"gauge panels, CPU load at eighteen percent, memory at six of thirty-two gigabytes, disk at "
"forty-one percent across the merged pool, and network throughput peaking at four hundred "
"megabits during the nightly backup window, while the middle band lists every service as a "
"green pill (proxmox, pihole, nginx proxy manager, gitea, immich, postgres, valkey, and the "
"rag indexer), and the bottom strip is a time-series graph of requests per second whose "
"legend reads ingress, api, and embeddings; the sidebar carries the clock, a weather "
"widget, and a small map pin marking the rack location beside the hall closet. Every label "
"on the capture stays legible at full width, from the uptime counter in the corner to the "
"footnote that names the exporter version and the scrape interval."
)
#: The ``.txt`` body: ONE ~860-char line (the intra-line wrap proof)
#: plus a second paragraph (the paragraph-break proof).
_FOX = "the quick brown fox jumps over the lazy dog "
TXT_LINE = "notes/prose-long.txt reads: " + _FOX * 18
TXT_PARA2 = (
"The second paragraph exists to prove the paragraph BREAK survives the prose block: "
"pre-wrap carries it."
)
TXT_CONTENT = TXT_LINE + "\n\n" + TXT_PARA2
#: The negative control: one ~900-char code line — locked A1 (phase
#: 116): code NEVER wraps, it keeps the horizontal scroller.
CODE_LINE = "result = render_report(" + '"' + _FOX * 20 + '"' + ")"
assert len(DESCRIPTION) >= 800, "the vision description must be a long single line"
assert "\n" not in DESCRIPTION, "the description must be ONE line (the un-wrapped defect shape)"
assert len(TXT_LINE) >= 800 and "\n" not in TXT_LINE
assert len(CODE_LINE) >= 800 and "\n" not in CODE_LINE
#: A real 1×1 transparent PNG (the phase-122 fixture bytes — the route
#: serves the real bytes; naturalWidth 1 is the served-proof probe).
PNG_1X1 = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
"AAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
)
# ---------------------------------------------------------------------------
# DB helpers (the test_document_viewer direct-seed idiom)
# ---------------------------------------------------------------------------
def _truncate_kb() -> None:
"""Fresh KB per module run (the E2E isolation pattern of
tests/e2e/test_image_documents.py — every catalogue table the
seeded rows could touch resets, so leftovers from an earlier
suite can never corrupt this one's counts or rows)."""
with SessionLocal() as db:
db.execute(text(
"TRUNCATE chunks, documents, query_log, kb_overview, "
"git_sources, sources_meta, folder_summaries"
))
db.commit()
@pytest.fixture(scope="module")
def seeded(app_url: str, tmp_path_factory: pytest.TempPathFactory) -> Iterator[dict[str, str]]:
"""Seed the three docs (image / txt / code) straight into the KB —
the viewer is database-only, so no chunks and no LLM are involved
(the ``_seed_long_line_doc`` pattern) — and write the image PNG
into the module's scratch dir so ``image_path`` points at bytes the
PUBLIC bytes route actually serves (probed here over HTTP with the
right Content-Type). Yields the identity map for the assertions;
truncates on teardown."""
try:
body = httpx.get(f"{app_url}/api/health", timeout=5).json()
except Exception: # noqa: BLE001 — unreachable is the skip case
body = {"db": "down"}
if body.get("db") != "up":
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
_truncate_kb()
scratch = tmp_path_factory.mktemp("bor_prose_wrap")
png = scratch / "landscape.png"
png.write_bytes(PNG_1X1)
with SessionLocal() as db:
image_doc = Document(
source=SOURCE,
path=IMAGE_PATH,
full_path=str(png),
title="Landscape Screenshot",
content=DESCRIPTION,
summary=DESCRIPTION, # the importer invariant: summary IS the description
is_image=True,
image_path=str(png),
content_hash="a1" * 32,
indexed_at=datetime.now(UTC),
)
db.add(image_doc)
db.add(
Document(
source=SOURCE,
path=TXT_PATH,
full_path="/tmp/prose-long.txt",
title="Prose Long Txt",
content=TXT_CONTENT,
content_hash="b2" * 32,
indexed_at=datetime.now(UTC),
)
)
db.add(
Document(
source=SOURCE,
path=CODE_PATH,
full_path="/tmp/code-long.py",
title="Code Long Py",
content=CODE_LINE + "\n",
content_hash="c3" * 32,
indexed_at=datetime.now(UTC),
)
)
db.commit()
image_url = f"/api/documents/{image_doc.id}/image"
# The seeded image_url serves REAL bytes (phase 122's route is
# public — phase 124 A4): the render assertions below are about a
# genuinely delivered image, never a broken one.
served = httpx.get(f"{app_url}{image_url}", timeout=10)
assert served.status_code == 200, "the seeded image bytes must serve through the route"
assert served.headers["content-type"] == "image/png"
yield {
"image_url": image_url,
"description": DESCRIPTION,
}
_truncate_kb()
# ---------------------------------------------------------------------------
# Surface helpers
# ---------------------------------------------------------------------------
def _drill(page: Page, *names: str) -> None:
"""Phase 97: drill the catalogue tree (exact name match, one per
name) to the level that holds the asserted file (the
test_document_viewer idiom)."""
for name in names:
page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
def _open_doc_page(page: Page, app_url: str, doc_path: str) -> None:
"""The STANDALONE surface: sign in, then the direct
/document.html?source=…&path=… URL (the same percent-encoding the
chips and row links carry)."""
login(page, app_url, next="/")
page.goto(
f"{app_url}/document.html?source={SOURCE}&path={quote(doc_path, safe='')}",
timeout=30_000,
)
def _open_doc_modal(page: Page, app_url: str, path_needle: str) -> None:
"""The IN-PAGE modal surface: the Sources row's path link (phase
26 — no navigation, the modal opens over the catalogue)."""
login(page, app_url)
_drill(page, "docs", "notes")
row = page.locator("#docs-tbody tr", has_text=path_needle)
expect(row).to_have_count(1)
row.locator("td:nth-child(2) a.doc-link").click()
expect(page.locator("#doc-modal")).to_be_visible(timeout=30_000)
def _assert_wrapping_prose(page: Page, content_sel: str, full_text: str) -> dict[str, object]:
"""The phase-165 prose contract on ONE surface's content host:
* exactly ONE ``.doc-prose`` block, NO ``pre.doc-raw`` sibling;
* the FULL text — to_have_text (whitespace-normalized) AND the raw
``textContent`` byte-for-byte (nothing clamped, nothing dropped);
* ``scrollWidth <= clientWidth + 1`` — THE TODO's literal defect:
no horizontal overflow, no sideways scrollbar (an 858-char
unwrapped line would report ~6000px here — this bar alone proves
the wrap once the full raw text is present);
* ``height > 2.2 × line-height`` — the block stands several line
boxes tall (the task's proof bar; with the two bars above it
means the FULL text occupies real wrapped lines, not a clipped
or scrolled single line);
* computed ``white-space == "pre-wrap"`` and ``max-width``
resolving to the 72rem reading column (1152px at the 16px root),
never wider than the content host.
"""
prose = page.locator(f"{content_sel} .doc-prose")
expect(prose).to_have_count(1)
expect(prose).to_have_text(full_text, timeout=30_000)
expect(page.locator(f"{content_sel} pre.doc-raw")).to_have_count(0)
m = prose.evaluate(
"""(el) => {
const host = el.parentElement;
const cs = getComputedStyle(el);
return {
raw: el.textContent,
inner: el.innerText,
scrollWidth: el.scrollWidth,
clientWidth: el.clientWidth,
height: el.getBoundingClientRect().height,
width: el.getBoundingClientRect().width,
hostWidth: host.getBoundingClientRect().width,
whiteSpace: cs.whiteSpace,
maxWidth: cs.maxWidth,
lineHeight: parseFloat(cs.lineHeight),
};
}"""
)
assert m["raw"] == full_text, "the prose block must hold the FULL content (raw equality)"
assert m["scrollWidth"] <= m["clientWidth"] + 1, (
f"horizontal overflow — the TODO defect is back: {m}"
)
assert m["lineHeight"] > 0, f"line-height did not resolve to px: {m}"
assert m["height"] > 2.2 * m["lineHeight"], (
f"the block is {m['height']}px tall, want > 2.2 line-heights "
f"({2.2 * m['lineHeight']:.0f}px) — the text did not wrap"
)
assert m["whiteSpace"] == "pre-wrap", f"white-space is {m['whiteSpace']!r}, not pre-wrap"
assert m["maxWidth"] == "1152px", (
f"max-width is {m['maxWidth']!r}, want 1152px (the 72rem --chat-column)"
)
assert m["width"] <= m["hostWidth"] + 1, (
f"the block ({m['width']}px) is wider than its content host ({m['hostWidth']}px)"
)
return m
def _assert_code_scrolls(page: Page, content_sel: str) -> None:
"""The locked code contract (phase 116 A1 / phase 165 A3) on ONE
surface: the code doc renders ``pre.doc-raw`` with
``white-space: pre`` + ``overflow-x: auto`` and REAL horizontal
overflow — no prose block anywhere."""
pre = page.locator(f"{content_sel} pre.doc-raw")
expect(pre).to_have_count(1, timeout=30_000)
expect(pre).to_have_text(CODE_LINE + "\n", timeout=30_000)
expect(page.locator(f"{content_sel} .doc-prose")).to_have_count(0)
dims = pre.evaluate(
"""(el) => {
const cs = getComputedStyle(el);
return {
sw: el.scrollWidth,
cw: el.clientWidth,
ws: cs.whiteSpace,
ox: cs.overflowX,
};
}"""
)
assert dims["sw"] > dims["cw"], f"the code line stopped overflowing: {dims}"
assert dims["ws"] == "pre", f"code white-space is {dims['ws']!r}, not pre (A3 broken)"
assert dims["ox"] == "auto", f"overflow-x is {dims['ox']!r}, not auto"
# ---------------------------------------------------------------------------
# 1–2. The image doc: the vision description wraps on both surfaces
# ---------------------------------------------------------------------------
def test_image_doc_description_wraps_on_the_page(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
"""The standalone /document.html page: the ``is_image`` doc renders
the ``<img>`` (from the served bytes route) ABOVE the prose block
(the phase-122 ordering), the labeled Summary panel stays
SUPPRESSED (summary === content), and the full ~860-char
description reads as wrapping prose — zero horizontal overflow,
pre-wrap, the 72rem column, no ``pre.doc-raw`` anywhere."""
page.set_default_timeout(30_000)
_open_doc_page(page, app_url, IMAGE_PATH)
expect(page.locator("#doc-title")).to_have_text("Landscape Screenshot")
_assert_wrapping_prose(page, "#doc-content", seeded["description"])
# The image is served AND rendered, and it sits ABOVE the prose
# (phase 122's ordering contract — the image first, the wrapping
# description below it).
img = page.locator("#doc-content .doc-image-img")
expect(img).to_have_count(1)
expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("alt", seeded["description"])
expect(img).to_be_visible()
assert img.evaluate("el => el.naturalWidth") == 1, "the bytes route must deliver real PNG bytes"
assert page.evaluate(
"""() => {
const host = document.querySelector('#doc-content');
const img = host.querySelector('.doc-image-img');
const prose = host.querySelector('.doc-prose');
return (img.compareDocumentPosition(prose)
& Node.DOCUMENT_POSITION_FOLLOWING) !== 0;
}"""
), "the <img> must render ABOVE the prose block (phase 122 ordering)"
# The verbatim-description case keeps the Summary panel suppressed
# (the panel would duplicate the text right below the image).
expect(page.locator("#doc-content .doc-summary")).to_have_count(0)
expect(page.locator("#doc-content .doc-image-unavailable")).to_have_count(0)
def test_image_doc_description_wraps_in_the_modal(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
"""The SAME doc through the Sources-row modal: the identical prose
contract on ``#doc-modal-content`` — one shared renderDocument core
means the modal wraps exactly like the page (no per-surface drift),
inside the narrower (1100px) panel."""
page.set_default_timeout(30_000)
_open_doc_modal(page, app_url, "landscape.png")
expect(page.locator("#doc-modal-title")).to_have_text("Landscape Screenshot")
_assert_wrapping_prose(page, "#doc-modal-content", seeded["description"])
img = page.locator("#doc-modal-content .doc-image-img")
expect(img).to_have_count(1)
expect(img).to_have_attribute("src", seeded["image_url"])
assert img.evaluate("el => el.naturalWidth") == 1
assert page.evaluate(
"""() => {
const host = document.querySelector('#doc-modal-content');
const img = host.querySelector('.doc-image-img');
const prose = host.querySelector('.doc-prose');
return (img.compareDocumentPosition(prose)
& Node.DOCUMENT_POSITION_FOLLOWING) !== 0;
}"""
), "the modal must keep the <img> ABOVE the prose block too"
expect(page.locator("#doc-modal-content .doc-summary")).to_have_count(0)
# ---------------------------------------------------------------------------
# 3–4. The .txt doc: a 860-char line wraps, the paragraph break survives
# ---------------------------------------------------------------------------
def _assert_paragraph_break(m: dict[str, object]) -> None:
"""The SECOND paragraph proof (beyond the wrap proof in
_assert_wrapping_prose): the raw text node keeps the ``\\n\\n``
verbatim, and the presentation text (``innerText``, which in a
pre-wrap context carries the SOURCE hard breaks — soft wraps are
rendered line boxes, not innerText newlines) still shows the
three-paragraph source shape: long line, BLANK line, second
paragraph. The intra-line wrap itself is the scrollWidth/height
proof above — the 858-char source line is one innerText line that
measurably does NOT overflow horizontally."""
raw = str(m["raw"])
inner = str(m["inner"])
assert "\n\n" in raw, "the raw text node must keep the paragraph break"
assert "\n\n" in inner, (
"the pre-wrap presentation text must keep the blank paragraph line"
)
lines = inner.splitlines()
assert len(lines) == 3 and lines[1] == "", (
f"the paragraph shape is not (line, blank, paragraph): {lines}"
)
def test_txt_doc_wraps_on_the_page(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
# `seeded` is the module's seed+truncate barrier (this test reads
# the txt row it creates, not its return value).
"""The standalone page, the ``txt`` prose arm: the ~860-char line
wraps inside the reading column (zero horizontal overflow — the
TODO's literal ask), the full body is present raw, the paragraph
break renders as a blank line, ``pre-wrap`` + the 72rem column are
in effect, and no ``pre.doc-raw`` remains for a prose format."""
page.set_default_timeout(30_000)
_open_doc_page(page, app_url, TXT_PATH)
expect(page.locator("#doc-title")).to_have_text("Prose Long Txt")
expect(page.locator("#doc-meta .format-badge")).to_have_text("txt")
m = _assert_wrapping_prose(page, "#doc-content", TXT_CONTENT)
_assert_paragraph_break(m)
def test_txt_doc_wraps_in_the_modal(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
# `seeded`: the module's seed barrier (see test 3).
"""The same ``.txt`` doc through the modal: identical wrapping on
``#doc-modal-content`` (the shared core, the shared CSS)."""
page.set_default_timeout(30_000)
_open_doc_modal(page, app_url, "prose-long.txt")
expect(page.locator("#doc-modal-title")).to_have_text("Prose Long Txt")
m = _assert_wrapping_prose(page, "#doc-modal-content", TXT_CONTENT)
_assert_paragraph_break(m)
# ---------------------------------------------------------------------------
# 5. The negative control: code keeps the horizontal scroller (locked A1/A3)
# ---------------------------------------------------------------------------
def test_code_doc_still_scrolls_horizontally(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
# `seeded`: the module's seed barrier (see test 3).
"""The wrapping fix is provably SCOPED: the ~900-char code line
still lands in ``pre.doc-raw`` and STILL overflows horizontally
with ``white-space: pre`` + ``overflow-x: auto`` on BOTH surfaces
— phase 116's locked A1 ("wrapping breaks code readability") alive
end-to-end, not just in the CSS pins (phase 165 D3)."""
page.set_default_timeout(30_000)
# Surface 1: the standalone page.
_open_doc_page(page, app_url, CODE_PATH)
expect(page.locator("#doc-title")).to_have_text("Code Long Py")
expect(page.locator("#doc-meta .format-badge")).to_have_text("py")
_assert_code_scrolls(page, "#doc-content")
# Surface 2: the in-page modal (the phase-116 themed scrollbar
# owner — tests/e2e/test_document_viewer.py pins the bar itself;
# here the overflow contract is what must not have drifted).
_open_doc_modal(page, app_url, "code-long.py")
expect(page.locator("#doc-modal-title")).to_have_text("Code Long Py")
_assert_code_scrolls(page, "#doc-modal-content")
+74 -15
View File
@@ -27,7 +27,10 @@ the short-page stretch. This suite proves the BROWSER behavior:
* the short-page layout is EXACTLY as it was: body stretched to the * the short-page layout is EXACTLY as it was: body stretched to the
viewport, footer pinned to the viewport bottom, the phase-52 viewport, footer pinned to the viewport bottom, the phase-52
pinned composer (``position: sticky; bottom: env(safe-area-inset- pinned composer (``position: sticky; bottom: env(safe-area-inset-
bottom)``) resting in its in-flow slot above the footer. bottom)``) resting in its in-flow slot above the footer. The pin
is POST-boot (phase 88 keeps the cluster static until app.js adds
``#view-chat.chat-booted``), so test 3 waits for that sentinel
before reading it — the house boot-settle pattern.
Seeding (A3): the long-scroll surfaces are the seeded Sources table Seeding (A3): the long-scroll surfaces are the seeded Sources table
(``tests/fixtures/docs/`` — 13 docs, same harness as the phase-10/26 (``tests/fixtures/docs/`` — 13 docs, same harness as the phase-10/26
@@ -37,14 +40,27 @@ short unique docs + one ``doc-long.md`` with a ~40,000-char body).
mock embeddings server (the ``test_document_viewer.py`` pattern) — no mock embeddings server (the ``test_document_viewer.py`` pattern) — no
real LLM is involved. real LLM is involved.
Phase 156 re-target (owner-locked A1): the catalogue card became the
tables' OWN bounded scroller (``.table-wrap`` max-height
min(70vh, 64rem)), so a Sources file level no longer grows the
DOCUMENT past a full viewport — its scroll is the residual chrome
height. Test 1 therefore proves the stick across the page's REAL
scroll range (mid + bottom, card scroller pinned too); the DEEP-scroll
proof (the audit's ~−1264px repro, scroll far past one viewport) is
test 2's long-document surface, which still grows the document many
viewports.
Test → story mapping (Playwright Mapping Rule): Test → story mapping (Playwright Mapping Rule):
1. ``test_app_header_stuck_at_top_on_sources`` 1. ``test_app_header_stuck_at_top_on_sources`` — stuck mid-page AND at
2. ``test_doc_header_stuck_at_top_on_long_document`` the page bottom; the 41-row table scrolls inside the phase-156 card
2. ``test_doc_header_stuck_at_top_on_long_document`` — stuck at the
bottom of a many-viewport scroll (the deep-scroll proof)
3. ``test_short_page_stretch_and_resting_composer_intact`` 3. ``test_short_page_stretch_and_resting_composer_intact``
""" """
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import re
from collections.abc import Iterator from collections.abc import Iterator
from pathlib import Path from pathlib import Path
from threading import Thread from threading import Thread
@@ -223,26 +239,49 @@ def test_app_header_stuck_at_top_on_sources(
page.click('#folders-tbody a.folder-link:text-is("gen")') page.click('#folders-tbody a.folder-link:text-is("gen")')
expect(page.locator("#docs-tbody tr")).to_have_count(TOTAL_DOCS - 13) expect(page.locator("#docs-tbody tr")).to_have_count(TOTAL_DOCS - 13)
# The precondition the story needs: the page must actually scroll — # Phase 156 (owner-locked A1): the 41-row table scrolls INSIDE the
# fail loudly if the table ever stops being long enough. # bounded card (max-height min(70vh, 64rem)) — the card is the rows'
# scroller, and the DOCUMENT keeps the residual page scroll (header
# / stats / toolbar / padding / footer). Preconditions, both fail
# loudly: the card must actually scroll (every row reachable), and
# the page must still scroll at all.
sh = page.evaluate("() => document.documentElement.scrollHeight") sh = page.evaluate("() => document.documentElement.scrollHeight")
assert sh > 1.5 * VIEWPORT_H, ( assert sh > VIEWPORT_H + 200, (
f"the Sources table must be long enough to scroll " f"the Sources page must be scrollable (scrollHeight={sh}, want > "
f"(scrollHeight={sh}, want > {1.5 * VIEWPORT_H:.0f} at a " f"{VIEWPORT_H + 200} at a {VIEWPORT_W}×{VIEWPORT_H} viewport) — "
f"{VIEWPORT_W}×{VIEWPORT_H} viewport) — add more generated docs" "the page collapsed to a fixed layout"
) )
card = page.evaluate(
"""() => {
const c = document.querySelector(
".table-wrap[aria-label='Indexed documents']");
return c ? { sh: c.scrollHeight, ch: c.clientHeight } : null;
}"""
)
assert card is not None and card["sh"] > card["ch"], (
f"the phase-156 card must be the table's scroller "
f"(scrollHeight={None if card is None else card['sh']}, "
f"clientHeight={None if card is None else card['ch']})"
)
max_scroll = sh - VIEWPORT_H
# Mid-page scroll (~1 viewport down — the pre-fix un-pin point): the # Mid-page scroll (half the document's available scroll): the
# navbar is still stuck at the top. # navbar is still stuck at the top. The DEEP-scroll proof — a scroll
page.evaluate("() => window.scrollTo(0, 800)") # far past one viewport, the audit's ~−1264px repro — is test 2's
# long-document surface: phase 156's bounded card left Sources as
# the residual-scroll page.
page.evaluate(f"() => window.scrollTo(0, {max_scroll // 2})")
assert ( assert (
page.evaluate("() => window.scrollY") >= VIEWPORT_H - TOP_TOL_PX abs(page.evaluate("() => window.scrollY") - max_scroll // 2) <= TOP_TOL_PX
), "the test scroll must land" ), "the test scroll must land"
_assert_top_is_zero(page, ".app-header") _assert_top_is_zero(page, ".app-header")
# And at the very bottom (the audit's −1264px repro): stuck, and # And at the very bottom of the page scroll: stuck, and still
# still visible. # visible.
page.evaluate("() => window.scrollTo(0, 999999)") page.evaluate("() => window.scrollTo(0, 999999)")
assert page.evaluate("() => window.scrollY") >= max_scroll - TOP_TOL_PX, (
"the test scroll must land at the document bottom"
)
_assert_top_is_zero(page, ".app-header") _assert_top_is_zero(page, ".app-header")
expect(page.locator(".app-header")).to_be_visible() expect(page.locator(".app-header")).to_be_visible()
@@ -290,12 +329,32 @@ def test_doc_header_stuck_at_top_on_long_document(
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _wait_chat_booted(page: Page) -> None:
"""The house boot-settle wait (test_chip_sizing_question_cap.py /
test_mobile_chat_hamburger_boot.py): app.js adds
``#view-chat.chat-booted`` two frames AFTER the boot settles. The
phase-88 pre-boot rule
(``#view-chat:not(.chat-booted) .composer { position: static }``)
keeps the cluster NON-sticky until then — reading the pin before
the sentinel lands races app boot (the flake this removes: the
sticky assertion measured a pre-boot static composer)."""
page.wait_for_function(
"() => document.getElementById('view-chat')?."
"classList.contains('chat-booted')",
timeout=15_000,
)
expect(page.locator("#view-chat")).to_have_class(re.compile(r"\bchat-booted\b"))
def test_short_page_stretch_and_resting_composer_intact( def test_short_page_stretch_and_resting_composer_intact(
page: Page, app_url: str page: Page, app_url: str
) -> None: ) -> None:
page.set_default_timeout(30_000) page.set_default_timeout(30_000)
page.goto(app_url) page.goto(app_url)
expect(page.locator("#empty-state")).to_be_visible() expect(page.locator("#empty-state")).to_be_visible()
# The pin under test only exists POST-boot (phase 88 defers the
# sticky promotion) — settle the boot BEFORE any measurement.
_wait_chat_booted(page)
# The stretch holds: the body is exactly one viewport tall, and the # The stretch holds: the body is exactly one viewport tall, and the
# grow must not invent scrollable space. # grow must not invent scrollable space.
+292
View File
@@ -83,6 +83,17 @@ loosened):
bar, the phase-12/34 breach) + the ``#doc-copy-announcer`` bar, the phase-12/34 breach) + the ``#doc-copy-announcer``
(role=status) + the ``#doc-meta`` row the shared core builds the (role=status) + the ``#doc-meta`` row the shared core builds the
control into. control into.
Phase 165 (task 01) — the prose content block (TODO.md L3): the
prose-set constant (``PROSE_FORMATS`` + ``isProseDoc`` — the ONE
single-site rule), ``renderDocument``'s three-way content branch
(markdown → ``.doc-md``, prose (image descriptions, txt/text) →
``.doc-prose`` via textContent (never innerHTML), everything else →
``.doc-raw`` untouched), the ``isProseDoc`` truth table and the
format table as node behavioral runs over the real module, and the
``.doc-prose`` CSS (the 72rem reading column + ``pre-wrap`` +
``overflow-wrap: anywhere``, tokens only) beside the ``.doc-raw``
contract that stays locked (phase 116 A1 / phase 165 A3).
""" """
from __future__ import annotations from __future__ import annotations
@@ -1810,3 +1821,284 @@ def test_page_render_wires_the_copy_path_with_the_composed_string() -> None:
assert out["failOut"]["selectCount"] == 1 assert out["failOut"]["selectCount"] == 1
assert out["failOut"]["status"] == "Path ready — copy it from the field." assert out["failOut"]["status"] == "Path ready — copy it from the field."
assert out["failOut"]["className"] == "doc-copy-pop" # the floating block assert out["failOut"]["className"] == "doc-copy-pop" # the floating block
# ---------------------------------------------------------------------------
# Phase 165 (task 01) — the prose content block: PROSE_FORMATS /
# isProseDoc / renderDocument's third branch / the .doc-prose CSS
# (TODO.md L3 — image descriptions and txt/text docs WRAP)
# ---------------------------------------------------------------------------
STYLES_CSS = FRONTEND / "assets" / "styles.css"
def _content_branch(js: str) -> str:
"""The THREE-WAY content branch of renderDocument: from the
markdown ``if`` to the close of the raw ``else`` (the exact end
anchor: the raw arm's ``slot.appendChild(pre);`` + its closing
brace — the _stmt_block brace-walk would swallow the trailing
strip-mount statement, so this slice is literal)."""
start = js.index("if (doc.format === \"md\" || doc.format === \"markdown\") {")
end = js.index("slot.appendChild(pre);\n }") + len("slot.appendChild(pre);\n }")
return js[start:end]
def _prose_branch(js: str) -> str:
"""The prose arm alone: ``} else if (isProseDoc(doc)) {`` to its
closing brace after ``slot.appendChild(prose);``."""
start = js.index("} else if (isProseDoc(doc)) {")
end = js.index("slot.appendChild(prose);\n }") + len("slot.appendChild(prose);\n }")
return js[start:end]
def _prose_rule(js: str) -> str:
"""The phase-165 prose rule at module scope: the PROSE_FORMATS
const through the close of isProseDoc (extracted together — the
function references the const; the _fn brace-walk house pattern)."""
start = js.index("const PROSE_FORMATS")
end = js.index("}", js.index("function isProseDoc")) + 1
return js[start:end]
def test_prose_set_is_one_named_constant() -> None:
"""LOCKED A1: the prose rule lives in ONE named constant pair —
``const PROSE_FORMATS = new Set(["txt", "text"])`` +
``isProseDoc(doc)`` with the image arm UNCONDITIONAL (phase 122
A3 — an image doc's content IS the vision description, prose by
construction). Grep-verified single-site rule: the content-branch
block itself names no format literal — it calls isProseDoc."""
js = _read(DOCUMENT_JS)
rule = _prose_rule(js)
assert 'const PROSE_FORMATS = new Set(["txt", "text"]);' in rule
assert "doc.is_image === true || PROSE_FORMATS.has(doc.format)" in rule
branch = _content_branch(js)
assert "isProseDoc(doc)" in branch, "the prose arm gates on the ONE helper"
assert '"txt"' not in branch and '"text"' not in branch, (
"no second format literal in the content branch (single-site rule)"
)
def test_content_branch_is_three_way_md_prose_raw() -> None:
"""renderDocument's content branch is markdown → prose → raw, in
that order, all three appending into the SAME ``slot`` (phase 122
image block FIRST, phase 36 summary between, phase 163 additive
strip last — all unchanged). The markdown and raw arms stay
byte-identical to pre-phase (locked A3); the prose arm is
textContent-only — the XSS contract (never innerHTML for document
content) is PINNED on the branch source."""
js = _read(DOCUMENT_JS)
render = _fn(js, "renderDocument")
md_i = render.find('if (doc.format === "md" || doc.format === "markdown") {')
prose_i = render.find("} else if (isProseDoc(doc)) {", md_i)
raw_i = render.find("} else {", prose_i)
assert -1 < md_i < prose_i < raw_i, "the prose arm sits BETWEEN markdown and raw"
# The untouched markdown / raw arms (byte-identical contracts).
for stmt in (
'wrap.className = "doc-md";',
"wrap.innerHTML = renderMarkdown(doc.content);",
"slot.appendChild(wrap);",
'pre.className = "doc-raw";',
"pre.textContent = doc.content;",
"slot.appendChild(pre);",
):
assert stmt in render, f"the pre-phase arms must keep {stmt!r}"
# The prose arm: div + class + text node + the same slot.
prose_branch = _prose_branch(js)
assert 'document.createElement("div")' in prose_branch
assert 'prose.className = "doc-prose";' in prose_branch
assert "prose.textContent = doc.content;" in prose_branch
assert "slot.appendChild(prose);" in prose_branch
assert "innerHTML" not in prose_branch, (
"the prose arm never touches innerHTML (the XSS contract)"
)
#: (doc, expected isProseDoc) — LOCKED A1's exact set:
#: {is_image: true} ∪ {format ∈ txt, text}; code, code-ish, pdf,
#: unknown, extension-less (no format) and the string-``"true"``
#: lookalike all stay on the scrolling side (locked A3).
_PROSE_CASES: list[tuple[dict[str, object], bool]] = [
({"is_image": True, "format": "png"}, True),
({"is_image": True, "format": "webp"}, True),
({"is_image": True, "format": "md"}, True), # the image arm wins
({"is_image": True, "format": "pdf"}, True), # …regardless of format
({"is_image": True}, True), # …even without one
({"format": "txt"}, True),
({"format": "text"}, True),
({"is_image": False, "format": "txt"}, True),
({"is_image": False, "format": "text"}, True),
({"is_image": "true", "format": "py"}, False), # strict === true
({"is_image": False, "format": "py"}, False),
({"is_image": False, "format": "yaml"}, False),
({"is_image": False, "format": "yml"}, False),
({"is_image": False, "format": "json"}, False),
({"is_image": False, "format": "j2"}, False),
({"is_image": False, "format": "container"}, False), # quadlet family
({"is_image": False, "format": "network"}, False),
({"is_image": False, "format": "volume"}, False),
({"is_image": False, "format": "kube"}, False),
({"is_image": False, "format": "pdf"}, False), # phase 165 A4 (deferred)
({"is_image": False, "format": "md"}, False),
({"is_image": False, "format": "markdown"}, False),
({"is_image": False, "format": "dockerfile"}, False),
({"is_image": False, "format": "rst"}, False), # unknown suffix
({"is_image": False, "format": ""}, False),
({"is_image": False}, False), # extension-less (no format at all)
]
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_is_prose_doc_truth_table() -> None:
"""Behavioral (node): the REAL isProseDoc over the full truth
table — a future format cannot drift into prose by accident
(phase 165 D3's pin)."""
js = _read(DOCUMENT_JS)
script = (
_prose_rule(js)
+ "\nconst cases = "
+ json.dumps([c for c, _ in _PROSE_CASES])
+ ";\nconsole.log(JSON.stringify(cases.map((d) => isProseDoc(d))));\n"
)
assert json.loads(_run_node(script)) == [e for _, e in _PROSE_CASES]
# The renderDocument content-branch harness: the minimal fake DOM of
# test_viewer_additive_toggle.py (the house pattern), the real
# docImageBlock standing in for the image arm's first block.
_CONTENT_BRANCH_FAKE_DOM = """
function makeEl(tag) {
return {
tag, className: "", textContent: "", innerHTML: "", attrs: {}, children: [],
setAttribute(k, v) { this.attrs[k] = String(v); },
getAttribute(k) { return k in this.attrs ? this.attrs[k] : null; },
appendChild(c) { this.children.push(c); return c; },
append(...cs) { this.children.push(...cs); },
replaceChildren(...cs) { this.children = cs; },
addEventListener() {},
};
}
globalThis.document = { createElement: (t) => makeEl(t) };
function fmtDate(x) { return "DATE:" + x; }
function metaBadge(cls, text, title) {
const s = makeEl("span");
s.className = cls;
s.textContent = text;
if (title !== undefined) s.setAttribute("title", title);
return s;
}
function copyPathButton() { return makeEl("button"); }
function originAnchor() { return makeEl("a"); }
async function docAdminReady() { return false; }
function wireDateEdit() {}
function wireSummaryEdit() {}
function renderMarkdown(s) { return s; }
function docPagesStrip() { return makeEl("div"); }
"""
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_render_document_content_branch_format_table() -> None:
"""Behavioral (node, fake DOM): the REAL renderDocument per
format — txt/text → .doc-prose (the TODO's wrapping ask), md /
markdown → .doc-md (the escape-first renderer, unchanged), and
py/yaml/json/j2/quadlet/pdf/""/unknown → .doc-raw (locked A1/A3:
the code scroller unchanged). An is_image doc lands in
.doc-prose whatever its format — never a doc-raw sibling."""
js = _read(DOCUMENT_JS)
script = (
_CONTENT_BRANCH_FAKE_DOM
+ _prose_rule(js)
+ _fn(js, "renderDocument")
+ _fn(js, "docImageBlock")
+ """
const base = {
title: "T", path: "p", source: "S", content: "BODY", summary: null,
created_at: "c", indexed_at: "i", chunks: 1,
};
function classesFor(doc) {
const contentEl = makeEl("div");
renderDocument(doc, { titleEl: makeEl("h1"), metaEl: makeEl("div"), contentEl });
return contentEl.children.map((c) => c.className);
}
const out = {};
for (const format of ["md", "markdown", "txt", "text", "py", "yaml", "yml",
"json", "j2", "conf", "pdf", "container", "network", "volume", "kube",
"dockerfile", "rst", ""]) {
out[format] = classesFor({ ...base, format });
}
out["image:png"] = classesFor({ ...base, format: "png", is_image: true,
image_url: "/api/documents/x/image" });
console.log(JSON.stringify(out));
"""
)
got = json.loads(_run_node(script))
expected = {
"md": ["doc-md"],
"markdown": ["doc-md"],
"txt": ["doc-prose"],
"text": ["doc-prose"],
"py": ["doc-raw"],
"yaml": ["doc-raw"],
"yml": ["doc-raw"],
"json": ["doc-raw"],
"j2": ["doc-raw"],
"conf": ["doc-raw"],
"pdf": ["doc-raw"], # phase 165 A4: PDF text stays raw (owner call)
"container": ["doc-raw"],
"network": ["doc-raw"],
"volume": ["doc-raw"],
"kube": ["doc-raw"],
"dockerfile": ["doc-raw"],
"rst": ["doc-raw"],
"": ["doc-raw"],
"image:png": ["doc-image", "doc-prose"], # image block first, prose below
}
assert got == expected
def test_doc_prose_css_wraps_in_the_reading_column() -> None:
"""styles.css: the ``.doc-prose`` rule (immediately after
``.doc-raw``) is the .doc-md column with prose rhythm —
``white-space: pre-wrap`` + ``overflow-wrap: anywhere`` (the
TODO's literal no-horizontal-scroll ask) inside the 72rem
``var(--chat-column)`` reading column, phase-08 TOKENS only (zero
color literals — the :root-drift contract; --ink on --surface is
the documented AA pair), no animation. The ``.doc-raw`` rule
keeps the locked code contract (A3): ``white-space: pre`` +
``overflow-x: auto``."""
css = _read(STYLES_CSS)
raw_i = css.find(".doc-raw {")
prose_i = css.find(".doc-prose {")
assert -1 < raw_i < prose_i, ".doc-prose must sit immediately after .doc-raw"
# Provenance comment directly above the rule (house style) — it
# cites the TODO, the defect and the locked decisions.
comment_start = css.rfind("/*", 0, prose_i)
provenance = css[comment_start:prose_i]
assert -1 < comment_start < prose_i
for citation in ("phase 165", "TODO.md L3", ".doc-md", ".doc-raw", "A1", "A2"):
assert citation in provenance, f"the comment must cite {citation!r}"
assert "no animation" in provenance
rule = css[prose_i : css.find("}", prose_i)]
for decl in (
"width: 100%",
"max-width: var(--chat-column)",
"margin-inline: auto",
"background: var(--surface)",
"border: 1px solid var(--line)",
"border-radius: var(--radius)",
"box-shadow: var(--shadow)",
"padding: 1.5rem 1.75rem",
"color: var(--ink)",
"white-space: pre-wrap",
"overflow-wrap: anywhere",
"line-height: 1.55",
):
assert decl in rule, f".doc-prose must carry {decl!r}"
# Tokens only — zero color literals (the :root-drift test parses
# built-ins from this file; no new hue — the monochrome invariant).
assert not re.search(r"#[0-9a-fA-F]{3,8}\b", rule)
assert "rgb(" not in rule and "hsl(" not in rule and "color-mix(" not in rule
# The untouched code scroller (phase 116 locked A1, phase 165 A3).
raw_rule = css[raw_i : css.find("}", raw_i)]
assert "white-space: pre" in raw_rule
assert "overflow-x: auto" in raw_rule
+176 -3
View File
@@ -39,7 +39,10 @@ frontend source contracts: the viewer renders the ``<img>`` from
Sources row's 48px lazy thumbnail falls back to the document glyph on Sources row's 48px lazy thumbnail falls back to the document glyph on
a failed fetch (the glyph swap keeps the module's single-innerHTML a failed fetch (the glyph swap keeps the module's single-innerHTML
pin). The ext→mime map pin (task 03's ``IMAGE_MIMES`` — one map, one pin). The ext→mime map pin (task 03's ``IMAGE_MIMES`` — one map, one
truth with the serve route's ``Content-Type``). truth with the serve route's ``Content-Type``). Phase 165 (task 01,
TODO.md L3): the description now renders in the WRAPPING ``.doc-prose``
block, not the horizontally-scrolling ``.doc-raw`` pre — the node
behavioral pins for the image-doc content branch close this module.
Task 05 (the RAG display): the shared frame builder Task 05 (the RAG display): the shared frame builder
(``retriever.source_ref_with_image`` — the chat API's cited AND (``retriever.source_ref_with_image`` — the chat API's cited AND
@@ -1038,8 +1041,10 @@ def test_viewer_renders_the_image_block_from_image_url_with_alt_summary() -> Non
assert 'note.className = "doc-image-unavailable"' in js assert 'note.className = "doc-image-unavailable"' in js
assert 'note.setAttribute("role", "status")' in js assert 'note.setAttribute("role", "status")' in js
assert "Image unavailable" in js assert "Image unavailable" in js
# The description stays in the normal content slot below (the # The content slot below is a text node (never parsed as HTML) —
# existing plain-content path — doc-raw via textContent). # the raw arm's statement; phase 165 (task 01) moved the IMAGE
# description to the wrapping .doc-prose arm (the node behavioral
# pins at the end of this module).
assert "pre.textContent = doc.content" in js assert "pre.textContent = doc.content" in js
# ONE innerHTML in the module CODE (the XSS contract, unchanged — # ONE innerHTML in the module CODE (the XSS contract, unchanged —
# the escape-first markdown render; comments may name it, the # the escape-first markdown render; comments may name it, the
@@ -1489,3 +1494,171 @@ def test_chat_sources_figure_css_is_compact_and_themed() -> None:
"the hover is the flat underline (no background swap)" "the hover is the flat underline (no background swap)"
) )
assert "background" not in (hover.group(1) if hover else "") assert "background" not in (hover.group(1) if hover else "")
# ---------------------------------------------------------------------------
# Phase 165 (task 01) — the image-doc content branch renders the
# description as WRAPPING prose (.doc-prose), never the scrolling pre
# (TODO.md L3: "Need word wrap for image explanations ... rather than
# scrolling horizontally"). Node behavioral runs over the real
# renderDocument + docImageBlock on the minimal fake DOM (the
# test_document_viewer.py house pattern).
# ---------------------------------------------------------------------------
import json # noqa: E402
import shutil # noqa: E402
import subprocess # noqa: E402
HAVE_NODE = shutil.which("node") is not None
_IMAGE_FAKE_DOM = """
function makeEl(tag) {
return {
tag, className: "", textContent: "", innerHTML: "", attrs: {}, children: [],
setAttribute(k, v) { this.attrs[k] = String(v); },
getAttribute(k) { return k in this.attrs ? this.attrs[k] : null; },
appendChild(c) { this.children.push(c); return c; },
append(...cs) { this.children.push(...cs); },
replaceChildren(...cs) { this.children = cs; },
addEventListener() {},
};
}
globalThis.document = { createElement: (t) => makeEl(t) };
function fmtDate(x) { return "DATE:" + x; }
function metaBadge(cls, text, title) {
const s = makeEl("span");
s.className = cls;
s.textContent = text;
if (title !== undefined) s.setAttribute("title", title);
return s;
}
function copyPathButton() { return makeEl("button"); }
function originAnchor() { return makeEl("a"); }
async function docAdminReady() { return false; }
function wireDateEdit() {}
function wireSummaryEdit() {}
function renderMarkdown(s) { return s; }
function docPagesStrip() { return makeEl("div"); }
"""
def _run_node(script: str) -> str:
proc = subprocess.run(
["node", "-e", script], capture_output=True, text=True, timeout=60
)
assert proc.returncode == 0, f"node failed: {proc.stderr}"
return proc.stdout
def _fn(js: str, name: str) -> str:
"""The source of a top-level function (paren-walk THEN brace-walk —
the test_viewer_additive_toggle.py pattern; the module's
_function_body starts at the first brace, which on renderDocument
is the destructured parameter, not the body)."""
start = js.find(f"function {name}(")
assert start != -1, f"{name}() must exist in document.js"
i = js.find("(", start)
depth = 0
close = i
while i < len(js):
if js[i] == "(":
depth += 1
elif js[i] == ")":
depth -= 1
if depth == 0:
close = i
break
i += 1
brace = js.find("{", close)
depth = 0
for j in range(brace, len(js)):
if js[j] == "{":
depth += 1
elif js[j] == "}":
depth -= 1
if depth == 0:
return js[start : j + 1]
raise AssertionError(f"unbalanced braces in {name}()")
def _image_render_script() -> str:
"""The real renderDocument + docImageBlock + the phase-165 prose
rule (the PROSE_FORMATS const, which isProseDoc reads — extracted
with the function because the const lives outside it)."""
js = DOCUMENT_JS.read_text(encoding="utf-8")
prose_start = js.index("const PROSE_FORMATS")
prose_end = js.index("}", js.index("function isProseDoc")) + 1
parts = [
js[prose_start:prose_end],
_fn(js, "renderDocument"),
_fn(js, "docImageBlock"),
]
return _IMAGE_FAKE_DOM + "\n".join(parts)
_DESCRIPTION = (
"A network diagram of the homelab VLANs: the core switch, the router, "
"and three labeled subnets with their gateways."
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_image_doc_content_branch_is_doc_prose() -> None:
"""The image doc's content branch (phase 165, LOCKED A1/A2): the
<img> block FIRST, then the description as a .doc-prose text node
— className "doc-prose", textContent the FULL description, and NO
doc-raw sibling (the pre was the defect: white-space: pre made
every generated line unbreakable). The verbatim-description
Summary suppression is unchanged (summary === content → no panel)."""
out = _run_node(
_image_render_script()
+ f"""
const doc = {{
title: "pic", path: "pic.png", source: "S", format: "png", is_image: true,
image_url: "/api/documents/x/image", summary: {json.dumps(_DESCRIPTION)},
content: {json.dumps(_DESCRIPTION)},
created_at: "c", indexed_at: "i", chunks: 2,
}};
const contentEl = makeEl("div");
renderDocument(doc, {{ titleEl: makeEl("h1"), metaEl: makeEl("div"), contentEl }});
console.log(JSON.stringify(contentEl.children.map(
(c) => ({{ className: c.className, text: c.textContent }}))));
"""
)
children = json.loads(out)
assert [c["className"] for c in children] == ["doc-image", "doc-prose"], (
"the image block stays FIRST; the description is prose (phase 165) "
"— no doc-raw sibling and no duplicate Summary panel"
)
assert children[1]["text"] == _DESCRIPTION, "the full description, as a text node"
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_image_doc_edited_summary_renders_above_the_prose_block() -> None:
"""The admin-edited-summary case (summary !== content): the
Summary panel renders between the image and the prose exactly as
before (phase 36/57 shape unchanged) — and the content still lands
in .doc-prose, never the pre."""
out = _run_node(
_image_render_script()
+ f"""
const doc = {{
title: "pic", path: "pic.png", source: "S", format: "png", is_image: true,
image_url: "/api/documents/x/image", summary: "An admin note.",
content: {json.dumps(_DESCRIPTION)},
created_at: "c", indexed_at: "i", chunks: 2,
}};
const contentEl = makeEl("div");
renderDocument(doc, {{ titleEl: makeEl("h1"), metaEl: makeEl("div"), contentEl }});
console.log(JSON.stringify(contentEl.children.map(
(c) => ({{ className: c.className, text: c.textContent }}))));
"""
)
children = json.loads(out)
assert [c["className"] for c in children] == [
"doc-image",
"doc-summary",
"doc-prose",
]
assert children[1]["text"] == "" # the panel body carries the summary text
assert children[2]["text"] == _DESCRIPTION
+409
View File
@@ -0,0 +1,409 @@
"""Unit: phase 165 task 02 — the word-wrap fix is PROVABLY SCOPED.
TODO.md L3 asks for word wrap on "image explanations and other long
text". Phase 116's LOCKED A1 keeps horizontal scroll (NO line-wrap) for
the code content — "wrapping breaks code readability" — so the fix must
wrap exactly the prose set and nothing else. These pins make a future
"we fixed wrapping" mean "we wrapped prose", never "we wrapped code" or
"we wrapped wide tables":
* **the prose set is pinned as data** — ``PROSE_FORMATS`` holds exactly
``"txt"`` and ``"text"`` (a set-membership assertion over the source
line: adding a format is a deliberate, test-editing act), and
``isProseDoc`` tests ``doc.is_image`` BEFORE the format arm (phase 122
A3 — an image doc's ``content`` IS the vision description, prose by
construction whatever its extension);
* **the code scrollers are unchanged** — ``.doc-raw`` keeps
``white-space: pre`` + ``overflow-x: auto``, ``.bubble pre`` keeps
``overflow-x: auto`` with NO ``white-space`` added (the chat code
block), and ``.md-table-wrap`` / ``.md-table`` still carry the
phase-44 declaration (``overflow-x: auto`` on the wrapper,
``min-width: max-content`` on the table) so wide tables still scroll
sideways inside the reading column;
* **phase 116's scrollbar theme still applies** — the four
``.doc-modal .doc-raw`` rules (``scrollbar-width``,
``scrollbar-color`` and the three ``::-webkit-scrollbar`` pseudos) are
all present and still SCOPED to the modal (no global scrollbar
restyle — the TODO's scope);
* **``.doc-prose`` never reaches for the raw scroller** — the block has
no ``overflow-x`` and no ``white-space: pre;``, only ``pre-wrap`` +
``overflow-wrap: anywhere`` (the wrapping contract);
* **the AA/contrast pin for the new block** — ``color: var(--ink)`` on
``background: var(--surface)`` (the documented 14.5:1 pair, PLAN
§7.2) and ZERO color literals in the block (the phase-92
zero-literal / monochrome invariant — the ``:root``-drift test parses
built-ins out of styles.css).
Repo-hygiene finding (task 02 step 3, ``grep -rn "doc-raw" frontend/``)
— recorded here so the check is repeatable and its result on record:
``doc-raw`` appears in exactly TWO writers, both expected. (1) The raw
arm of ``app.renderDocument`` in ``frontend/assets/document.js`` (the
ONE viewer core, ``pre.className = "doc-raw"``) plus its doc comments;
(2) ``frontend/assets/styles.css`` (the base rule, the mobile squeeze,
the phase-116 modal scrollbar rules and the ``[data-view="pages"]``
hide list). Every other hit in ``frontend/`` is a comment.
``frontend/doc-edit.html`` + ``assets/doc-edit.js`` do NOT render
document content at all — they fill a ``<textarea>`` (``bodyInput.value
= draft.body``, textContent for the status/error lines) for the admin
draft editor, so the body never becomes markup and there is no scroller
to wrap; ``frontend/shared.html`` + ``assets/shared.js`` render
CHAT history through ``renderMarkdown`` into ``.bubble`` (the shared
renderer's own ``.md-table-wrap`` / ``.bubble pre`` scrollers, asserted
byte-identical above), never ``.doc-raw`` / ``.doc-md`` /
``.doc-prose``. Neither is a second content renderer, so the prose rule
correctly lives in the ONE place: ``isProseDoc`` + ``.doc-prose``.
ASSUMPTION (phase 165 A4, owner call — recorded in ``00_phase.md``
"Deferred"): PDF-extracted text (``format == "pdf"``) is deliberately
OUT of the prose set and stays in ``.doc-raw``. Three completed suites
pin ``pre.doc-raw`` for PDFs (``test_pdf_pages_in_sources.py``,
``test_pdf_strip_additive_selfheal.py``, ``test_garbage_pdf.py``) and
the phase-163 strip contract depends on it; changing it is an owner
call, one line (``PROSE_FORMATS.add("pdf")`` + those pins) once made —
not an executor choice and not an oversight.
All assertions are static-source (the ``test_markdown_tables.py`` /
``test_date_editor.py`` house pattern): no app, no browser. The
measured wrapping proof is the phase's E2E suite
(``tests/e2e/test_prose_word_wrap.py``, task 03).
"""
from __future__ import annotations
import re
from pathlib import Path
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
DOCUMENT_JS = FRONTEND / "assets" / "document.js"
STYLES_CSS = FRONTEND / "assets" / "styles.css"
def _js() -> str:
assert DOCUMENT_JS.is_file(), f"missing frontend file: {DOCUMENT_JS}"
return DOCUMENT_JS.read_text(encoding="utf-8")
def _css() -> str:
assert STYLES_CSS.is_file(), f"missing frontend file: {STYLES_CSS}"
return STYLES_CSS.read_text(encoding="utf-8")
def _rule(selector: str, css: str | None = None) -> str:
"""The FULL rule for ``selector`` (selector text + its ``{…}``
block) matched at line start, so ``.doc-raw`` cannot accidentally
match ``.doc-modal .doc-raw`` or ``.doc-views … .doc-raw`` (the
``_rule_block`` brace-walk of ``test_wide_column_css.py``)."""
text = _css() if css is None else css
m = re.search(rf"^{re.escape(selector)} \{{", text, re.MULTILINE)
assert m, f"styles.css must define a rule for {selector}"
start = text.index("{", m.start())
depth = 0
for i in range(start, len(text)):
if text[i] == "{":
depth += 1
elif text[i] == "}":
depth -= 1
if depth == 0:
return text[m.start() : i + 1]
raise AssertionError(f"unbalanced braces in the {selector} rule")
def _decls(selector: str) -> str:
"""Only the declaration block of ``selector`` (the braces stripped)."""
rule = _rule(selector)
return rule[rule.index("{") + 1 : rule.rindex("}")]
def _rules() -> list[tuple[str, str]]:
"""Every (selector, declarations) pair in styles.css, comments
stripped (so a provenance comment naming a property never counts as
a declaration) — the scope pins below walk this list."""
text = re.sub(r"/\*.*?\*/", "", _css(), flags=re.DOTALL)
pairs: list[tuple[str, str]] = []
for m in re.finditer(r"([^{}]+)\{", text):
selector = re.sub(r"\s+", " ", m.group(1)).strip()
# An at-rule header (@media …) is a container, not a selector.
if not selector or selector.startswith("@"):
continue
start = m.end()
depth = 1
i = start
while i < len(text) and depth:
if text[i] == "{":
depth += 1
elif text[i] == "}":
depth -= 1
i += 1
pairs.append((selector, text[start : i - 1]))
return pairs
def _hide_list_rule() -> str:
"""The phase-163 ``[data-view="pages"]`` hide list — a MULTI-selector
rule (the selectors are comma-separated lines, ``.doc-prose``
joined it in task 01), so it is sliced from its first selector to
the ``display: none; }`` that closes it rather than matched with
the single-selector ``_rule`` helper."""
css = _css()
start = css.index('.doc-views[data-view="pages"] .doc-summary')
end = css.index("display: none; }", start) + len("display: none; }")
return css[start:end]
def _code(text: str) -> str:
"""Comment-free source: block comments (``/* … */``, the JSDoc
headers that legitimately NAME the classes they describe) and
whole-line ``//`` comments stripped — so a writer scan counts CODE
only, not prose about the code."""
no_block = re.sub(r"/\*.*?\*/", "", text, flags=re.DOTALL)
return "\n".join(
line for line in no_block.splitlines() if not line.lstrip().startswith("//")
)
def _prose_rule_js(js: str) -> str:
"""The phase-165 prose rule at module scope: ``const
PROSE_FORMATS`` through the close of ``isProseDoc`` (the
``test_document_viewer.py`` extraction — the function references
the const, so they are read as one unit)."""
start = js.index("const PROSE_FORMATS")
end = js.index("}", js.index("function isProseDoc")) + 1
return js[start:end]
# ---------- the prose set is pinned as data (LOCKED A1) ----------
def test_prose_formats_set_contains_exactly_txt_and_text() -> None:
"""``PROSE_FORMATS`` contains EXACTLY {"txt", "text"} — parsed out
of the source line as a set of quoted members, so adding (or
renaming) a format fails this test and has to be an intentional,
test-editing decision (phase 165 D3: "a future format cannot drift
into prose by accident"). Code, code-ish and pdf are NOT here."""
line = re.search(r"const PROSE_FORMATS = new Set\(\[(.*?)\]\);", _js())
assert line, "PROSE_FORMATS must be a Set literal at module scope"
members = set(re.findall(r'"([^"]*)"', line.group(1)))
assert members == {"txt", "text"}, (
f"the prose format set is {sorted(members)}, not exactly ['text', 'txt'] "
"(locked A1: image descriptions + plain-text prose only)"
)
# The literal is the only place those strings gate the content arm.
assert len(re.findall(r"const PROSE_FORMATS", _js())) == 1, (
"the prose set is declared ONCE (the single-site rule)"
)
def test_is_prose_doc_reads_is_image_before_the_format_arm() -> None:
"""``isProseDoc`` tests ``doc.is_image === true`` FIRST and the
``PROSE_FORMATS`` arm second (locked A1 + phase 122 A3): the image
arm is unconditional, so an image doc is prose whatever its
extension — and the strict ``=== true`` keeps a string lookalike
out (the truth table in test_document_viewer.py covers behavior)."""
rule = _prose_rule_js(_js())
returns = re.findall(r"return (.*?);", rule)
assert len(returns) == 1, f"isProseDoc is a one-return predicate, got {returns}"
body = returns[0]
image_i = body.index("doc.is_image")
format_i = body.index("PROSE_FORMATS.has(doc.format)")
assert image_i < format_i, "the is_image arm must be evaluated first"
assert body[image_i:].startswith("doc.is_image === true ||"), (
"the image arm is strict (=== true) and ORs into the format arm"
)
# ---------- the code scrollers are present and unchanged (A3) ----------
def test_doc_raw_keeps_the_locked_pre_and_horizontal_scroller() -> None:
""".doc-raw — the code/code-ish/unknown/PDF content block — is
byte-identical to the pre-phase declaration in the two properties
that ARE the contract: white-space: pre (no wrapping) and
overflow-x: auto (the scroller). Phase 116's locked A1, restated
verbatim as phase 165's locked A3."""
decls = _decls(".doc-raw")
assert re.search(r"white-space:\s*pre;", decls), (
".doc-raw must keep white-space: pre — code never wraps (locked A3)"
)
assert re.search(r"overflow-x:\s*auto", decls), (
".doc-raw must keep overflow-x: auto — the horizontal scroller"
)
assert "pre-wrap" not in decls, ".doc-raw must not gain pre-wrap"
def test_bubble_pre_scrolls_and_never_gains_a_white_space_rule() -> None:
"""The chat/table-surface code block: .bubble pre still scrolls
sideways (overflow-x: auto) and still declares NO white-space —
the phase-165 fix touched the viewer's prose arm only, so a code
span in a chat answer wraps exactly as it did before (wrapping
breaks code readability — the reason behind phase 116's locked A1)."""
decls = _decls(".bubble pre")
assert re.search(r"overflow-x:\s*auto", decls), (
".bubble pre must keep its horizontal scroller"
)
assert "white-space" not in decls, (
".bubble pre must carry no white-space declaration (unchanged)"
)
def test_markdown_table_scrollers_match_the_phase_44_declaration() -> None:
"""Wide GFM tables still scroll, never wrap: .md-table-wrap is
STILL the one-declaration scroller (overflow-x: auto) and
.md-table still carries the phase-44 task-03 set —
border-collapse + width: 100% + min-width: max-content + 0.9rem —
byte-for-byte, so the wrapper stays the real scroller and the
72rem reading column never breaks."""
assert _rule(".md-table-wrap") == ".md-table-wrap { overflow-x: auto; }", (
"the table wrapper must stay the single-declaration horizontal scroller"
)
decls = _decls(".md-table")
for declaration in (
"border-collapse: collapse",
"width: 100%",
"min-width: max-content",
"font-size: 0.9rem",
):
assert declaration in decls, f".md-table must keep {declaration!r}"
# ---------- phase 116's scrollbar theme still applies (A1) ----------
def test_phase_116_modal_scrollbar_rules_are_all_present_and_scoped() -> None:
"""The four .doc-modal .doc-raw rules — scrollbar-width: thin,
scrollbar-color: var(--scrollbar-thumb) var(--scrollbar-track),
and the ::-webkit-scrollbar / -track / -thumb pseudos — survive
this phase intact and stay SCOPED under .doc-modal (no global
scrollbar restyle). What still lands in .doc-raw still gets the
themed bar: the prose block has no scrollbar to theme because it
never overflows horizontally."""
css = _css()
firefox = _rule(".doc-modal .doc-raw")
assert re.search(r"scrollbar-width:\s*thin", firefox)
assert re.search(
r"scrollbar-color:\s*var\(--scrollbar-thumb\) var\(--scrollbar-track\)", firefox
)
for pseudo, needle in (
(".doc-modal .doc-raw::-webkit-scrollbar", "height: 8px"),
(".doc-modal .doc-raw::-webkit-scrollbar-track", "background: var(--scrollbar-track)"),
(".doc-modal .doc-raw::-webkit-scrollbar-thumb", "background: var(--scrollbar-thumb)"),
):
rule = _rule(pseudo, css)
assert needle in rule, f"{pseudo} must keep {needle!r}"
# Scope pin (phase 116: "no global scrollbar restyle"): every rule
# that themes a scrollbar — a scrollbar-color declaration or a
# ::-webkit-scrollbar pseudo — is scoped under `.doc-modal .doc-raw`.
themed = [
sel
for sel, decls in _rules()
if "scrollbar-color" in decls or "scrollbar" in sel
]
assert len(themed) == 4, f"expected the 4 modal scrollbar rules, got {themed}"
for selector in themed:
assert selector.startswith(".doc-modal .doc-raw"), (
f"scrollbar rule {selector!r} escaped the modal scope (phase 116 scope pin)"
)
# The prose block never asks for a themed bar (it has none to theme).
assert not [s for s, d in _rules() if ".doc-prose" in s and "scrollbar" in d + s], (
".doc-prose must carry no scrollbar theming"
)
# ---------- .doc-prose never reaches for the raw scroller ----------
def test_doc_prose_declares_no_horizontal_scroller_and_no_pre() -> None:
""".doc-prose is the OPPOSITE contract of .doc-raw: no overflow-x
of any kind, and white-space is pre-WRAP (never the bare ``pre``
that made every generated line unbreakable). The wrapping comes
from white-space: pre-wrap + overflow-wrap: anywhere inside the
72rem column — nothing else."""
decls = _decls(".doc-prose")
assert "overflow-x" not in decls, (
".doc-prose must never declare a horizontal scroller (the TODO's ask)"
)
assert re.search(r"white-space:\s*pre-wrap", decls)
assert not re.search(r"white-space:\s*pre;", decls), (
".doc-prose must not carry the bare pre (the defect this phase fixes)"
)
assert re.search(r"overflow-wrap:\s*anywhere", decls)
assert re.search(r"max-width:\s*var\(--chat-column\)", decls), (
"the prose block rides the 72rem reading column"
)
# ---------- the AA/contrast pin for the new block ----------
def test_doc_prose_uses_the_documented_aa_pair_and_zero_literals() -> None:
""".doc-prose pairs color: var(--ink) with background:
var(--surface) — the documented 14.5:1 pair (PLAN §7.2, WCAG AA) —
and carries NO color literal at all (no hex, no rgb()/rgba(), no
hsl(), no color-mix): the phase-92 zero-literal invariant that
keeps the block correct under every saved theme, and what the
:root-drift test relies on when it parses styles.css."""
decls = _decls(".doc-prose")
assert re.search(r"color:\s*var\(--ink\)", decls)
assert re.search(r"background:\s*var\(--surface\)", decls)
for pattern, name in (
(r"#[0-9a-fA-F]{3,8}\b", "hex"),
(r"\brgba?\(", "rgb()/rgba()"),
(r"\bhsla?\(", "hsl()/hsla()"),
(r"color-mix\(", "color-mix()"),
):
assert not re.search(pattern, decls), f".doc-prose must carry no {name} literal"
# ---------- the raw writers are the two expected ones (step 3) ----------
def test_doc_raw_writers_are_the_raw_branch_and_the_phase_116_css_only() -> None:
"""The task-02 hygiene pass as an executable pin: the only FILE that
writes the doc-raw element is document.js's raw arm (one
``className`` assignment, inside renderDocument), and the only
other doc-raw writers in frontend/ are styles.css rules (the base
rule, the mobile squeeze, the phase-116 modal scrollbar rules and
the [data-view="pages"] hide list). doc-edit.js renders a
<textarea> and shared.js renders chat bubbles — neither is a second
content renderer, so neither needs the prose rule (see the module
docstring for the full finding)."""
js = _js()
assert js.count('className = "doc-raw"') == 1, (
"document.js must assign doc-raw in exactly one place (the raw arm)"
)
# A WRITER is code that emits the class (JS className/innerHTML, an
# HTML class attribute) — a doc comment naming it is not one.
writer = re.compile(r'className\s*=\s*"doc-raw"|class="[^"]*\bdoc-raw\b')
for path in sorted(FRONTEND.rglob("*")):
if not path.is_file() or path.suffix not in {".js", ".html"}:
continue
writes = len(writer.findall(_code(path.read_text(encoding="utf-8"))))
if path == DOCUMENT_JS:
assert writes == 1, f"{path.name} must hold exactly the one raw arm"
else:
assert writes == 0, (
f"{path.name} must not write doc-raw — the writers are "
"renderDocument's raw arm and styles.css only"
)
# The two expected writers DO hold it (the pin is not vacuous).
assert "doc-raw" in _css(), "styles.css is the second expected doc-raw writer"
def test_prose_block_mounts_where_the_raw_block_did() -> None:
"""The prose arm appends into the SAME ``slot`` the raw arm uses,
so the phase-122 image block stays FIRST, the phase-36 summary
panel between, and the phase-163 additive strip last — the ordering
both surfaces rely on. The [data-view="pages"] hide list carries
.doc-prose alongside .doc-raw (the additive/PDF case's text view),
so Pages still hides the text view for a prose doc."""
js = _js()
for arm in ("wrap", "prose", "pre"):
assert js.count(f"slot.appendChild({arm});") == 1, (
f"the {arm} arm must mount into slot exactly once"
)
# One slot binding, three arms (the shared mount point), and the
# strip still lands in the wrapper last.
assert "let slot = contentEl;" in js, "the three arms share one slot binding"
assert "views.appendChild(docPagesStrip(doc, pageUrls));" in js
hide = _hide_list_rule()
for cls in (".doc-summary", ".doc-md", ".doc-raw", ".doc-prose", ".doc-image"):
assert cls in hide, f"the [data-view=\"pages\"] hide list must include {cls}"
assert hide.count("display: none;") == 1, "the hide list stays ONE grouped declaration"
+24 -6
View File
@@ -82,6 +82,15 @@ def _code(js: str) -> str:
return re.sub(r"//.*?$|/\*.*?\*/", "", js, flags=re.S | re.M) return re.sub(r"//.*?$|/\*.*?\*/", "", js, flags=re.S | re.M)
def _prose_rule(js: str) -> str:
"""The phase-165 prose rule (the PROSE_FORMATS const + isProseDoc,
module scope) — renderDocument's prose gate, needed beside the
extracted functions for the node runs."""
start = js.index("const PROSE_FORMATS")
end = js.index("}", js.index("function isProseDoc")) + 1
return js[start:end]
def _run_node(script: str) -> str: def _run_node(script: str) -> str:
proc = subprocess.run( proc = subprocess.run(
["node", "-e", script], capture_output=True, text=True, timeout=60 ["node", "-e", script], capture_output=True, text=True, timeout=60
@@ -193,8 +202,9 @@ def test_view_toggle_css() -> None:
style, never color alone, B5) + the CSS-driven view swap style, never color alone, B5) + the CSS-driven view swap
(``.doc-views .doc-pages`` hidden by default; ``[data-view= (``.doc-views .doc-pages`` hidden by default; ``[data-view=
"pages"]`` hides the Summary panel / .doc-md / .doc-raw / "pages"]`` hides the Summary panel / .doc-md / .doc-raw /
.doc-image and shows the strip as the grid). Provenance comment .doc-prose (phase 165, task 01) / .doc-image and shows the strip as
above the block (house style).""" the grid — one grouped declaration). Provenance comment above the
block (house style)."""
css = _read(STYLES_CSS) css = _read(STYLES_CSS)
toggle_i = css.find(".doc-view-toggle {") toggle_i = css.find(".doc-view-toggle {")
assert toggle_i != -1, "styles.css must style .doc-view-toggle" assert toggle_i != -1, "styles.css must style .doc-view-toggle"
@@ -212,10 +222,18 @@ def test_view_toggle_css() -> None:
"the pressed state is text weight + fill — never color alone (B5)" "the pressed state is text weight + fill — never color alone (B5)"
) )
# The CSS-driven swap: hidden strip by default; pages hides the # The CSS-driven swap: hidden strip by default; pages hides the
# text view and shows the strip. # text view and shows the strip. The hide list is ONE grouped
# declaration (phase 165: .doc-prose joins it — defensive today,
# load-bearing if the owner ever adds "pdf" to PROSE_FORMATS).
assert '.doc-views .doc-pages { display: none; }' in css assert '.doc-views .doc-pages { display: none; }' in css
for sel in (".doc-summary", ".doc-md", ".doc-raw", ".doc-image"): expected = ",\n".join(
assert f'.doc-views[data-view="pages"] {sel}' in css f'.doc-views[data-view="pages"] {sel}'
for sel in (".doc-summary", ".doc-md", ".doc-raw", ".doc-prose", ".doc-image")
) + " { display: none; }"
assert css.count(expected) == 1, (
"the text-view hide list is exactly ONE grouped declaration "
"over .doc-summary/.doc-md/.doc-raw/.doc-prose/.doc-image"
)
assert '.doc-views[data-view="pages"] .doc-pages { display: grid; }' in css assert '.doc-views[data-view="pages"] .doc-pages { display: grid; }' in css
@@ -266,7 +284,7 @@ function describe(el) {
def _render_script() -> str: def _render_script() -> str:
js = _read(DOCUMENT_JS) js = _read(DOCUMENT_JS)
parts = [ parts = [_prose_rule(js)] + [
_fn(js, n) _fn(js, n)
for n in ("renderDocument", "docViewToggle", "viewToggleButton", "docPagesStrip") for n in ("renderDocument", "docViewToggle", "viewToggleButton", "docPagesStrip")
] ]
+12 -6
View File
@@ -17,9 +17,12 @@ without a browser:
``.shared-shell``, ``.doc-md``, ``.doc-image`` (phase 122, task 04), ``.shared-shell``, ``.doc-md``, ``.doc-image`` (phase 122, task 04),
``.doc-pages`` (phase 148, task 03 — the page strip), ``.doc-pages`` (phase 148, task 03 — the page strip),
``.doc-view-toggle`` (phase 163, task 04 — the Text/Pages toggle ``.doc-view-toggle`` (phase 163, task 04 — the Text/Pages toggle
above the additive views), and ``.doc-summary:has(+ .doc-md)`` — above the additive views), ``.doc-prose`` (phase 165, task 01 — the
each capped with ``max-width: var(--chat-column)`` and NOTHING else wrapping prose block: image descriptions and txt/text docs ride the
in the file uses the token for a max-width (exactly seven rules); SAME reading column as their .doc-md sibling), and
``.doc-summary:has(+ .doc-md)`` — each capped with
``max-width: var(--chat-column)`` and NOTHING else
in the file uses the token for a max-width (exactly eight rules);
* the flipped negative pin — ZERO literal ``max-width: 46rem`` rules * the flipped negative pin — ZERO literal ``max-width: 46rem`` rules
remain (the phase-27/59/91 form-column caps are retired): the form remain (the phase-27/59/91 form-column caps are retired): the form
shells (``.tuning-shell``, ``.theme-shell``, ``.doc-edit-shell``) shells (``.tuning-shell``, ``.theme-shell``, ``.doc-edit-shell``)
@@ -124,8 +127,10 @@ def test_the_reading_columns_use_the_token() -> None:
viewer's page strip rides the SAME reading column in the shared viewer's page strip rides the SAME reading column in the shared
#doc-content slot), .doc-view-toggle (phase 163, task 04 — the #doc-content slot), .doc-view-toggle (phase 163, task 04 — the
Text/Pages toggle sits above the additive views, aligned with Text/Pages toggle sits above the additive views, aligned with
them) and .doc-summary:has(+ .doc-md) each cap with them), .doc-prose (phase 165, task 01 — the wrapping prose block
max-width: var(--chat-column) — and exactly those seven rules use is .doc-md's column with prose rhythm) and
.doc-summary:has(+ .doc-md) each cap with
max-width: var(--chat-column) — and exactly those eight rules use
the token for a max-width (no other selector).""" the token for a max-width (no other selector)."""
css = _css() css = _css()
for selector in ( for selector in (
@@ -135,12 +140,13 @@ def test_the_reading_columns_use_the_token() -> None:
".doc-image", ".doc-image",
".doc-pages", ".doc-pages",
".doc-view-toggle", ".doc-view-toggle",
".doc-prose",
".doc-summary:has(+ .doc-md)", ".doc-summary:has(+ .doc-md)",
): ):
assert "max-width: var(--chat-column)" in _rule_block(css, selector), ( assert "max-width: var(--chat-column)" in _rule_block(css, selector), (
f"{selector} must cap with max-width: var(--chat-column)" f"{selector} must cap with max-width: var(--chat-column)"
) )
assert css.count("max-width: var(--chat-column)") == 7, ( assert css.count("max-width: var(--chat-column)") == 8, (
"exactly the reading-column selectors use the token" "exactly the reading-column selectors use the token"
) )