chore(agent): phase roadmap from TODO.md — 8 phases (40–47), 24 tasks

Converts the 9 TODO items into an executable phase roadmap (Protocol B,
appended after phase 39):

- 40 tuning toggle anonymous flash (TODO L3)
- 41 sync fail-fast + modal when a model is down (TODO L4)
- 42 no reply autoscroll (TODO L5)
- 43 thinking scroll back — user scroll + gated autoscroll (TODO L7)
- 44 markdown tables (TODO L6)
- 45 agent unlimited tool calls behind BOR_AGENT_MAX_ROUNDS (TODO L8)
- 46 mobile hamburger nav (TODO L9)
- 47 quadlet + jinja import formats, A9 revision (TODO L10–L11)

Each phase carries a user story, a dedicated Playwright E2E suite plan,
and owner-locked decisions (R1 A9 format extension, R2 phase-37 budget
revision, A1–A5 scope decisions) confirmed 2026-08-27.

Also records the completed phases 30–39 todo/ -> complete/ moves that
were pending in the working tree. TODO.md is cleared (items now live in
.agent/phases/todo/).
This commit is contained in:
2026-08-27 18:25:53 -04:00
parent 492d8275e7
commit 02c76ad328
66 changed files with 1906 additions and 0 deletions
@@ -0,0 +1,103 @@
# Story: Agent makes as many tool calls as it wants
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md` L8 ·
**E2E:** `tests/e2e/test_agent_unlimited_tools.py`
## Bug report (verbatim, `TODO.md` L8)
> "Allow the LLM to make as many tool calls as it wants, remove the
> restrictions, they're causing problems getting correct answers"
## Narrative
As **the owner**, the phase-37 per-turn budgets — one `list_documents`
call and one `read_document` call — cap answers: complex questions need
several documents, and the "No reading budget left" refusal is where
correct answers die. The per-tool restrictions are being **removed**:
the model may call the tools as many times as it needs, within a single
configurable **round cap** that exists only to stop a pathological
infinite loop (and doubles as the no-tools kill switch).
- **Given** a grounded chat turn (retrieval found relevant docs)
- **When** the model calls `list_documents` / `read_document`
- **Then** every valid call is executed — re-lists included — until the
model answers or the round cap is reached, at which point a final
no-tools answer is forced.
## Acceptance criteria
1. **No per-tool budgets:** `BOR_AGENT_LIST_CALLS` /
`BOR_AGENT_READ_CALLS` are gone from `app/config.py`,
`.env.example`, and the agent loop; the `LIST_EXHAUSTED` /
`READ_EXHAUSTED` refusals no longer exist.
2. **Round cap only:** new `agent_max_rounds`
(`BOR_AGENT_MAX_ROUNDS`, default **10**) counts tool rounds; at the
cap the loop forces one final `tools=None` answer. `0` disables the
tools entirely — the request goes out with `tools=None`,
byte-identical to the pre-phase-37 path (the kill switch survives,
per the owner-locked revision).
3. **Non-budget rejections kept:** unknown tool → `"Unknown tool."`,
missing args → the `MISSING_READ_ARGS` refusal, already-in-context
document → `"Already in your context."` — none of these consume a
round's *budget* (there is none) but the round cap still bounds a
stream that keeps emitting rejected calls.
4. **Everything downstream unchanged:** the `tool` SSE event shape, the
per-turn `tool_calls=N` log field (budget-consuming → now: executed),
`done.sources` extension, and the UI tool lines are untouched.
5. **PLAN recorded:** the phase-37 locked decision
("budgets-as-kill-switch") is revised in `.agent/PLAN.md` with an
owner-permission note (2026-08-27, `TODO.md` L8), following the
established revision-note pattern (phases 16/19/24/37).
## Owner-confirmed (2026-08-27, roadmap R2)
1. **Both budget env vars removed.** New single guard
`BOR_AGENT_MAX_ROUNDS` (default **10** tool rounds, then forced final
answer); **`0` = no-tools kill switch** (pre-phase-37 behavior).
2. **Re-listing is allowed** (a second `list_documents` is a normal
executed call — it even counts in `tool_calls=N`).
## UI Visualization & Structure
- **Config** (`app/config.py`): delete `agent_list_calls` /
`agent_read_calls`; add `agent_max_rounds: int = 10` with a docstring
(0 disables the tools entirely — the loop makes exactly one request
with `tools=None`). `.env.example`: the two `BOR_AGENT_*_CALLS` lines
are replaced by `BOR_AGENT_MAX_ROUNDS=10` with an updated comment.
- **Agent loop** (`app/rag/agent.py`): `run_agent` —
`max_rounds = settings.agent_max_rounds`; `tools = AGENT_TOOLS if
max_rounds > 0 else None`; the loop drops `list_left` / `read_left`
and the budget-driven `tools = None` transition; after each executed
call `rounds += 1` and at `rounds >= max_rounds` the existing forced
final answer path runs (now the *only* exit besides "no calls").
`_execute_tool` loses its budget parameters + the two exhaustion
constants; `AGENT_TOOLS`' `read_document` description drops "exactly
one more"; module/docstrings and the probe reference updated.
`AgentHolder` unchanged (`tool_calls` still counts executed calls).
- **Tests:** `tests/unit/test_agent.py` rewritten around the round cap
(always-listing mock LLM: N rounds then forced answer; `max_rounds=0`
→ single `tools=None` request; rejected-call spam bounded by the cap);
`tests/unit/test_config.py` (default 10, env override, 0);
`tests/integration/test_chat_api.py` budget fixtures →
`agent_max_rounds`; `tests/e2e/mock_llm.py` `_tool_flow` extended: the
existing deterministic 3-step flow stays, plus a multi-read variant
triggered by a marker (e.g. the user message containing
`"read two documents"`) — stateless classification by counting
`Document …:` tool messages (list → read #1 → read #2 → answer).
- **Non-goals:** no per-call cost cap, no streaming change, no new
endpoint.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_agent_unlimited_tools.py`** (mock
LLM; DB up):
1. `test_multi_read_turn` — a grounded question carrying both
`TOOLS_TRIGGER` and the multi-read marker: the turn streams tool
lines for `list_documents` **and two** `read_document` calls, then a
final answer; the bubble is not deflected.
2. `test_done_sources_include_reads` — the final `done` (observed via
the source chips) lists the retrieved doc(s) plus **both** read
documents, deduped.
3. `test_relist_allowed` (unit-level via integration, plus E2E
observable state) — a re-listed catalog does not produce a refusal
line; the UI shows a tool line per executed call.
4. `test_single_tool_flow_regression` (phase 37) — the original
3-step flow still answers with exactly one read (runs against the
unchanged `tests/e2e/test_agent_document_tools.py` in the
regression pass, not duplicated here).
+100
View File
@@ -0,0 +1,100 @@
# Story: Markdown tables in chat (and everywhere the renderer runs)
**Phase:** `44_markdown_tables` · **Source:** `TODO.md` L6 ·
**E2E:** `tests/e2e/test_markdown_tables.py`
## Bug report (verbatim, `TODO.md` L6)
> "Certain markdown formatting isn't working - tables for example don't
> get rendered as tables in the chat response."
## Narrative
As **a user**, when Brain answers with a markdown table (services and
ports, versions, schedules — the concrete specifics the persona is built
around), I expect a real table: aligned columns, borders, readable.
Today the shared renderer (`frontend/assets/markdown.js`, ~60 lines,
no-CDN by A11) has no table support — a pipe table renders as one
paragraph of raw `|` text.
- **Given** Brain's answer (or a document / thinking block) contains a
GFM pipe table
- **When** the renderer runs
- **Then** it renders a semantic `<table>` with a `<thead>` header row
and `<tbody>` body rows, XSS-safe (escape-first, as the rest of the
renderer).
## Acceptance criteria
1. **Pipe tables render:** header row + `|---|` separator row + body
rows → `<table class="md-table">` with `<th scope="col">` header
cells; leading/trailing pipes and in-cell whitespace are handled;
cells keep their inline markdown (bold/em/code).
2. **XSS-safe:** cell content is escaped before any transform — a cell
containing `<img onerror=…>` renders inert (the escape-first
guarantee, same as all other content).
3. **Code fences win:** a `|`-heavy block inside a ``` fence is never
parsed as a table (fence protection runs first, as today).
4. **Non-tables stay put:** a single `|` in prose, a lone separator
without a header, or a 1-line "table" is left as text.
5. **Wide tables:** the table sits in an `overflow-x: auto` wrapper so a
wide table scrolls horizontally instead of breaking the 46rem chat
column (PLAN §7.1).
6. **Shared everywhere:** the same renderer serves the chat answer, the
document viewer, and the thinking block — all three render tables.
7. **Style:** `.md-table` uses the existing dark-tech palette
(PLAN §7.2 tokens, contrast ≥4.5:1); alignment colons in the
separator are parsed but all cells render left-aligned (owner
decision).
## Owner-confirmed (2026-08-27, roadmap A3)
1. **Scope = GFM pipe tables** (header + separator + body rows). Links,
blockquotes, and hr are **not** in scope for this story.
2. **Alignment colons parsed, rendered left.**
3. **Wide tables get a horizontal scroll wrapper** inside the bubble.
## UI Visualization & Structure
- **Renderer** (`frontend/assets/markdown.js`): a table-protection pass
between the existing fence pass and the escape pass — consecutive
lines forming a table (every line contains `|`; line 2 matches the
separator `^\s*\|?(\s*:?-{3,}:?\s*\|)+\s*:?-{3,}:?\s*\|?\s*$`-style
rule with ≥1 cell) are pulled out, each cell is escaped + inline-
transformed, and the block is reinserted as protected HTML (same
`\u0000CODE…\u0000` placeholder mechanism as code fences, or a
sibling placeholder — the existing restore step is the only place
placeholders are re-expanded). Output shape:
`<div class="md-table-wrap"><table class="md-table"><thead><tr>
<th scope="col">…` / `</table></div>`.
- **CSS** (`frontend/assets/styles.css`): `.md-table-wrap
{ overflow-x: auto; }` (the wrapper is the scroller — the table keeps
natural width); `.md-table` border-collapse, `th`/`td` borders from
`--line`, padding ≈0.4rem 0.6rem, `thead` tinted from the surface
palette; fits the 46rem column without a new container.
- **Mock** (`tests/e2e/mock_llm.py`): new `TABLE_TRIGGER` (a substring
like `"show me a table"`) → `compose_answer` returns a fixed
deterministic GFM table answer (checked before the default tail-echo
branch), including one deliberately wide table for the overflow
assertion.
- **Non-goals:** no new library (A11), no CDN, no change to the escape-
first architecture, no table editing in the steering/tuning UI.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_markdown_tables.py`** (mock LLM; DB
up):
1. `test_chat_table_renders` — ask a question containing `TABLE_TRIGGER`;
the brain bubble contains `<table class="md-table">` with a
`<thead>` (header cells as `<th scope="col">`) and the expected
body-cell texts; no raw `|---|` separator text in the bubble.
2. `test_wide_table_scrolls` — the mock's wide table: the bubble's
`.md-table-wrap` has `scrollWidth > clientWidth` and horizontal
wheel/scroll moves it; the 46rem column itself does not overflow the
page.
3. `test_table_xss_safe` — a table whose cell contains an HTML tag
(mock variant or document content) renders the tag as text
(no injected element).
4. `test_viewer_table_renders` (shared renderer) — an indexed fixture
document containing a pipe table opens in the document modal and
renders the same `<table class="md-table">`.
5. `test_fence_not_a_table` (regression) — a fenced code block full of
`|` pipes renders as `<pre><code>`, no `<table>`.
6. `test_plain_pipe_stays_text` (regression) — a prose answer with a
single `|` renders as text, no `<table>`.
+110
View File
@@ -0,0 +1,110 @@
# Story: Mobile hamburger nav
**Phase:** `46_mobile_hamburger_nav` · **Source:** `TODO.md` L9 ·
**E2E:** `tests/e2e/test_mobile_hamburger_nav.py`
## Bug report (verbatim, `TODO.md` L9)
> "The navbar on mobile is way too squished. Make it a hamburger
> dropdown menu with a nice animation"
## Narrative
As **a mobile user**, the 58px header currently crams up to four text
nav pills (Chat / Sources / Git sources / Tuning) next to the brand and
four icon action pills — the phase-34/35 squeeze at 360–375px leaves
0.72rem-font pills that are hard to hit and hard to read. The nav links
move into a **hamburger dropdown menu** on small screens: one
`#nav-toggle` button in the bar, and the links open as an animated
panel below the header with comfortable touch targets.
- **Given** a viewport ≤640px
- **When** I tap the hamburger
- **Then** the nav menu drops down with a short slide+fade animation and
full-size links; tapping a link navigates and closes the menu.
- **Given** a viewport >640px
- **When** the page loads
- **Then** nothing changes — the inline nav pills render exactly as
today.
## Acceptance criteria
1. **Mobile (≤640px):** the inline nav pills are hidden from the bar; a
hamburger button (`#nav-toggle`, `aria-label="Menu"`,
`aria-controls="app-nav"`, `aria-expanded`) appears — 44px touch
target, icon-only.
2. **Menu:** `.app-nav` (now `id="app-nav"`) becomes a dropdown panel
below the header — vertical full-width rows, ≥44px targets, readable
font size; the **auth visibility contract is preserved inside the
menu** (anonymous: only "Chat"; admin: Chat / Sources / Git sources /
Tuning — the same ship-hidden `hidden` attributes the whoami gate
already drives).
3. **Animation:** opening/closing animates (slide-down + fade, ≈180ms);
`prefers-reduced-motion: reduce` stills it (no transition).
4. **Behavior:** toggle flips `aria-expanded`; `Esc` closes while open;
a link click navigates **and** closes the menu; resizing back to
>640px closes it (the inline nav reappears, no stale state).
5. **Bar layout:** the action pills (Tuning toggle, Sync, New chat,
Sign in/out) stay in the bar icon-only — the phase-35 tightest
squeeze rules on `.nav-link` / `.app-nav` gaps are replaced by the
roomier menu; the bar fits 360px with the brand intact or clipped as
today.
6. **All six pages** get the identical toggle + menu (the phase-34
"same bar on every page" contract).
## Owner-confirmed (2026-08-27, roadmap A5)
1. **The hamburger contains the nav links only** (Chat / Sources / Git
sources / Tuning). The action pills stay in the bar.
2. **Animation:** slide-down + fade, 180ms; `prefers-reduced-motion`
stills it.
3. **Breakpoint:** the existing ≤640px mobile block (no new breakpoint).
## UI Visualization & Structure
- **Markup (all six pages — `index.html`, `sources.html`,
`document.html`, `git-sources.html`, `login.html`, `tuning.html`):**
a `#nav-toggle` button inserted before `<nav class="app-nav"
aria-label="Primary">` (which gains `id="app-nav"`); the nav keeps
its existing links and `hidden` attributes byte-identically.
- **CSS** (`frontend/assets/styles.css`, the `@media (max-width: 640px)`
block): `.nav-toggle { display: none }` outside, `display:
inline-flex` + 44px target inside; `.app-nav` becomes the dropdown:
absolute below the header, `flex-direction: column`, surface background
+ bottom border/shadow, full-width row links; closed state
(`visibility: hidden; opacity: 0; transform: translateY(-8px);
pointer-events: none`) → `.is-open` (`visible; opacity: 1; transform:
none`), `transition: opacity/transform 180ms ease`; a
`prefers-reduced-motion` override kills the transition. The old
nav-pill squeeze rules (`.nav-link` 0.72rem, `.app-nav` gap 0.05rem)
are superseded for the menu rows.
- **JS** (`frontend/assets/header.js`, module-import binding like
sign-out): null-safe `#nav-toggle` / `#app-nav` — click toggles
`.is-open` + `aria-expanded`; delegated click on nav links closes it;
`document` keydown `Esc` closes while open; a
`matchMedia("(max-width: 640px)")` change listener closes on
desktop. No other header.js behavior touched.
- **Non-goals:** no change to the 900px tablet rules, the action pills,
the viewer's title bar height, or the no-CDN/A11 constraints.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_mobile_hamburger_nav.py`** (mock
LLM; DB up):
1. `test_mobile_hamburger_visible_and_bar_roomy` — 375×812: `#nav-toggle`
visible with `aria-expanded="false"`; the inline nav links are not
visible in the bar (menu closed); the header has no horizontal
overflow.
2. `test_anonymous_menu_contents` — anonymous: open the menu → exactly
"Chat" is visible; open/close flips `aria-expanded`.
3. `test_admin_menu_contents` — login: open the menu → Chat / Sources /
Git sources / Tuning all visible (auth contract inside the menu).
4. `test_link_click_navigates_and_closes` — open, click "Sources"
(admin): navigates to `/sources.html`, and the menu on the arrival
page is closed.
5. `test_esc_and_backdrop_close` — open, `Esc` closes (aria-expanded
false); open again, click outside the panel closes.
6. `test_animation_and_reduced_motion` — with motion allowed, the menu
has a transition (computed `transition-duration` ≈180ms on the
opacity/transform pair); with `reducedMotion: "reduce"` emulated,
the transition is none/0s and the menu still opens/closes.
7. `test_desktop_unchanged` (regression) — 1280×800: no hamburger,
inline nav pills exactly as before (the phase-34/35 bar contract,
`test_nav_consistency` / `test_header_consistency` pass in the
regression pass).
@@ -0,0 +1,91 @@
# Story: No reply autoscroll
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md` L5 ·
**E2E:** `tests/e2e/test_no_reply_autoscroll.py`
## Bug report (verbatim, `TODO.md` L5)
> "Get rid of the chat reply autoscroll, it's breaking things like
> making it impossible for the user to scroll while a reply generates."
## Narrative
As **a user reading a long answer**, I want full control of the
viewport while Brain replies. Today the page auto-scrolls to follow the
stream (phase 18 "follow-the-bottom"): while I'm in the 200px
near-bottom band the page is yanked down on every thinking / tool /
delta frame, which fights my own scrolling mid-answer. The reply
autoscroll is being **removed** — the page only scrolls when I
explicitly cause it.
- **Given** a reply is streaming (thinking, tool calls, or answer text)
- **When** I scroll up to read earlier context
- **Then** the viewport stays exactly where I put it for the rest of the
turn — no frame yanks it back.
## Acceptance criteria
1. **No streaming autoscroll:** during a long thinking stream, a tool
call, and a long answer, the page never auto-scrolls — sampled
`window.scrollY` is stable (within 1px) across frames while the
viewport is away from the bottom.
2. **Submit reveals my message:** sending a question while scrolled up
still scrolls the viewport down so my own message is visible
(user-initiated — kept by owner decision).
3. **Restore landing kept:** reloading a persisted conversation
(phase 14) still lands one-shot on the latest message.
4. **The phase-18 gate is gone:** `NEAR_BOTTOM_PX` /
`isNearBottom()` and the per-frame `scrollReveal` calls in the
thinking / tool / delta handlers are removed from `app.js`; the unit
pin (`tests/unit/test_frontend_scroll.py`) is rewritten for the new
contract (scrolls happen only on submit + restore landing).
5. **Everything else unchanged:** the thinking window's *internal*
bottom-pin (phase 17 — `textEl.scrollTop`, not the page) is untouched
in this phase (phase 43 reworks it separately); message rendering,
persistence, UI states, and the 120 s guard are unchanged.
## Owner-confirmed (2026-08-27, roadmap A1)
1. **"Reply autoscroll" = the phase-18 follow-the-bottom auto-follow on
thinking / tool / delta frames.** Removed.
2. **Kept:** scroll-on-submit (reveal the user's own message) and the
one-shot restore landing on page load.
## UI Visualization & Structure
- **The functional change is in `frontend/assets/app.js` only:**
- delete `export const NEAR_BOTTOM_PX = 200`, `isNearBottom()`, and
the `force`-optional gating in `scrollReveal` — the helper becomes an
unconditional `scrollIntoView` (still smooth, still still under
`prefers-reduced-motion` via the existing `SCROLL` constant);
- `addMessage(...)` gains an explicit "scroll" intent: the **user
submit** path scrolls (shows my message), brain bubble creation and
the typing indicator do **not**;
- the thinking / tool / delta handlers drop their `scrollReveal(wrap)`
calls (the thinking handler keeps its `textEl.scrollTop` window-pin —
phase 17, reworked in phase 43);
- the phase-14 restore landing keeps its one-shot forced scroll;
- module docstrings updated (the phase-18 contract block is replaced
by the new "no reply autoscroll (owner direction 2026-08-27)"
contract).
- **Non-goals:** no new UI element, no "↓ new content" pill (the owner
wants silence, not a substitute affordance), no change to the
composer / safe-area layout.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_no_reply_autoscroll.py`** (mock LLM;
DB up; the phase-18 suite `tests/e2e/test_follow_bottom_scroll.py` is
**deleted** in this phase — its behavior is intentionally removed):
1. `test_no_autoscroll_during_long_answer` — `LONG_ANSWER_TRIGGER`
question; once the answer starts, scroll the viewport up ~2× the
answer height; sample `window.scrollY` across ≥10 streaming frames:
stable within 1px; after `done` the viewport is still where it was.
2. `test_no_autoscroll_during_thinking` — `THINKING_TRIGGER` question;
scroll up during the ~4.5 s thinking stream; the viewport stays
pinned (no per-chunk page follow).
3. `test_submit_reveals_user_message` — scroll to the very top of a
populated conversation, send a question; the viewport ends with the
user's message visible (bottom in view).
4. `test_restore_landing_one_shot` (phase 14 regression) — settle a
conversation, reload; the page lands on the latest message one-shot
and stays there while no stream is active.
5. `test_answer_content_intact` (regression) — the long answer streams
to completion with sources and (for thinking) the collapsed block,
persisted and restorable.
@@ -0,0 +1,98 @@
# Story: Import quadlet + jinja files
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md` L10–L11 ·
**E2E:** `tests/e2e/test_quadlet_jinja_import.py`
## Bug reports (verbatim, `TODO.md` L10–L11)
> "Add \".container\", \".network\", \".volume\" and other quadlet files
> to the list of allowed/parsed files"
> "Add \".j2\" jinja files to the list of allowed/parsed files"
## Narrative
As **the owner**, my homelab notes increasingly live in Podman quadlet
unit files (`.container`, `.network`, `.volume`, …) and Jinja templates
(`.j2`). Neither is in the A9 import format list, so the KB is blind to
exactly the config files I ask questions about. Both families join the
allowed + default import formats and are parsed (chunked) by the
importer.
- **Given** a source directory containing quadlet and/or `.j2` files
- **When** the importer (CLI or sync) runs
- **Then** those files are indexed (chunked, embedded, upserted) like any
other A9-format file, and their content is retrievable.
## Acceptance criteria
1. **Allowed set:** `_ALLOWED_IMPORT_EXTENSIONS` in `app/config.py`
gains `container, network, volume, image, pod, kube, swap, os,
endpoint` (the full Podman quadlet family) and `j2`; a
`BOR_IMPORT_EXTENSIONS` env value may name any of them (the
never-widen validator keeps rejecting truly unknown extensions).
2. **Default set:** the default `import_extensions` CSV includes all ten
new formats after the existing seven — a default import now picks
them up with no env configuration.
3. **Parsing:** `chunker.py` dispatches every new suffix to
plain-text paragraph packing (`chunk_text`) — quadlet files are TOML
unit files and `.j2` files are templates; no format-specific
splitter (owner decision). Every chunk still honors
`HARD_MAX_CHARS` (1200).
4. **Title:** no H1 → `extract_title` falls back to the file stem, as
with other non-markdown formats (no change needed, verified).
5. **Behavior parity:** hidden (dot) directories are still skipped, the
exclusion list is unchanged, sha256 delta detection / prune work
unchanged for the new formats.
6. **Docs:** `.env.example`'s `BOR_IMPORT_EXTENSIONS` comment, the
README's format list, and an **A9 revision note** in
`.agent/PLAN.md` (owner permission 2026-08-27, `TODO.md` L10–L11)
record the extended set.
## Owner-confirmed (2026-08-27, roadmap R1)
1. **Full quadlet family:** `container, network, volume, image, pod,
kube, swap, os, endpoint` — plus `j2`.
2. **Chunked as plain text** — no TOML/Jinja-aware splitting.
3. **A9 is revised** with a PLAN.md revision note (the established
owner-permission pattern).
## UI Visualization & Structure
- **Config** (`app/config.py`): extend the `_ALLOWED_IMPORT_EXTENSIONS`
frozenset + the `import_extensions` default string (comment cites the
A9 revision 2026-08-27). No validator change — it already
normalizes/dedups and rejects unknowns.
- **Chunker** (`app/rag/chunker.py`): ten new `_FORMAT_CHUNKERS` entries
→ `chunk_text`; module docstring's format list updated.
- **Fixtures:** `tests/fixtures/docs/homelab/quadlet/compose.container`
(realistic quadlet TOML: `[Unit]` / `[Service]` / `[Container]`
sections, one unique sentinel token, >1200 chars to exercise
sub-splitting), `…/quadlet/lan.network`, `…/quadlet/cache.volume`,
and `tests/fixtures/docs/homelab/templates/deploy.j2` (Jinja snippet
with `{{ … }}` / `{% … %}` tags + its own sentinel).
- **Tests:** unit (`test_config.py` allowed-set/default/validator;
`test_chunker.py` dispatch for every new suffix + fixture chunking;
`test_importer.py` directory walk picks the new files up);
integration (import_sources over a temp dir with quadlet+j2 files →
documents + chunks rows).
- **Non-goals:** no new DB column, no format badge change (the viewer
shows the extension it already shows), no summary-model changes.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_quadlet_jinja_import.py`** (mock
LLM; DB up):
1. `test_quadlet_and_jinja_indexed` — truncate + import the fixture
tree (the `test_import_documents.py` pattern): `GET /api/docs` lists
the `.container` / `.network` / `.volume` / `.j2` files with
non-zero chunk counts.
2. `test_sources_table_shows_them` (admin) — the Sources table renders
rows for the new files; their path links open the document modal.
3. `test_container_content_viewable` — the viewer modal shows the
`.container` file's TOML content (sentinel token present) with its
stem as the title.
4. `test_jinja_retrievable_not_deflected` — ask a question containing
the `.j2` file's sentinel word: the FTS hit keeps the honesty gate
honest-positive (A8) — the answer bubble is **not**
`.is-deflected` and the source chip names the `.j2` document.
5. `test_default_walk_includes_new_formats` (regression, unit-backed) —
a default-extensions walk over a temp tree with all ten new
extensions indexes every file; hidden directories + the exclusion
list still filter (covered by `test_importer.py` in the regression
pass).
@@ -0,0 +1,97 @@
# Story: Sync fails fast + modal when a model is down
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md` L4 ·
**E2E:** `tests/e2e/test_sync_model_down.py`
## Bug report (verbatim, `TODO.md` L4)
> "If the embedding or lite model is not accessible the sync button
> should fail fast and there should be a modal error popup explaining
> that the model isn't available."
## Narrative
As **the admin**, when I press "Sync sources" with the aipi models
(`embed` or `lite`) unreachable, I don't want to wait through git clones
and a partial import to discover the KB can't be updated — and a tooltip
on a button is not a readable error. The sync should **fail fast**
(before any expensive work) with a clear "the model isn't available"
message, shown in a **modal dialog** I can read and dismiss.
- **Given** the `embed` or `lite` model endpoint is unreachable
- **When** I press "Sync sources"
- **Then** the run fails within a couple of seconds (before any git
clone), the button settles retry-ready, and a modal dialog explains
which model isn't available.
## Acceptance criteria
1. **Server fail-fast:** `POST /api/sync` with a dead LLM endpoint
reaches `state: "failed"` with a message naming the unavailable model
(embedding first, then summary/lite) **without** cloning any source —
the probe (one small embedding + one tiny completion against
`BOR_LLM_SUMMARY_MODEL`) runs before source resolution and before any
`clone_or_pull`.
2. **Modal:** on a failed sync the page shows a modal error dialog
(`role="alertdialog"`, `aria-modal="true"`) with a title, the
sanitized error text (rendered via `textContent` — XSS-safe), and a
close control; it closes on the close button, `Esc`, or backdrop
click; focus moves into the dialog on open and returns to `#sync-btn`
on close.
3. **Every page:** the modal is built by the shared header module
(`frontend/assets/header.js`), which owns the sync state machine — so
it appears wherever `#sync-btn` exists (all six pages from phase 34).
4. **Existing surfaces kept:** the button's failed-state `title` /
`aria-label` / `.is-error` affordance and the Sources page's
`#sync-error-banner` (via `bor:sync-status`) are unchanged — the modal
is the primary, readable surface.
5. **Success path unchanged:** a healthy model still runs clone → import
→ overview exactly as phase 32/35/38 define it (regression).
## Owner-confirmed (2026-08-27, roadmap A4)
1. **The probe runs before git clones** — the fastest possible failure;
it costs one small embedding request and one ~1-token completion.
2. **The modal is the primary failure surface on every page;** the
button-title affordance and the Sources banner stay as secondary
surfaces.
## UI Visualization & Structure
- **Server** (`app/rag/llm.py`, `app/api/sync.py`): a new
`ModelUnavailableError` (subclass of `LLMError`) + an `async
check_models(llm)` probe: `embed_one("sync model check")` then a tiny
`chat([...])` against the summary model; each failure mode maps to a
message naming the model and that it isn't available (the sync
sanitizer's credential masking still applies downstream). `_run_sync`
calls it first, after `LLMClient()` construction — before
`effective_sources`, before any clone.
- **UI** (`frontend/assets/header.js`, `frontend/assets/styles.css`):
`applySyncFailure(status)` additionally opens `showSyncModal(status)`:
a lazily-created backdrop + `role="alertdialog"` panel appended to
`<body>` (so no page markup changes), error text via `textContent`,
close button + `Esc` + backdrop-click dismissal, focus management as
in AC 2. Styled with the existing dark-theme error palette
(PLAN §7.2: `#fca5a5` on `#2d1318` class, error border), `:focus-visible`
per the global rule, no motion under `prefers-reduced-motion`.
- **Non-goals:** no new endpoint, no retry-from-modal button (the button
itself is retry-ready), no change to the 2 s poll lifecycle.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_sync_model_down.py`** (mock LLM; DB
up). The suite boots its **own module-scoped app** on a distinct port
(conftest pattern used by `test_sync_button.py`) with
`BOR_LLM_BASE_URL=http://127.0.0.1:9/v1` (dead port — connection
refused) and a local `file://` fixture repo as the configured source, so
a (regressed, non-fail-fast) run would spend time cloning before failing:
1. `test_model_down_fails_fast_with_modal` — admin login, click
`#sync-btn`; within a short wall-clock budget (≤ ~10 s, vs the 60 s
generous budget of the healthy-run suite) the button settles
retry-ready **and** the modal is visible with an error naming the
model; assert the dialog role/aria contract.
2. `test_modal_dismissal` — close via button, `Esc`, and backdrop click
(one fresh failure per path); focus returns to `#sync-btn` each time.
3. `test_sync_error_surfaces_unaffected` (phase 32 regression) — after
the failure the button keeps its `title` / `.is-error` affordance; on
`/sources.html` the `#sync-error-banner` still renders off
`bor:sync-status`.
4. `test_healthy_sync_still_succeeds` (phase 32/35 regression) — the
session mock-backed app (or a second healthy module app) still runs
the full clone → import → overview pipeline to "Synced HH:MM".
@@ -0,0 +1,98 @@
# Story: Thinking scroll back (user scroll + generate-time autoscroll)
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md` L7 ·
**E2E:** `tests/e2e/test_thinking_scroll.py`
## Bug report (verbatim, `TODO.md` L7)
> "Add scrolling back to the thinking block, but have it autoscroll
> while thinking content is generating."
## Narrative
As **a user watching Brain reason**, the Thinking block should work like
a well-behaved live console: it **follows the tail while the reasoning
is generating** — but the moment I scroll up to re-read an earlier line,
it must **stop yanking me down**, and it must let me scroll the window
freely (phase 21's no-scroll clip is being reversed by owner direction).
- **Given** the Thinking block is streaming reasoning content
- **When** I'm at the bottom of the 320px window
- **Then** each new chunk keeps the window pinned to the live tail.
- **When** I scroll up to read earlier reasoning
- **Then** the window stays where I put it (no more re-pinning);
- **When** I return to the bottom
- **Then** tail-following resumes on the next chunk.
## Acceptance criteria
1. **User scroll restored:** computed `overflow-y` of
`.thinking-text` is `auto`; wheel / mouse-drag / keyboard move the
window (phase 21's `overflow-y: hidden` is gone).
2. **Follow while generating:** with the user pinned at the window's
bottom (within a small near-bottom band — the phase-18 pattern,
now applied to the *window* instead of the page, exported constant
`THINKING_NEAR_BOTTOM_PX = 32`), each streamed chunk re-pins the
window to the tail (within 1px).
3. **Paused on scroll-up:** scrolled up, the window stops being re-pinned
— `scrollTop` stays stable across subsequent chunks (within 1px).
4. **Resumes on return:** scrolling back to the bottom (within the band)
resumes tail-following on the next chunk.
5. **Kept from phases 17/21:** the fixed 320px `max-height` window, the
auto-collapse on the first answer token, the reduced-motion stillness,
and the answer-bubble scroll behavior (phase 11) are all unchanged.
## Owner-confirmed (2026-08-27, roadmap A2)
1. **The 320px window stays** — only the overflow mode and the pinning
logic change.
2. **Follow-the-bottom for the window:** autoscroll only while the user
is pinned near the window's bottom (≈32px band); scroll-up pauses,
return-to-bottom resumes. (The phase-18 page-level band is removed in
phase 42; this is its window-level successor.)
## UI Visualization & Structure
- **CSS** (`frontend/assets/styles.css`): `details.thinking
.thinking-text` — `overflow-y: hidden` → `overflow-y: auto`; the
owner-choice comment is replaced with the 2026-08-27 direction
(user-scrollable window; JS follows the tail only while pinned).
`max-height: 320px` and all other declarations untouched.
- **JS** (`frontend/assets/app.js`, the `thinking` SSE handler):
- new exported `const THINKING_NEAR_BOTTOM_PX = 32` +
`isThinkingNearBottom(textEl)` (`scrollHeight - scrollTop -
clientHeight <= band`);
- the phase-17 unconditional pin
(`textEl.scrollTop = textEl.scrollHeight`) becomes gated:
`if (block.open && isThinkingNearBottom(textEl)) { textEl.scrollTop
= textEl.scrollHeight; }` — a scrolled-up user is never re-pinned,
and returning to the bottom re-arms the pin automatically (the check
runs on every chunk).
- **Non-goals:** no "↓ more" affordance, no auto-height growth, no change
to the summary/chevron, the tool-call lines, or the answer bubble.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_thinking_scroll.py`** (mock LLM; DB
up; phase 21 lengthened `mock_llm.compose_thinking` to ~2 700 chars ≈
4.5 s of paced frames so the scratchpad overflows the 320px window by
~2×, and the phase-20 hesitation trigger gives a deterministic 4 s
frozen-tail state with the block open). The phase-21 suite
`tests/e2e/test_thinking_no_scroll.py` is **deleted** in this phase:
1. `test_thinking_window_user_scrollable` — frozen live tail (4 s
hesitation): focus `.thinking-text`, wheel up, `Home`, mouse-drag up
— `scrollTop` moves; the window shows earlier content.
2. `test_thinking_window_follows_while_pinned` — during the live stream:
with the user at the bottom, after the 2nd-to-last and the last chunk
the window is pinned to the tail (within 1px) and the last chunk's
text renders inside the visible rectangle.
3. `test_thinking_window_stops_on_scroll_up` — mid-stream: scroll up
~half the window; over the next ≥5 chunks `scrollTop` stays stable
(within 1px) — no re-pin.
4. `test_thinking_window_resumes_on_return` — from the paused state,
scroll the window back to its bottom; on the next chunk the window is
re-pinned to the tail (within 1px).
5. `test_thinking_window_css_contract` — computed `overflow-y: auto`,
`max-height: 320px`, and the clip is real (`scrollHeight >
clientHeight` for the long scratchpad).
6. `test_answer_bubble_still_scrollable` (phase 11 regression) — a long
answer: the page scrolls, the bubble's overflow is untouched.
7. `test_restored_collapsed_thinking_unaffected` (phase 17 regression) —
a settled thinking turn reloads as a collapsed block with its full
text.
@@ -0,0 +1,87 @@
# Story: Tuning toggle anonymous flash
**Phase:** `40_tuning_toggle_flash` · **Source:** `TODO.md` L3 ·
**E2E:** `tests/e2e/test_tuning_toggle_flash.py`
## Bug report (verbatim, `TODO.md` L3)
> "Loading the page briefly shows the 'Tuning' button in the header even
> when the user isn't authenticated. Only show that if the user is
> authenticated."
## Narrative
As **an anonymous visitor**, the header must never show admin-only
controls — not even for a frame. Today the tuning-notes toggle
(`#steering-toggle`, the header button labeled **"Tuning"**) ships
*visible* in all six pages' markup and is only removed from the DOM after
`/api/whoami` resolves — so every anonymous page load flashes the button
for the length of the whoami round-trip. The admin-only *nav links*
(`#nav-sources`, `#nav-git-sources`, `#nav-tuning`) already ship `hidden`
(phase-19 "absent, not hidden" contract) and are not the issue.
- **Given** an anonymous visitor loads any page
- **When** the page renders (before `/api/whoami` resolves)
- **Then** no "Tuning" control is ever visible — not for a single frame.
- **Given** a signed-in admin loads any page
- **When** whoami resolves
- **Then** the toggle is revealed (and the note list refreshes, as today).
## Acceptance criteria
1. Anonymous load of **every** page: `#steering-toggle` is never
attached-visible — a MutationObserver installed via `addInitScript`
records zero visible frames of the toggle from first paint to settled
state; after load the toggle is absent from the DOM (the existing
remove-from-DOM behavior).
2. Admin load: the toggle is visible after whoami, `aria-expanded`
works, the count badge refreshes — identical to today's admin
behavior (phase 15/34 contract).
3. No-JS visitors: the toggle is hidden (the control is JS-gated by
design — whoami decides).
4. No regression to the shared-header contract (phase 19/34): nav links
ship hidden, sign-in/sign-out pair, sync button, new-chat binding
unchanged.
## Owner-confirmed (2026-08-27, roadmap A — confirmed with the
conversion interview)
1. **The flashing control is the steering toggle**, not the admin
"Tuning" nav link (which already ships `hidden`) — confirmed by code
inspection: `#steering-toggle` ships visible in all six pages
(`index.html`, `sources.html`, `document.html`, `git-sources.html`,
`login.html`, `tuning.html`) and is removed post-whoami.
2. **Fix = ship `hidden`, reveal for admin, keep anonymous removal** —
the same ship-hidden / reveal-for-admin contract the admin-only nav
links already use; anonymous still gets "absent, not hidden".
## UI Visualization & Structure
- **The whole functional change is one attribute + one JS line:**
- `#steering-toggle` gains `hidden` in all six pages' header markup
(the button element only — the `#steering-panel` region already ships
`hidden`).
- `frontend/assets/header.js` `initSharedHeader()`: in the admin branch,
unhide the toggle (`steeringToggle.hidden = false`) before
`refreshSteering()`; the anonymous branch (`steeringToggle?.remove()`)
is unchanged.
- **Non-goals:** no change to the nav links, the panel, the steering
API, or any other shared-header control.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_tuning_toggle_flash.py`** (mock LLM;
DB up):
1. `test_anonymous_never_sees_toggle` — `addInitScript` a MutationObserver
that records every frame in which `#steering-toggle` exists in the
DOM and is not `[hidden]`; load `/` anonymously; after load assert the
observer recorded **zero** such frames, and the toggle is absent from
the DOM (removed, per the phase-16 contract).
2. `test_anonymous_other_pages_never_flash` — same observer assertion on
`/sources.html`, `/tuning.html`, `/login.html` (the page set the
contract must hold on).
3. `test_admin_toggle_revealed_and_working` — login via
`e2e.auth_helpers.login`; reload `/`; the toggle is visible
(`hidden` removed), clicking opens `#steering-panel`
(`aria-expanded="true"`), and the count badge matches the panel.
4. `test_nav_contract_regression` (phase 19/34) — anonymous: nav links
`#nav-sources` / `#nav-git-sources` / `#nav-tuning` stay hidden and
absent from the visible header; admin: they are revealed — the
ship-hidden contract this phase relies on is intact.