chore(agent): phase roadmap from TODO.md — 8 phases (40–47), 24 tasks

Converts the 9 TODO items into an executable phase roadmap (Protocol B,
appended after phase 39):

- 40 tuning toggle anonymous flash (TODO L3)
- 41 sync fail-fast + modal when a model is down (TODO L4)
- 42 no reply autoscroll (TODO L5)
- 43 thinking scroll back — user scroll + gated autoscroll (TODO L7)
- 44 markdown tables (TODO L6)
- 45 agent unlimited tool calls behind BOR_AGENT_MAX_ROUNDS (TODO L8)
- 46 mobile hamburger nav (TODO L9)
- 47 quadlet + jinja import formats, A9 revision (TODO L10–L11)

Each phase carries a user story, a dedicated Playwright E2E suite plan,
and owner-locked decisions (R1 A9 format extension, R2 phase-37 budget
revision, A1–A5 scope decisions) confirmed 2026-08-27.

Also records the completed phases 30–39 todo/ -> complete/ moves that
were pending in the working tree. TODO.md is cleared (items now live in
.agent/phases/todo/).
This commit is contained in:
2026-08-27 18:25:53 -04:00
parent 492d8275e7
commit 02c76ad328
66 changed files with 1906 additions and 0 deletions
@@ -0,0 +1,37 @@
# Phase 40 — Tuning toggle anonymous flash
**Source:** `TODO.md` L3 — "Loading the page briefly shows the 'Tuning' button in the header even when the user isn't authenticated. Only show that if the user is authenticated."
**Story:** `.agent/user_stories/tuning-toggle-flash.md`
**Context:** Phase 15/34 shared header (`frontend/assets/header.js` owns `#steering-toggle` / `#steering-panel` on all six pages; anonymous → `remove()` post-whoami). The admin-only **nav links** already ship `hidden` (phase-19 contract) — the flashing control is the **steering toggle button labeled "Tuning"**, which ships visible in all six pages and is removed only after `/api/whoami` resolves.
## Objective
Kill the anonymous flash: the tuning toggle ships `hidden` in every page's markup and is revealed only when whoami says admin (the exact ship-hidden / reveal-for-admin contract the nav links use), so an anonymous user never sees the "Tuning" button — not for a single frame.
## Dependencies
- `39_configurable_brand` (complete; last existing phase) — current header state: full shared bar on all six pages.
- `19_shared_header` / `16_admin_auth` / `34_consistent_navbar` (complete) — the `fetchIsAdmin()` gate, the ship-hidden nav contract, and the module-owned steering controls this task modifies.
## Tasks
1. `01_toggle_ships_hidden.md` — add `hidden` to `#steering-toggle` in all six pages and reveal-for-admin in `header.js`; pin at source level.
2. `02_flash_e2e_and_regression.md` — story E2E suite (never-visible-for-anonymous, admin reveal, nav-contract regression) + regression pass + commit.
## Testing & Quality
- Unit: new `tests/unit/test_steering_toggle_visibility.py` — `hidden` present on `#steering-toggle` in all six HTML pages; `header.js` unhides for admin (line before `refreshSteering()`) and the anonymous `remove()` path is intact; any existing source-pin test asserting the exact old markup is updated (check `tests/unit/test_shared_header.py`, `test_steering.py`).
- Coverage: frontend-only — the `app/` >90% gate is unaffected (must stay unchanged).
- E2E (mandatory, A16): `tests/e2e/test_tuning_toggle_flash.py`, run in isolation.
## Completion Criteria
- [ ] Anonymous load of every page: the toggle is never visible (MutationObserver records zero visible frames) and is absent from the DOM after load.
- [ ] Admin load: toggle visible, panel opens, count badge correct — admin behavior unchanged.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged.
- [ ] `uv run pytest tests/e2e/test_tuning_toggle_flash.py -v --no-cov` green in isolation.
- [ ] Regression E2E suites green in isolation: `test_shared_header.py`, `test_global_tuning.py`, `test_steering.py`, `test_tuning_nav_link.py`, `test_smoke.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] UI Structure Check (AGENTS.md rule 5): no new focus targets; landmarks/contrast unchanged; no CDN.
- [ ] One `--no-gpg-sign` commit; phase dir moved `.agent/phases/todo/` → `.agent/phases/complete/`.
## Locked decisions
- **A10 untouched** — no API change; the fix is pure UI visibility off the existing `/api/whoami` gate.
- **A11 untouched** — no new assets, no CDN.
- **Phase-16 contract preserved** — anonymous still gets "absent, not hidden" (remove-from-DOM); this phase only removes the pre-whoami flash window.
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
@@ -0,0 +1,26 @@
# Task 01 — Toggle ships hidden, admin-only reveal
**Phase:** `40_tuning_toggle_flash` · **Source:** `TODO.md:3` — "Loading the page briefly shows the 'Tuning' button in the header even when the user isn't authenticated. Only show that if the user is authenticated."
**Story:** `.agent/user_stories/tuning-toggle-flash.md`
## Objective
Ship `#steering-toggle` `hidden` in all six pages and unhide it in `header.js` only when whoami says admin — zero flash for anonymous, identical admin UX.
## Work
1. `frontend/index.html`, `frontend/sources.html`, `frontend/document.html`, `frontend/git-sources.html`, `frontend/login.html`, `frontend/tuning.html` — add the `hidden` attribute to the existing `#steering-toggle` `<button>` (the element that ships `aria-expanded="false" aria-controls="steering-panel"`; keep every other attribute, icon, label, and count badge byte-identical). The `#steering-panel` section already ships `hidden` — do not touch it.
2. `frontend/assets/header.js` — in `initSharedHeader()`, in the `if (admin)` branch, add `if (steeringToggle) steeringToggle.hidden = false;` **before** `if (steeringPanel) refreshSteering();`. The anonymous branch (`steeringToggle?.remove(); steeringPanel?.remove();`) stays byte-identical. Update the module docstring: the steering controls are now ship-hidden / reveal-for-admin (2026-08-27, `TODO.md` L3), matching the admin-only nav links.
3. `tests/unit/test_steering_toggle_visibility.py` (new) — source pins in the house style (regex/substring over the HTML + JS files, see `tests/unit/test_sync_button.py`):
- `#steering-toggle` carries `hidden` in **all six** pages;
- `header.js` contains the admin unhide (`steeringToggle.hidden = false`) inside `initSharedHeader`;
- the anonymous removal (`steeringToggle?.remove()`) is still present;
- `#nav-tuning` still ships `hidden` (the contract this phase relies on).
4. Grep the existing suites for exact-markup pins of the toggle (`tests/unit/test_shared_header.py`, `tests/unit/test_steering.py`, `tests/e2e/test_global_tuning.py`, `test_steering.py`) and update any assertion that breaks purely because of the new `hidden` attribute — behavior assertions stay.
## Testing & Quality
- Unit: the new pin file above; full unit suite green.
- Coverage: **>90%** on `app/` (no Python change — TOTAL must be unchanged; run `uv run pytest --cov=app --cov-report=term-missing`).
## Completion Criteria
- [ ] All six pages ship `#steering-toggle` with `hidden`; `header.js` reveals for admin and still removes for anonymous.
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
- [ ] No behavior change in completed work (admin steering flow byte-identical: open/close, count badge, delete, announcer).
@@ -0,0 +1,26 @@
# Task 02 — Flash E2E + regression + commit
**Phase:** `40_tuning_toggle_flash` · **Source:** `TODO.md:3` — "Loading the page briefly shows the 'Tuning' button in the header even when the user isn't authenticated. Only show that if the user is authenticated."
**Story:** `.agent/user_stories/tuning-toggle-flash.md`
## Objective
Prove the flash is gone at the browser level (never visible, not even for a frame) and that the shared-header contract is intact; commit the phase.
## Work
1. `tests/e2e/test_tuning_toggle_flash.py` (new) — mock-only suite (DB up), per the story's Playwright Mapping Rule:
- a helper `install_visibility_observer(page)`: `page.add_init_script` a MutationObserver on `document.documentElement` that appends to `window.__tuningVisibleFrames` every time `#steering-toggle` is added/attribute-changed and is both in the DOM **and** not `[hidden]` (check `el.offsetParent !== null` or `!el.hidden`);
- `test_anonymous_never_sees_toggle` — load `/` anonymously, wait for network idle + header settle (whoami resolved), assert `window.__tuningVisibleFrames` is empty and `#steering-toggle` is absent from the DOM;
- `test_anonymous_other_pages_never_flash` — same on `/sources.html`, `/tuning.html`, `/login.html`;
- `test_admin_toggle_revealed_and_working` — `login()` (e2e.auth_helpers), reload `/`, toggle visible + clickable (opens `#steering-panel`, `aria-expanded="true"`), count badge matches the list;
- `test_nav_contract_regression` — anonymous: `#nav-sources` / `#nav-git-sources` / `#nav-tuning` stay hidden; admin: revealed.
2. Regression pass (isolation runs, per A16): `test_shared_header.py`, `test_global_tuning.py`, `test_steering.py`, `test_tuning_nav_link.py`, `test_smoke.py` — all green; fix only true regressions.
3. `uv run pytest` (unit+integration) green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `fix(header): ship the tuning toggle hidden — no anonymous flash`, staging this phase's changed files; move `.agent/phases/todo/40_tuning_toggle_flash/` → `.agent/phases/complete/` (force-add per AGENTS.md rule 8 if the history tracks the tree).
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_tuning_toggle_flash.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`).
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
## Completion Criteria
- [ ] The story E2E file passes in isolation; the four regression suites pass in isolation.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,38 @@
# Phase 41 — Sync fails fast + modal when a model is down
**Source:** `TODO.md` L4 — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
**Context:** `app/api/sync.py::_run_sync` (phase 32/35/38) runs source resolution → git clones → `import_sources` (embeds) → overview (lite) — with a dead LLM endpoint the run discovers it only mid-import, after slow clones. The sync state machine is module-owned by `frontend/assets/header.js` (`applySyncFailure` → button title/aria + `.is-error` + `bor:sync-status` event; the Sources page renders `#sync-error-banner`). No dialog component exists yet.
## Objective
When `embed` or `lite` is unreachable, the sync fails **before any expensive work** with a message naming the model, and the failure is readable in a **modal dialog** on every page that carries `#sync-btn`.
## Dependencies
- `40_tuning_toggle_flash` (todo) — current shared-header state (sequential; no code overlap, but both touch `header.js` — keep this phase's changes confined to the sync section).
- `32_admin_sync_button` / `35_git_sources_admin` / `38_local_directory_sources` (complete) — the pipeline, the status contract, and the module-owned button lifecycle this phase extends.
## Tasks
1. `01_model_probe_fail_fast.md` — `check_models()` probe in `app/rag/llm.py`, called first in `_run_sync`; unit + integration tests.
2. `02_sync_error_modal.md` — `header.js` modal (built in JS, all pages) + CSS; source pins.
3. `03_model_down_e2e_and_commit.md` — dedicated E2E suite (dead-LLM module app) + phase-32 regressions + commit.
## Testing & Quality
- Unit: probe success/failure paths with a fake LLM client (embed-down, lite-down, both up); the sync task's fail-fast ordering (probe before source resolution — assert no clone call happens).
- Integration: `POST /api/sync` with a stubbed failing client → `GET /api/sync/status` reaches `failed` with the model-naming error; healthy path regression.
- Coverage: **>90%** on `app/` including the new probe code.
- E2E (mandatory, A16): `tests/e2e/test_sync_model_down.py`, run in isolation.
## Completion Criteria
- [ ] With a dead LLM endpoint: sync fails within seconds, **before** any clone, error names the unavailable model; the modal shows it; button settles retry-ready.
- [ ] Modal contract: `role="alertdialog"`, `aria-modal`, text via `textContent`, close via button / `Esc` / backdrop, focus in-and-out.
- [ ] Healthy sync pipeline (clone → import → overview) unchanged — phase-32 suite green in isolation.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
- [ ] `uv run pytest tests/e2e/test_sync_model_down.py -v --no-cov` green in isolation (DB up).
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **A12 untouched** — still in-process, no queue, no new service; the probe is two cheap model calls.
- **A10 untouched** — no new endpoint; `/api/sync` + `/api/sync/status` keep their shapes (a model failure is just another `failed` state).
- **Phase-32 contract kept** — 2 s poll, 202/409, no client timeout, `bor:sync-status` event, button title/aria affordance, Sources banner (the modal is additive).
- **Owner-locked (2026-08-27, roadmap A4):** probe runs **before** git clones; the modal is the primary failure surface on every page.
@@ -0,0 +1,37 @@
# Task 01 — Model probe: fail fast before any clone
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
## Objective
The sync run verifies both models it needs (`embed` + the summary `lite`) **first** — before source resolution, before any `clone_or_pull` — and fails the run with a clear, model-naming error when either is unreachable.
## Work
1. `app/rag/llm.py` — add `class ModelUnavailableError(LLMError)` and:
```python
async def check_models(llm: LLMClient) -> None:
"""Verify the models a sync needs (embed + summary) before any
expensive work; raise ModelUnavailableError naming the model."""
```
- `await llm.embed_one("sync model check")` — wrap `EmbeddingError` (and any other exception) in `ModelUnavailableError`: message names the **embedding model** (use `llm.settings.llm_embed_model`, e.g. "The embedding model ('embed') is not available — check the model endpoint and retry.")
- `await llm.chat([{"role": "user", "content": "ping"}])` (defaults to `llm_summary_model`) — wrap `LLMError`/other in `ModelUnavailableError` naming the **summary model** (`llm.settings.llm_summary_model`, e.g. "The summary model ('lite') is not available — check the model endpoint and retry.").
- Docstring notes the probe is deliberately tiny (one short embedding + one 1-token-scale completion) and that the sync sanitizer downstream still masks any embedded credentials.
2. `app/api/sync.py` — in `_run_sync()`, construct `llm = LLMClient()` **before** the DB/source block and call `await check_models(llm)` as the **first** pipeline step (before `effective_sources`, before the clone loop). Update the module docstring's pipeline list (the probe is step 1: "verify `embed` + summary model availability — fail fast before any clone") and renumber. `ModelUnavailableError` falls into the existing `except Exception` → `failed` state with the sanitized error (no special-casing needed — verify the message survives `_sanitize_error` unchanged).
3. `tests/unit/test_sync_model_probe.py` (new) — with a fake LLM client (duck-typed `embed_one`/`chat`, see `tests/fakes.py` `FakeEmbedder` for the shape):
- both up → `check_models` returns, both methods called;
- embed raises → `ModelUnavailableError` mentioning the embed model name, `chat` never called;
- chat raises → `ModelUnavailableError` mentioning the summary model name;
- message content assertions (model name present, "not available" wording).
4. `tests/integration/test_sync_api.py` — extend:
- **fail-fast:** monkeypatch `app.api.sync.LLMClient` (or `check_models`) so the probe raises `ModelUnavailableError`; also monkeypatch `clone_or_pull` to *assert it is never called*; `POST /api/sync` → poll `GET /api/sync/status` until terminal → `state == "failed"`, `error` names the model;
- **ordering:** a spy on `effective_sources` shows the probe ran before it;
- **healthy regression:** the existing success/failure tests stay green (they stub the LLM — the stub must now satisfy the probe: `FakeEmbedder` already implements `chat`; if the existing stub lacks `embed_one`, add it — `FakeEmbedder.embed` exists, so subclass or delegate).
## Testing & Quality
- Unit/integration as above; the probe must be covered (success + both failure modes) to keep `app/` **>90%**.
- `uv run pytest tests/unit/test_sync_model_probe.py tests/integration/test_sync_api.py -v` green; full suite green.
## Completion Criteria
- [ ] `check_models` exists, is called first in `_run_sync`, and names the failing model in `ModelUnavailableError`.
- [ ] A dead-model sync fails **before** any clone (spy-asserted) with a sanitized, model-naming error in the `failed` state.
- [ ] Healthy pipeline behavior unchanged (existing sync integration tests green).
@@ -0,0 +1,29 @@
# Task 02 — Sync error modal (module-owned, every page)
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
## Objective
A readable, accessible modal dialog for sync failures, built by the shared header module (which owns the sync state machine), so every page carrying `#sync-btn` gets it with zero page-markup changes.
## Work
1. `frontend/assets/header.js` — in the sync section, add:
- `showSyncModal(error)`: lazily create the dialog **once** and append to `document.body` (module-level `let syncModal = null`):
- backdrop `<div class="sync-modal-backdrop">`;
- panel `<div class="sync-modal" role="alertdialog" aria-modal="true" aria-labelledby="sync-modal-title" aria-describedby="sync-modal-error">` with an `<h2 id="sync-modal-title">Sync failed</h2>`, a `<p id="sync-modal-error">` whose text is set via **`textContent`** (the sanitized error — XSS-safe, never `innerHTML`), and a `<button type="button" class="sync-modal-close" aria-label="Close error dialog">` (×);
- opening: add a `.is-open` class (or remove `hidden`), move focus to the close button, remember `document.activeElement` (expected `#sync-btn`);
- closing: reverse (focus returns to the remembered element — `#sync-btn` when present), `Esc` keydown on `document` while open, backdrop click (click on the backdrop element itself, not the panel), and the close button all call the same close function; a second failure while open **updates the error text in place** (no stacking).
- call `showSyncModal(status.error)` from `applySyncFailure(status)` **after** the existing button-title/aria/`.is-error` + `emitSyncStatus` lines (those stay byte-identical — the Sources page's `#sync-error-banner` keeps rendering off the event).
- null-safe: everything guards on `syncBtn`/`document.body`; pages without `#sync-btn` never create the modal (the function is only reachable from the sync state machine).
- Update the module docstring's sync bullet: the failed state now also opens the module-owned error modal (2026-08-27, `TODO.md` L4).
2. `frontend/assets/styles.css` — `.sync-modal-backdrop` (fixed, full-viewport, `rgba` dim over the page, `z-index` above the header) + `.sync-modal` (centered panel, max-width ≈28rem, the dark-theme **error palette** from PLAN §7.2: panel on the error-surface `#2d1318` family, text `#fca5a5`-class ink, 1px error border; title in ink, error text ink-soft-on-error-surface ≥4.5:1); open/close via `.is-open` (visibility/opacity, no motion under `prefers-reduced-motion`); the close button keeps the global `:focus-visible` 3px outline; 44px touch floor.
3. `tests/unit/test_sync_button.py` — add source pins (house style): `header.js` contains `role="alertdialog"`, the `textContent` assignment of the modal error, the `Esc` close binding, the backdrop-click close, focus return to `#sync-btn`, and the `showSyncModal` call inside `applySyncFailure` (after `emitSyncStatus`); `styles.css` carries the `.sync-modal` rules + the reduced-motion stilling. Update any pin that asserts the exact `applySyncFailure` body.
## Testing & Quality
- Unit: the pins above; full suite green (no `app/` change — coverage TOTAL unchanged).
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] `applySyncFailure` opens the modal with the sanitized error; button title/aria + `bor:sync-status` event behavior byte-identical.
- [ ] Modal: `role="alertdialog"`, `aria-modal`, labeled, `textContent`-rendered error, close via button/`Esc`/backdrop, focus in-and-out to `#sync-btn`.
- [ ] No page HTML changed (the modal is JS-built); no CDN (A11).
@@ -0,0 +1,28 @@
# Task 03 — Model-down E2E + regressions + commit
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
## Objective
Prove the whole story in the browser against a **dead model endpoint** — fast failure, readable modal, dismissal, unchanged secondary surfaces, and an untouched healthy pipeline — then commit the phase.
## Work
1. `tests/e2e/test_sync_model_down.py` (new) — mock-only, DB up, git on PATH. Follow the `test_sync_button.py` module-app pattern, but boot **two** module-scoped apps on distinct ports (import `APP_PORT`, `ADMIN_PASSWORD`, `SESSION_SECRET`, `_wait_http` from `e2e.conftest`; use e.g. `APP_PORT + 41` for the dead-model app so the isolated run never clashes with a session app):
- **dead-model app** env: `BOR_LLM_BASE_URL=http://127.0.0.1:9/v1` (closed port — instant connection refused), `BOR_GIT_SOURCES=file://<the test_sync_button fixture repo pattern>` (a local `file://` fixture repo, built the same way `test_sync_button.py` does — a (regressed, non-fail-fast) run would therefore spend real time cloning before failing), its own `BOR_SOURCES_DIR` under `tmp_path`;
- `test_model_down_fails_fast_with_modal` — admin login, click `#sync-btn`; expect (budget ≤ ~10 s, contrast with the 60 s healthy budget) the button settling retry-ready **and** the modal visible: `role="alertdialog"`, title "Sync failed", error text naming the model ("embedding model" / the model id), `aria-modal="true"`;
- `test_modal_dismissal` — one fresh failure, then close via the × button (focus returns to `#sync-btn`); a fresh failure, close via `Esc`; a fresh failure, close via backdrop click;
- `test_sync_error_surfaces_unaffected` — after a failure the button keeps `title` + `.is-error`; on `/sources.html` (same dead-model app) the `#sync-error-banner` renders the error off `bor:sync-status`;
- `test_healthy_sync_still_succeeds` — a **healthy** module app (same port scheme, `BOR_LLM_BASE_URL` = the session mock like `test_sync_button.py`) runs the full pipeline to "Synced HH:MM" (counts + idempotency as in phase 32) — proves the probe didn't break the happy path.
- Module fixture teardown: terminate both apps (the conftest pattern).
2. Regression pass (isolation runs): `tests/e2e/test_sync_button.py` (phase 32 — must stay green unmodified), `test_git_sources_admin.py`, `test_local_directory_sources.py`.
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(sync): fail fast with a modal when a model is unavailable`, staging this phase's files; move `.agent/phases/todo/41_sync_fail_fast_models/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_sync_model_down.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (the probe code is fully covered by task 01's tests).
## Completion Criteria
- [ ] The model-down suite passes in isolation: fast fail before clones, modal with model-naming error, all three dismissal paths, secondary surfaces intact, healthy run unaffected.
- [ ] Phase-32/35/38 regression suites green in isolation.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,36 @@
# Phase 42 — No reply autoscroll
**Source:** `TODO.md` L5 — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
**Context:** Phase 18 ("follow-the-bottom", owner choice 2026-08-23) added `NEAR_BOTTOM_PX = 200` / `isNearBottom()` / `scrollReveal(wrap, behavior, force)` in `frontend/assets/app.js`: the page auto-scrolls on every `thinking` / `tool` / `delta` frame while the user is within 200px of the bottom. The owner now finds that fighting their own scroll. The gate and the per-frame scrolls are **removed**; scrolling happens only on explicit user intent (submit, restore landing).
## Objective
The chat page never auto-scrolls during a turn. The viewport moves only when the user submits (their message is revealed) or when a persisted conversation is restored (one-shot landing) — both user-initiated.
## Dependencies
- `41_sync_fail_fast_models` (todo) — sequential execution only (no code overlap).
- `18_follow_bottom_scroll` (complete) — the code being removed; `14_chat_persistence` (complete) — the restore landing that must survive; `17_thinking_display` / `11_long_answers` (complete) — the thinking window-pin and long-answer behavior this phase must not break.
## Tasks
1. `01_remove_autofollow.md` — strip the phase-18 gate + per-frame scrolls from `app.js`; rewrite the unit pin for the new contract.
2. `02_no_autoscroll_e2e_and_commit.md` — replace the phase-18 E2E with the inverse-contract suite + regressions + commit.
## Testing & Quality
- Unit: `tests/unit/test_frontend_scroll.py` **rewritten** — pins the new contract: no `NEAR_BOTTOM_PX` / `isNearBottom` in `app.js`; the scroll helper scrolls unconditionally (smooth / reduced-motion-aware); the user-submit path scrolls; the thinking/tool/delta handlers contain **no** page-scroll call; the restore landing keeps its one-shot forced scroll.
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
- E2E (mandatory, A16): `tests/e2e/test_no_reply_autoscroll.py`, run in isolation. `tests/e2e/test_follow_bottom_scroll.py` is **deleted** (behavior intentionally removed by owner direction 2026-08-27).
## Completion Criteria
- [ ] During thinking / tool / answer streaming, `window.scrollY` is stable (±1px) while the viewport is scrolled up.
- [ ] Submit still reveals the user's message; reload still lands one-shot on the latest message.
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
- [ ] `uv run pytest tests/e2e/test_no_reply_autoscroll.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_thinking_display.py`, `test_chat_persistence.py`, `test_long_answers.py`, `test_smoke.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **Owner direction (2026-08-27, roadmap A1)** revises the phase-18 owner choice (2026-08-23): follow-the-bottom auto-follow is removed; submit-reveal + restore-landing are kept. Recorded in the story file and the `app.js` docstring (PLAN.md §7.4's Scroll row is a PLAN-side revision to be noted by the owner — this phase does not edit PLAN.md).
- **A15 unchanged** — the SSE contract is untouched; this is pure client-side behavior.
- **The thinking window's internal pin** (`textEl.scrollTop`, phase 17) is untouched here — phase 43 reworks it separately.
- **A16/A17 honoured** — one story E2E suite (replacing the removed one), one atomic commit.
@@ -0,0 +1,35 @@
# Task 01 — Remove the auto-follow gate and per-frame scrolls
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md:5` — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
## Objective
`app.js` scrolls only on explicit user intent: sending a message and the phase-14 restore landing. No scroll happens anywhere in the streaming path.
## Work
1. `frontend/assets/app.js` —
- **Delete** `export const NEAR_BOTTOM_PX = 200` and `function isNearBottom()`.
- **Simplify** `scrollReveal(wrap, behavior = SCROLL, force = false)` → an unconditional `wrap.scrollIntoView({ behavior })` (keep the `SCROLL` constant: smooth, `auto` under `prefers-reduced-motion`; keep the "Calm, don't remove" comment). Rename the gate comment block: the phase-18 "follow-the-bottom scroll contract" paragraph is replaced by the new contract — *"No reply autoscroll (owner direction 2026-08-27, `TODO.md` L5): the page never auto-scrolls while a turn streams. The only scroll call sites are the user submit (reveal my message) and the phase-14 restore landing (one-shot, load-time)."*
- **`addMessage(who, html, scrollBehavior = SCROLL, force = false)`** → change the signature to `addMessage(who, html, scroll = false)`: the internal `scrollReveal(wrap, scrollBehavior, force)` becomes `if (scroll) scrollReveal(wrap)`. Update the call sites (line numbers are pre-change anchors):
- the **user submit** call (`addMessage("user", renderMarkdown(text))`, ~line 896) → `addMessage("user", renderMarkdown(text), true)` (my message must be revealed — the owner-kept behavior);
- the **phase-14 restore** calls (~lines 772/775: `addMessage("user", …, "auto", true)` / `addMessage("brain", …, "auto", true)`) → keep the one-shot forced scroll under the new signature (e.g. `addMessage("user", renderMarkdown(m.text), true)` — the "auto" (non-smooth) behavior for the landing is preserved by passing it through if the new signature keeps a behavior param, otherwise the default `SCROLL` is acceptable and must be noted in the docstring);
- the brain first-bubble creations in the SSE handlers (`if (!wrap) wrap = addMessage("brain", "")`, ~lines 955/978/995, plus the `"…"` fallback ~1004 and the error fallback ~1046) → `scroll: false` (default) — a streaming turn never scrolls the page;
- `addTyping()` (~line 356): the `scrollReveal(wrap)` after `messagesEl.appendChild(wrap)` is **removed** (a typing bubble must not yank the page).
- **SSE handlers** — remove the page-scroll calls: in the `thinking` frame drop the `scrollReveal(wrap); // page follows only while pinned (phase 18)` line **but keep** `textEl.scrollTop = textEl.scrollHeight;` (the thinking *window* pin — phase 17, reworked in phase 43); in the `tool` frame drop its `scrollReveal(wrap);`; in the `delta` frame drop its `scrollReveal(wrap);`.
- Update the file-top docstring's scroll paragraph (lines ~73–81: "Scroll (phase 18, owner choice…)") to the new contract.
2. `tests/unit/test_frontend_scroll.py` — **rewrite** for the new contract (keep the house style — source pins over `app.js`):
- `NEAR_BOTTOM_PX` / `isNearBottom` are **absent** from `app.js`;
- the scroll helper scrolls unconditionally (no `force`-or-near-bottom condition in its body);
- the user-submit `addMessage` call passes the scroll intent; the brain-bubble creation does not;
- the `thinking` / `tool` / `delta` handler bodies contain no `scrollReveal` call (the thinking handler's `textEl.scrollTop` pin is still present);
- the restore landing still performs its one-shot scroll (pin the marker comment / call);
- the `SCROLL` reduced-motion handling is intact.
- Delete the now-obsolete phase-18 test functions (the band constant, the gate logic) — do not leave dead pins.
## Testing & Quality
- Unit: the rewritten pin file + the full suite green (no `app/` change — coverage TOTAL unchanged).
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] No page scroll happens in the streaming path (grep-verifiable + unit-pinned); submit and restore landing still scroll.
- [ ] `uv run pytest` green; the thinking window-pin and all message rendering are byte-identical elsewhere.
@@ -0,0 +1,27 @@
# Task 02 — No-autoscroll E2E (replaces phase 18) + regressions + commit
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md:5` — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
## Objective
Prove the inverse of the phase-18 contract in the browser: no streaming autoscroll, submit-reveal and restore-landing intact — then delete the obsolete phase-18 suite and commit.
## Work
1. `tests/e2e/test_no_reply_autoscroll.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule:
- `test_no_autoscroll_during_long_answer` — `LONG_ANSWER_TRIGGER` question (the mock's ~8 s long answer); once the answer starts streaming, `window.evaluate` a scroll up ~2× the answer's height; sample `window.scrollY` across ≥10 frames (and after `done`): stable within 1px;
- `test_no_autoscroll_during_thinking` — `THINKING_TRIGGER` question; scroll up during the ~4.5 s thinking stream; viewport stable across chunks (no per-chunk page follow);
- `test_submit_reveals_user_message` — in a populated conversation scrolled to the very top, send a question; after send the user's message is in view (its bounding box within the viewport);
- `test_restore_landing_one_shot` — settle a conversation (phase-14 persistence), reload; the page lands on the latest message and stays (no further movement while idle);
- `test_answer_content_intact` — the long answer completes with sources; a thinking turn persists + restores (collapsed block, phase 17).
2. **Delete** `tests/e2e/test_follow_bottom_scroll.py` (its behavior is intentionally removed — owner direction 2026-08-27; the unit pin was rewritten in task 01).
3. Regression pass (isolation runs): `test_chat_rag.py`, `test_thinking_display.py`, `test_chat_persistence.py`, `test_long_answers.py`, `test_smoke.py` — all green; fix only true regressions.
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `fix(chat): stop autoscrolling while a reply streams (owner direction)`, staging this phase's files (including the deleted E2E); move `.agent/phases/todo/42_no_reply_autoscroll/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_no_reply_autoscroll.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
## Completion Criteria
- [ ] The new suite passes in isolation; the phase-18 suite is gone; the five regression suites pass in isolation.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,38 @@
# Phase 43 — Thinking scroll back (user scroll + generate-time autoscroll)
**Source:** `TODO.md` L7 — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
**Story:** `.agent/user_stories/thinking-scroll-back.md`
**Context:** Phase 17 streams reasoning into the collapsible `.thinking` block with a per-chunk bottom-pin (`textEl.scrollTop = textEl.scrollHeight` in the `thinking` SSE handler). Phase 21 (owner choice 2026-08-24) made `.thinking-text` a no-scroll live tail: `overflow-y: hidden` (the JS pin is the sole scroller). The owner now reverses phase 21: the window is user-scrollable again, and the pin becomes **gated** — follow the tail only while the user is pinned near the window's bottom. This is the window-level successor of the phase-18 pattern (the page-level one is removed in phase 42, which runs first and touches the same `thinking` handler line — order matters).
## Objective
The Thinking block follows its live tail while reasoning is generating **and** the user is at the bottom; a scrolled-up user is never yanked down, and returning to the bottom resumes following.
## Dependencies
- `42_no_reply_autoscroll` (todo) — must run **first**: it strips the page-level scroll from the same `thinking` handler; this phase then reworks the window pin in the cleaned-up handler.
- `17_thinking_display` (complete) — the block, the pin, the auto-collapse on first delta.
- `21_thinking_no_scroll` (complete) — the `overflow-y: hidden` + 320px window being reversed (the 320px clip is kept).
## Tasks
1. `01_window_user_scrollable.md` — CSS: `overflow-y: auto` back, comment replaced (owner direction 2026-08-27).
2. `02_gated_bottom_pin.md` — `app.js`: `THINKING_NEAR_BOTTOM_PX = 32` + gated pin; unit pin rewritten (phase-21 file replaced).
3. `03_thinking_scroll_e2e_and_commit.md` — replace the phase-21 E2E with the new-contract suite + regressions + commit.
## Testing & Quality
- Unit: `tests/unit/test_thinking_no_scroll.py` **deleted**, replaced by `tests/unit/test_thinking_scroll.py` — pins: `overflow-y: auto` + `max-height: 320px` in the `.thinking-text` rule; the 2026-08-27 owner-direction comment; `export const THINKING_NEAR_BOTTOM_PX = 32`; the pin is gated on `isThinkingNearBottom(textEl)` (no unconditional pin).
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
- E2E (mandatory, A16): `tests/e2e/test_thinking_scroll.py`, run in isolation. `tests/e2e/test_thinking_no_scroll.py` is **deleted** (behavior intentionally reversed).
## Completion Criteria
- [ ] Wheel/drag/keyboard move `.thinking-text` (frozen-tail state); computed `overflow-y: auto`, `max-height: 320px`.
- [ ] While pinned at the window bottom: each chunk re-pins to the tail (±1px). Scrolled up: no re-pin across chunks. Return to bottom: following resumes.
- [ ] Auto-collapse on first delta, reduced-motion stillness, answer-bubble scroll (phase 11), restored-collapsed block (phase 17) all unchanged.
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
- [ ] `uv run pytest tests/e2e/test_thinking_scroll.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_thinking_display.py`, `test_chat_persistence.py`, `test_no_reply_autoscroll.py`, `test_smoke.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **Owner direction (2026-08-27, roadmap A2)** reverses the phase-21 owner choice (2026-08-24): the window is user-scrollable again; autoscroll only while pinned near the bottom (32px band). The 320px clip is kept (owner-confirmed).
- **A15 unchanged** — SSE contract untouched; pure client-side.
- **A16/A17 honoured** — one story E2E suite (replacing the removed one), one atomic commit.
@@ -0,0 +1,28 @@
# Task 01 — Window user-scrollable again (CSS + unit pin swap)
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
**Story:** `.agent/user_stories/thinking-scroll-back.md`
## Objective
Restore user scrolling on the Thinking window — `overflow-y: auto`, 320px clip kept, comment updated — and swap the phase-21 unit pins for the new contract so the suite stays green.
## Work
1. `frontend/assets/styles.css` — in the phase-17/21 thinking section, the `details.thinking .thinking-text` rule:
- `overflow-y: hidden;` → `overflow-y: auto;`
- replace the phase-21 comment (*"no user scroll back (owner choice 2026-08-24): the window is a live tail only — the phase-17 JS bottom-pin … is the sole scroller"*) with: *"user-scrollable window (owner direction 2026-08-27, `TODO.md` L7): autoscroll follows the live tail only while the user is pinned near the window's bottom — the phase-17 pin, gated in app.js (task 02: `THINKING_NEAR_BOTTOM_PX`); scrolling up pauses the follow, returning to the bottom resumes it."*
- `max-height: 320px` and **every other declaration in the rule stay byte-identical**; the tightened `p`/`ul` margins rule and the reduced-motion chevron block are untouched.
2. **Delete** `tests/unit/test_thinking_no_scroll.py` (its pins assert the reversed behavior) and create `tests/unit/test_thinking_scroll.py` (house style — source pins, same slicing helpers as the deleted file) with, for now, the CSS contract only:
- the `.thinking-text` rule body contains `overflow-y: auto`, `max-height: 320px`, and the 2026-08-27 owner-direction comment (assert `"owner direction 2026-08-27"` and `"TODO.md L7"`);
- `overflow-y: hidden` / `overflow-y: scroll` are absent from that rule body;
- the phase-17 bottom-pin marker (`textEl.scrollTop = textEl.scrollHeight`) is still present in `app.js` (it becomes gated in task 02 — the pin's existence is asserted now so task 02's diff stays minimal and reviewable).
- The JS-gate pins (`THINKING_NEAR_BOTTOM_PX`, `isThinkingNearBottom`, gated call) are added in task 02 — do not assert them yet.
3. Check `tests/e2e/test_thinking_no_scroll.py` still passes at this checkpoint: it asserts computed `overflow-y: hidden` — **it will fail** (the behavior is intentionally changed). Per the gate, the phase's E2E replacement is task 03; to keep the per-task gate green, **delete** that E2E file in this task as well (its behavior is reversed; task 03 lands the replacement suite). Note the deletion in the final commit message of task 03.
## Testing & Quality
- Unit: `tests/unit/test_thinking_scroll.py` green; full `uv run pytest` green (the deleted E2E file does not run under the unit/integration gate, but the full pytest run must not collect it either — it is gone from the tree).
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] `overflow-y: auto` + 320px clip + new comment in the CSS rule; all other declarations byte-identical.
- [ ] Old unit + old E2E phase-21 files deleted; new unit file pins the CSS contract and the surviving pin marker.
- [ ] `uv run pytest` green at this checkpoint.
@@ -0,0 +1,59 @@
# Task 02 — Gated bottom pin (follow while pinned)
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
**Story:** `.agent/user_stories/thinking-scroll-back.md`
## Objective
The phase-17 per-chunk pin becomes a **gate**: the window follows the live tail only while the user is near its bottom; a scrolled-up user is never re-pinned; returning to the bottom re-arms the pin automatically.
## Work
1. `frontend/assets/app.js` —
- add (near the existing `SCROLL` constant, with the phase-18 comment block already removed by phase 42):
```js
/* Thinking-window follow-the-tail contract (owner direction
* 2026-08-27, `TODO.md` L7): the scratchpad autoscrolls to its live
* tail only while the user is pinned near the window's bottom —
* the 32px band is the "window bottom in view" threshold. Scrolling
* up pauses the follow; returning to the bottom resumes it (the
* check runs on every chunk). Exported so the band is unit-pinned
* (same pattern as TURN_TIMEOUT_MS). */
export const THINKING_NEAR_BOTTOM_PX = 32;
function isThinkingNearBottom(textEl) {
return (
textEl.scrollHeight - textEl.scrollTop - textEl.clientHeight <=
THINKING_NEAR_BOTTOM_PX
);
}
```
- in the `thinking` SSE handler, replace the phase-17 block:
```js
if (block.open) {
textEl.scrollTop = textEl.scrollHeight; // pin the stream to the bottom
}
```
(phase 42 already removed the `scrollReveal(wrap)` line there) with the gated pin:
```js
if (block.open && isThinkingNearBottom(textEl)) {
// Follow the live tail only while the user is pinned to the window
// bottom (owner direction 2026-08-27); a scrolled-up reader is
// never re-pinned — returning to the bottom re-arms the pin.
textEl.scrollTop = textEl.scrollHeight;
}
```
- everything else in the handler (acc, sawThinking, clearTurnTimeout, ensureThinkingBlock, `textEl.innerHTML = renderMarkdown(thinkingAcc)`) stays byte-identical.
2. `tests/unit/test_thinking_scroll.py` — extend (from task 01) with the JS pins:
- `app.js` exports `const THINKING_NEAR_BOTTOM_PX = 32`;
- `isThinkingNearBottom` computes `scrollHeight - scrollTop - clientHeight <= THINKING_NEAR_BOTTOM_PX`;
- the thinking handler's pin is gated — the pin line is preceded by `isThinkingNearBottom(textEl)` in the same `if` (assert the combined condition; assert there is **no** unconditional `if (block.open) { textEl.scrollTop = ... }` left);
- `block.open` is still part of the gate (closed blocks never pin);
- the restore path renders collapsed blocks (phase 17) — keep the surviving assertion from task 01.
3. `uv run pytest` green at this checkpoint (E2E not run by the unit gate; the replacement suite lands in task 03).
## Testing & Quality
- Unit: the extended pin file; full suite green.
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] The pin fires only when the block is open **and** the window is within 32px of its bottom; scrolled-up users are never re-pinned; the gate re-arms on return (by construction — the check runs per chunk).
- [ ] `uv run pytest` green at this checkpoint.
@@ -0,0 +1,28 @@
# Task 03 — Thinking-scroll E2E (replaces phase 21) + regressions + commit
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
**Story:** `.agent/user_stories/thinking-scroll-back.md`
## Objective
Prove the full contract in the browser — user scroll restored, follow-while-pinned, pause-on-scroll-up, resume-on-return, CSS contract, and the phase-11/17 regressions — then commit the phase.
## Work
1. `tests/e2e/test_thinking_scroll.py` (new) — mock-only, DB up. Reuse the phase-21 determinism machinery (`mock_llm.compose_thinking` is already ~2 700 chars ≈ 4.5 s of paced frames, overflowing the 320px window ~2×; the phase-20 hesitation trigger gives a deterministic 4 s frozen-tail state with the block open). Per the story's Playwright Mapping Rule:
- `test_thinking_window_user_scrollable` — frozen tail: focus `.thinking-text`, wheel up / `Home` / mouse-drag up → `scrollTop` moves and earlier content is visible;
- `test_thinking_window_follows_while_pinned` — live stream: at the window bottom, after the 2nd-to-last and the last chunk the window is pinned to the tail (±1px); the last chunk's text renders inside the visible rectangle;
- `test_thinking_window_stops_on_scroll_up` — mid-stream: scroll up ~half the window; over the next ≥5 chunks `scrollTop` stable (±1px);
- `test_thinking_window_resumes_on_return` — from the paused state, set `scrollTop` to the bottom; on the next chunk the window is re-pinned to the tail (±1px);
- `test_thinking_window_css_contract` — computed `overflow-y: auto`, `max-height: 320px`, `scrollHeight > clientHeight` (real clip);
- `test_answer_bubble_still_scrollable` (phase 11) — long answer: page scrolls, bubble overflow untouched;
- `test_restored_collapsed_thinking_unaffected` (phase 17) — settled thinking turn reloads collapsed with full text.
2. Regression pass (isolation runs): `test_thinking_display.py`, `test_chat_persistence.py`, `test_no_reply_autoscroll.py` (phase 42 — the cleaned `thinking` handler must not have lost the phase-42 contract), `test_smoke.py`.
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(chat): thinking window scrolls again, follows the tail only while pinned`, staging this phase's files **including the two deleted phase-21 test files** (`tests/unit/test_thinking_no_scroll.py`, `tests/e2e/test_thinking_no_scroll.py`) and the new unit + E2E files; move `.agent/phases/todo/43_thinking_scroll_back/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_thinking_scroll.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
## Completion Criteria
- [ ] The new suite passes in isolation (all seven tests); the four regression suites pass in isolation.
- [ ] One atomic `--no-gpg-sign` commit covering both deleted and both new test files; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,39 @@
# Phase 44 — Markdown tables (chat, viewer, thinking)
**Source:** `TODO.md` L6 — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
**Story:** `.agent/user_stories/markdown-tables.md`
**Context:** `frontend/assets/markdown.js` is the shared escape-first renderer (no libs, A11): fence protection → escape → inline transforms (`code`, `**bold**`, `*em*`, h1–h3, lists) → paragraph pass → fence restore. It has **no table support** — GFM pipe tables render as one raw `|`-littered paragraph. The renderer serves the chat answer, the document viewer/modal, and the thinking block, so one change covers all three.
## Objective
GFM pipe tables render as semantic, styled, XSS-safe `<table>` elements everywhere the shared renderer runs, with a horizontal-overflow guard for wide tables.
## Dependencies
- `43_thinking_scroll_back` (todo) — sequential only (the thinking block also renders markdown; no shared-file conflict beyond the renderer itself).
- `08_story_dark_tech_theme` (complete) — the palette tokens `.md-table` must use.
- `26_document_modal_viewer` / `10_story_document_viewer` (complete) — the second renderer consumer (viewer/modal).
## Tasks
1. `01_table_renderer_and_styles.md` — table pass in `markdown.js` + `.md-table` CSS.
2. `02_mock_table_trigger.md` — deterministic table answer (incl. a wide table) in `mock_llm.py`.
3. `03_tables_e2e_and_commit.md` — unit pins + story E2E suite + regressions + commit.
## Testing & Quality
- Unit: new `tests/unit/test_markdown_tables.py` — source pins in the house style (regex over `markdown.js` / `styles.css`): the table-protection pass exists and runs **after** the fence pass and **before** the escape pass; cells are escaped + inline-transformed; output carries `class="md-table"`, `<thead>`, `th scope="col"`, and the `.md-table-wrap` wrapper; `styles.css` has the wrapper overflow rule + table borders + reduced-motion-relevant rules. (Behavior is browser-proven by the E2E; unit pins catch silent regressions without a browser — the established frontend pattern.)
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
- E2E (mandatory, A16): `tests/e2e/test_markdown_tables.py`, run in isolation.
## Completion Criteria
- [ ] A pipe table in a chat answer renders `<div class="md-table-wrap"><table class="md-table">` with `<thead>`/`<tbody>`, `<th scope="col">` headers, correct cell texts; no raw `|---|` in the bubble.
- [ ] A wide table scrolls inside its wrapper; the 46rem column does not overflow the page.
- [ ] XSS-safe (escaped cells), fences win over tables, lone pipes stay text.
- [ ] The document viewer/modal renders the same table for a fixture document containing one.
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
- [ ] `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py`, `test_smoke.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **A11 untouched** — still the local ~90-line renderer, no library, no CDN.
- **Owner-locked (2026-08-27, roadmap A3):** scope = GFM pipe tables (header + separator + body); links/blockquotes/hr out of scope; alignment colons parsed but rendered left; wide tables get the `overflow-x: auto` wrapper.
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
@@ -0,0 +1,40 @@
# Task 01 — Table pass in the shared renderer + styles
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
**Story:** `.agent/user_stories/markdown-tables.md`
## Objective
`renderMarkdown` turns GFM pipe-table blocks into semantic tables (XSS-safe, inline markdown in cells), wrapped in a horizontal-overflow container, styled in the dark-tech palette.
## Work
1. `frontend/assets/markdown.js` — in `renderMarkdown(md)`, between step 1 (fence protection) and step 2 (escape + inline transforms), add a **table protection pass** using the same placeholder mechanism as the fences:
- **Detection** (line-oriented over the fence-protected text): a **table block** starts at a line containing `|` whose **next** line is a **separator** — the separator line consists of ≥1 pipe-separated cells, each matching `^\s*:?-+:?\s*$` (allowing a leading/trailing pipe and inter-cell whitespace). The block then extends over every following line that still contains `|` (the body; zero body rows is a valid table — header only). A maximal such block is one table. Anything else (a single `|` in prose, a separator with no `|`-header line above it, a 1-line "table") is left untouched.
- **Extraction:** for each table run, split each line on `|`, drop the leading/trailing empty entries produced by leading/trailing pipes, `trim()` each cell.
- **Cell rendering:** each cell goes through the same inline pipeline as the rest of the text — `escapeHtml(cell)` first (XSS-safe, invariant of the renderer), then the inline transforms (`` `code` ``, `**bold**`, `*em*` — the exact same `.replace` chain step 2 uses; factor the inline chain into a small local helper if it makes the cell path cleaner, keeping the whole-text path byte-identical in output).
- **Assembly:**
```html
<div class="md-table-wrap">
<table class="md-table">
<thead><tr><th scope="col">h1</th>…</tr></thead>
<tbody><tr><td>…</td>…</tr>…</tbody>
</table>
</div>
```
Rows with fewer cells than the header are padded with empty `<td>`; rows with more are truncated to the header width (defensive — the mock and real answers are well-formed). Alignment colons in the separator are **parsed but ignored** (all cells left — owner decision).
- **Placeholders:** reuse the `\u0000CODEn\u0000` array pattern — e.g. push the table HTML into a second array and emit `\u0000TABLEn\u0000`, restored alongside the code blocks in step 4 (update the restore step accordingly; tables inside the protected span are already final HTML — they must not re-enter the paragraph pass, which the placeholder guarantees).
- Update the file's header comment (the ~60-line no-CDN renderer now also does tables — 2026-08-27, `TODO.md` L6).
2. `frontend/assets/styles.css` — near the markdown/content styling (the chat bubble content rules):
- `.md-table-wrap { overflow-x: auto; }` — the wrapper is the scroller;
- `.md-table { border-collapse: collapse; width: 100%; font-size: 0.9rem; }`;
- `.md-table th, .md-table td { border: 1px solid var(--line); padding: 0.4rem 0.6rem; text-align: left; vertical-align: top; }`;
- `.md-table thead th { background: <surface-darker token>; color: var(--ink); }` — pick the existing token that keeps ≥4.5:1 (PLAN §7.2: ink `#e8ebf4` on surface `#121a2e` is 14.5:1 — use the plain surface family, not brand);
- ensure the rule set is inside or consistent with the reduced-motion constraints (no animation involved — nothing to still).
3. Sanity: run an existing markdown-consuming E2E (e.g. `test_chat_rag.py`) to confirm byte-identical output for non-table content (the inline-chain factor, if done, must not change any existing rendering).
## Testing & Quality
- Unit: `tests/unit/test_markdown_tables.py` (new) source pins per the phase overview (pass ordering, escape-first for cells, output markers, CSS rules). Full suite green.
- Coverage: **>90%** on `app/` (unchanged — frontend-only).
## Completion Criteria
- [ ] `renderMarkdown` handles the shapes in the story's acceptance criteria 1–4 (table, XSS cell, fence-wins, non-tables stay text) — verifiable via the unit pins now and the E2E in task 03.
- [ ] No existing rendering changes for non-table markdown (regression suite from step 3 green).
@@ -0,0 +1,40 @@
# Task 02 — Deterministic table answer in the mock
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
**Story:** `.agent/user_stories/markdown-tables.md`
## Objective
The E2E mock serves a byte-stable table answer (plus a deliberately wide table and an XSS cell) on demand, following the existing trigger convention.
## Work
1. `tests/e2e/mock_llm.py` —
- add `TABLE_TRIGGER = "show me a table"` (same case-insensitive-substring convention as `LONG_ANSWER_TRIGGER` / `THINKING_TRIGGER` / `TOOLS_TRIGGER`);
- in `compose_answer(body)`, **before** the default tail-echo branch (and before `DEFLECT_MODE` — a deflection prompt never carries the marker, same reasoning as `SUMMARY_MODE`): when the trigger is in the lowercased user message, return the fixed table answer:
```
Here's the shape, in a table:
| Service | Port | Host |
|---|---|---|
| Caddy | 80 | homelab-gw |
| GitLab | 8929 | homelab-git |
| ntfy | 2087 | homelab-ntfy |
<img src=x onerror=alert(1)>
And the wide one:
| A very long column header to force overflow | Second column with some padding text | Third column | Fourth | Fifth |
|---|---|---|---|---|
| value-one | value-two | value-three | value-four | value-five |
```
(The `<img onerror>` line is the XSS assertion's payload — it must survive the mock byte-for-byte so the E2E can prove the renderer neutralizes it; the wide table guarantees `scrollWidth > clientWidth` inside the 46rem column.)
- keep the answer a plain grounded response (no `DEFLECT_MODE` interplay): the trigger question is asked against an on-topic fixture so the honesty gate is HIGH in the E2E (the suite asserts non-deflection as part of the table test).
2. Update the module docstring's marker list (the file documents every trigger — add the table row).
3. `uv run pytest tests/e2e/mock_llm.py-related unit tests` — run `uv run pytest tests/unit -k "mock" tests/integration -x` (or the mock's existing test file, if any — check `tests/` for mock-specific tests) to prove the new branch breaks no existing flow; the full suite is green (the new branch only fires on the marker).
## Testing & Quality
- Unit/integration: full suite green; the new branch is covered by the E2E (task 03) — if a mock-level unit test file exists, add the table case there so the branch is unit-covered too.
- Coverage: **>90%** on `app/` (mock lives in `tests/` — the gate is unchanged).
## Completion Criteria
- [ ] `TABLE_TRIGGER` returns the fixed table answer (byte-stable), including the XSS line and the wide table; no existing mock behavior changes for marker-less requests.
@@ -0,0 +1,28 @@
# Task 03 — Tables E2E + regressions + commit
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
**Story:** `.agent/user_stories/markdown-tables.md`
## Objective
Prove the table contract in the browser — chat, overflow, XSS, viewer, and the two "not a table" regressions — then commit the phase.
## Work
1. `tests/e2e/test_markdown_tables.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule:
- `test_chat_table_renders` — ask an on-topic question containing `TABLE_TRIGGER` (pick a fixture topic that retrieves HIGH — reuse a question pattern from `test_chat_rag.py`); the brain bubble contains `<div class="md-table-wrap"><table class="md-table">`, a `<thead>` with three `<th scope="col">` (Service/Port/Host), the body cell texts ("Caddy", "8929", …), and **no** `|---|` separator text in the bubble;
- `test_wide_table_scrolls` — in the same answer, the wide table's wrapper has `scrollWidth > clientWidth`; horizontal scrolling (wheel/`scrollLeft`) moves it; the page itself has no horizontal overflow (`document.documentElement.scrollWidth <= clientWidth`);
- `test_table_xss_safe` — the `<img src=x onerror=…>` line renders as visible text (no `<img>` element inside the bubble; `onerror` can never fire — assert `page.evaluate` found zero injected img nodes and the tag text is present);
- `test_viewer_table_renders` — add a fixture document (extend `tests/fixtures/docs/homelab/` with a small `.md` file containing a pipe table — e.g. `tables.md` with a 3×3 table; re-import per the `test_document_documents.py`/`test_import_documents.py` fixture pattern), open it from the Sources table (admin) in the modal; the modal content renders `<table class="md-table">`;
- `test_fence_not_a_table` — a question/fixture whose content puts `|`-heavy lines inside a ``` fence (existing fixtures have fenced blocks — pick/extend one) renders `<pre><code>` with no `<table>`;
- `test_plain_pipe_stays_text` — an off-trigger grounded answer containing a single `|` in prose (assert via an existing deterministic answer or a minimal new fixture) renders as text, no `<table>`.
- Assert non-deflection (`.is-deflected` absent) in the table tests — the honesty gate interplay is part of the contract.
2. Regression pass (isolation runs): `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py` (the renderer is shared — summaries render through it too), `test_smoke.py`.
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(chat): render markdown tables in answers, viewer, and thinking`, staging this phase's files; move `.agent/phases/todo/44_markdown_tables/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
## Completion Criteria
- [ ] The story E2E suite passes in isolation (all six tests); the four regression suites pass in isolation.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,42 @@
# Phase 45 — Agent makes as many tool calls as it wants
**Source:** `TODO.md` L8 — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
**Context:** Phase 37 shipped the grounded-turn agent loop (`app/rag/agent.py::run_agent`) with per-turn budgets — `agent_list_calls` / `agent_read_calls` (default 1 each, `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS`), "budgets-as-kill-switch" locked decision. The exhaustion refusals (`LIST_EXHAUSTED` / `READ_EXHAUSTED`) are where correct multi-document answers die. Owner direction (2026-08-27): remove both budgets; the loop keeps one guard — a configurable **round cap** that also doubles as the no-tools kill switch (`0`).
## Objective
`list_documents` / `read_document` can be called as many times as the model needs (re-lists included), bounded only by `BOR_AGENT_MAX_ROUNDS` (default 10; `0` = no tools, byte-identical to the pre-phase-37 path).
## Dependencies
- `44_markdown_tables` (todo) — sequential only (no shared files: this phase is `app/` + tests + mock).
- `37_agent_document_tools` (complete) — the loop, the `tool` SSE event, the UI tool lines, the per-turn `tool_calls=N` log field, and the phase-37 locked decision being revised.
- `31_kb_overview_prompt` (complete) — the `lite` one-shot path is untouched by this phase.
## Tasks
1. `01_config_round_cap.md` — the server core, atomically: `agent_max_rounds` replaces the budgets in `app/config.py` + `app/rag/agent.py`, unit + integration rewrites, `.env.example` (one task so the per-task gate stays green).
2. `02_mock_multi_read_flow.md` — the E2E mock's deterministic multi-read (list → read #1 → read #2 → answer) flow.
3. `03_unlimited_tools_e2e_and_commit.md` — story E2E + phase-37 regression + PLAN.md revision note + commit.
## Testing & Quality
- Unit: `tests/unit/test_agent.py` rewritten around the round cap (always-calling mock LLM: N tool rounds then a forced `tools=None` final answer; `max_rounds=0` → exactly one request with `tools=None`; rejected-call spam — unknown tool / already-in-context — is bounded by the cap, not by budgets; re-lists execute and count in `tool_calls`); `tests/unit/test_config.py` (default 10, `BOR_AGENT_MAX_ROUNDS` override, `0`, the budget env vars are gone).
- Integration: `tests/integration/test_chat_api.py` — the `agent_list_calls=0, agent_read_calls=0` fixtures become `agent_max_rounds=0`; the tool SSE event shape and the `done.sources` extension assertions stay.
- Coverage: **>90%** on `app/` — `agent.py` + `config.py` fully covered.
- E2E (mandatory, A16): `tests/e2e/test_agent_unlimited_tools.py`, run in isolation.
## Completion Criteria
- [ ] `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS` are gone (config, `.env.example`, agent, tests); no exhaustion refusal remains.
- [ ] A multi-read turn (list + 2 reads) streams three tool lines, answers non-deflected, and `done.sources` lists the retrieved doc(s) + both reads deduped.
- [ ] `agent_max_rounds=0` → single `tools=None` request (kill switch); at the cap the loop forces a final no-tools answer (log warning kept).
- [ ] `tool` SSE event shape and `tool_calls=N` per-turn log field unchanged.
- [ ] `.agent/PLAN.md` carries the phase-37 revision note (owner permission 2026-08-27, `TODO.md` L8) — the only PLAN edit in this phase.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
- [ ] `uv run pytest tests/e2e/test_agent_unlimited_tools.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_agent_document_tools.py`, `test_chat_rag.py`, `test_smoke.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **Owner-locked revision (2026-08-27, roadmap R2):** the phase-37 "budgets-as-kill-switch" decision is **revised** — both per-tool budgets removed; `BOR_AGENT_MAX_ROUNDS` (default 10) is the only loop guard and the kill switch (`0`). Recorded as a PLAN.md revision note (the established owner-permission pattern, like the A10/A7/A9/A15 notes) — a recorded revision, not a silent deviation (AGENTS.md rule 3).
- **A15 extension unchanged** — the `tool` SSE event shape, the `done` shape, and the per-turn log line (`tool_calls=N`) are untouched; the revision note amends the phase-37 note's budget wording only.
- **Rejections kept:** `Unknown tool.`, `MISSING_READ_ARGS`, `Already in your context.` (non-budget rejections; the cap bounds their pathological repetition).
- **A17 honoured** — one atomic commit.
@@ -0,0 +1,52 @@
# Task 01 — Server core: round cap replaces the budgets (config + loop + unit/integration)
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md:8` — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
## Objective
One coherent server-side change, landed atomically so the suite is green at the checkpoint: `agent_max_rounds` (`BOR_AGENT_MAX_ROUNDS`, default 10; `0` = no tools) replaces both per-tool budgets in config, the agent loop, and every test that pins them.
## Work
1. `app/config.py` —
- **delete** the `agent_list_calls` and `agent_read_calls` fields (with docstrings);
- **add** in their place, same "RAG tuning" section:
```python
#: Hard cap on the agent tool rounds per grounded turn (phase 45,
#: revising phase 37's per-tool budgets — owner permission
#: 2026-08-27, TODO L8: "allow the LLM to make as many tool calls
#: as it wants"). Every tool call the model emits consumes a
#: round; at the cap the loop forces one final no-tools answer.
#: ``0`` disables the tools entirely — the turn is a single
#: request with ``tools=None`` (the pre-phase-37 path — the kill
#: switch).
agent_max_rounds: int = 10
```
- optional: a `field_validator` rejecting negative values (note it in the docstring if added).
2. `app/rag/agent.py` —
- `run_agent`: `max_rounds = settings.agent_max_rounds`; `tools = AGENT_TOOLS if max_rounds > 0 else None` (the kill switch — at 0 the loop makes exactly one request with `tools=None`, byte-identical to the pre-phase-37 path);
- delete `list_left` / `read_left` and the budget-driven `tools = None if (list_left == 0 and read_left == 0) else AGENT_TOOLS` transition — `tools` stays `AGENT_TOOLS` while rounds remain;
- after each executed call: `rounds += 1`; the existing cap branch becomes the **only** forced-exit: `if rounds >= max_rounds:` → the `logger.warning("agent round cap reached …")` + final `chat_stream(messages, tools=None)` (update the warning text: it is no longer belt-and-braces — it is the cap);
- `_execute_tool(db, call, seed_docs, holder)`: drop the `list_left` / `read_left` parameters and the `LIST_EXHAUSTED` / `READ_EXHAUSTED` early returns; keep the `ALREADY_IN_CONTEXT`, `UNKNOWN_TOOL`, `MISSING_READ_ARGS` rejections (non-budget — a repeated rejected call still consumes a *round* in the loop, so a pathological stream is bounded by `max_rounds`); return type simplifies to `str`;
- delete the `LIST_EXHAUSTED` / `READ_EXHAUSTED` constants;
- `AGENT_TOOLS`: `read_document` description "Add the full content of exactly one more indexed document to your context" → "Add the full content of one more indexed document to your context";
- module docstring: the budget paragraph (points 1, 3, 4) rewritten for the round cap (owner revision 2026-08-27, `TODO.md` L8); `run_agent` docstring updated (`seed_docs` note unchanged); `AgentHolder` unchanged (`tool_calls` still counts executed calls — now including re-lists);
- the per-call `logger.info("agent tool=… budget list_left=… read_left=…")` line becomes `logger.info("agent tool=%s args=%s round=%d/%d", …)` (or equivalent — the per-turn `tool_calls=N` field in `app/api/chat.py` is untouched).
3. `tests/unit/test_agent.py` — **rewrite** the budget tests around the round cap (keep the file's fake-LLM harness):
- an always-`list_documents`-calling mock with `agent_max_rounds=3`: exactly 3 tool rounds execute, then one forced `tools=None` request streams the answer; `holder.tool_calls == 3`;
- `agent_max_rounds=0`: exactly one request, `tools=None`, no tool lines, `holder.tool_calls == 0` (kill switch);
- an always-`read_document`-with-unknown-path mock (every call rejected — `No document at …`): the loop runs to `max_rounds` and forces the final answer (rejections no longer end the loop early via budgets, the cap bounds them);
- the existing rejections tests (`Unknown tool.`, `MISSING_READ_ARGS`, `Already in your context.`) keep passing — update their `_settings(...)` calls (`agent_max_rounds=…` instead of the budget kwargs);
- a **re-list** test: `list_documents` called twice in one turn executes both (the second returns the catalog again) and counts 2 in `holder.tool_calls`.
4. `tests/unit/test_config.py` — default 10; `BOR_AGENT_MAX_ROUNDS=0` / `=5` overrides; (negative validator, if added); delete the old budget assertions.
5. `tests/integration/test_chat_api.py` — the `agent_list_calls=0, agent_read_calls=0` fixture kwargs (~line 661) become `agent_max_rounds=0`; any other budget kwarg in the file the same; the tool SSE-event and `done.sources` assertions stay untouched.
6. `.env.example` — the two `BOR_AGENT_*_CALLS` lines become one: `# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools)` (file's optional-setting comment style).
7. Grep the repo for `agent_list_calls|agent_read_calls|BOR_AGENT_(LIST|READ)_CALLS|LIST_EXHAUSTED|READ_EXHAUSTED` — zero hits outside `.agent/phases/complete/**` (history).
## Testing & Quality
- Unit + integration: full `uv run pytest` green at this checkpoint (the mock/E2E multi-read flow lands in task 02 — the existing 3-step mock flow still works unmodified, so `test_agent_document_tools.py` E2E is not yet run by the gate).
- Coverage: **>90%** on `app/` — `agent.py` + `config.py` fully covered by the rewritten tests.
## Completion Criteria
- [ ] No per-tool budgets anywhere in `app/` or `tests/`; `agent_max_rounds` is the single knob (default 10, `0` = kill switch).
- [ ] Re-lists execute; non-budget rejections intact; the cap bounds pathological streams; `tool_calls=N` log field and `tool` SSE event unchanged.
- [ ] `uv run pytest` + coverage gate green at this checkpoint.
@@ -0,0 +1,27 @@
# Task 02 — Mock: deterministic multi-read tool flow
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md:8` — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
## Objective
The E2E mock gains a deterministic **multi-read** agent flow (list → read #1 → read #2 → answer) so "as many tool calls as it wants" is provable statelessly, without disturbing the existing 3-step flow.
## Work
1. `tests/e2e/mock_llm.py` —
- the existing phase-37 flow (documented in the module docstring and `_tool_flow`): marker `TOOLS_TRIGGER` ("use your tools") + `<tools>` system section → step classification **statelessly from the messages**: no tool results yet → `list`; one `tool`-role message with the catalog prefix → `read` (first catalog doc, parsed from the listing via the `rsplit("/", 1)` convention); one `tool`-role message with the `Document <source/path>:` prefix → forced answer.
- add a **multi-read variant**: when the user message contains **both** `TOOLS_TRIGGER` and a new marker `MULTI_READ_TRIGGER = "read two documents"`, the classifier reads the *count* of `tool`-role messages whose content starts with `"Document "` (the read-result prefix, `app.rag.agent`'s `_execute_tool` output):
- 0 read results (+ no catalog yet) → `list`;
- 0 read results (catalog present) → `read` the **first** catalog doc;
- 1 read result → `read` the **second** catalog doc (the listing minus the already-read doc — parse the catalog lines the same way the existing read step does, skipping the path already read);
- 2 read results → forced answer: the existing answer shape (tail echo) plus a deterministic line naming **both** read paths (e.g. `"I read <path1> and <path2>."` — byte-stable) so the E2E can assert the model actually used both;
- the single-read flow (no `MULTI_READ_TRIGGER`) stays byte-identical — the variant must be a strict superset (the existing `test_agent_document_tools.py` E2E keeps passing unmodified).
- update the module docstring's tool-flow documentation (the multi-read steps + the marker).
2. `uv run pytest` green (mock-only change; the existing 3-step E2E is not run by the unit gate but must stay conceptually intact — the regression run in task 03 proves it).
## Testing & Quality
- Unit: full suite green; if a mock-specific unit test file exists (check `tests/unit/`), add the multi-read classification case there (catalog → read #1 → read #2 → answer) so the new branch is unit-covered; otherwise the E2E (task 03) covers it.
- Coverage: **>90%** on `app/` (unchanged — `tests/` only).
## Completion Criteria
- [ ] `TOOLS_TRIGGER` + `MULTI_READ_TRIGGER` → deterministic 4-step flow (list, read #1, read #2, answer naming both paths); the 3-step flow is unchanged for marker-less turns.
- [ ] Full unit/integration suite green.
@@ -0,0 +1,39 @@
# Task 03 — Unlimited-tools E2E + PLAN revision note + commit
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md:8` — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
## Objective
Prove the multi-tool turn end to end, record the phase-37 decision revision in PLAN.md, run the regressions, and commit the phase.
## Work
1. `tests/e2e/test_agent_unlimited_tools.py` (new) — mock-only, DB up (the grounded-turn prerequisite: the fixture KB imported, per the `test_agent_document_tools.py` fixture pattern):
- `test_multi_read_turn` — a grounded question carrying `TOOLS_TRIGGER` + `MULTI_READ_TRIGGER`: the turn streams **three** tool lines (`.tool-call` rows: one `list_documents` — "is listing documents" — and two `read_document` — "is reading <source/path>") in order, then a final non-deflected answer containing the mock's "I read <path1> and <path2>." line;
- `test_done_sources_include_reads` — the source chips under the answer list the retrieval doc(s) **plus both** read documents, deduped (the phase-37 `done.sources` extension contract, now with 2 reads);
- `test_relist_allowed` — the listing tool ran without a "No listing budget left" refusal: assert no refusal text anywhere in the bubble/tool lines (the old refusal strings must be gone — grep the app for them is task 01's job; here assert the UI never shows one);
- `test_single_tool_flow_regression` (phase 37) — the original 3-step flow (marker without the multi-read trigger) still answers after exactly one read with its single tool pair (this may be a targeted re-assertion; the full suite `test_agent_document_tools.py` runs in the regression pass).
2. `.agent/PLAN.md` — **the only PLAN edit in this phase** (owner-locked revision, roadmap R2): in the §4 SSE-revision block, after the phase-37 revision note, add a new note in the established style:
> **SSE revision (phase 45, owner permission 2026-08-27):** the phase-37
> per-turn tool budgets are **removed** (owner: "allow the LLM to make
> as many tool calls as it wants — `TODO.md` L8): `BOR_AGENT_LIST_CALLS`
> / `BOR_AGENT_READ_CALLS` no longer exist; `BOR_AGENT_MAX_ROUNDS`
> (default 10) caps the tool rounds and `0` disables the tools
> entirely (the pre-phase-37 path). The `tool` event shape and the
> `done` shape are unchanged — a recorded revision of the phase-37
> note's budget wording, not a silent deviation.
Also update the phase-37 note's budget clause if it reads as current
truth ("budgeted by `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS`")
by appending "(removed in phase 45 — see the revision note below)".
Touch **nothing else** in PLAN.md (Protocol B: no roadmap-table edit for appended phases).
3. Regression pass (isolation runs): `test_agent_document_tools.py` (phase 37 — must pass **unmodified**), `test_chat_rag.py`, `test_smoke.py`.
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(rag): unbounded agent tool calls behind a round cap (owner revision)`, staging this phase's files **including the force-added `.agent/PLAN.md`** (AGENTS.md rule 8: `git add -f .agent/PLAN.md`) and the phase dir move `.agent/phases/todo/45_agent_unlimited_tools/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_agent_unlimited_tools.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (task 01's rewritten tests carry it).
## Completion Criteria
- [ ] The multi-read E2E suite passes in isolation; the phase-37 suite passes unmodified in isolation.
- [ ] PLAN.md carries the phase-45 revision note (owner permission 2026-08-27) and nothing else changed.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,40 @@
# Phase 46 — Mobile hamburger nav
**Source:** `TODO.md` L9 — "The navbar on mobile is way too squished. Make it a hamburger dropdown menu with a nice animation"
**Story:** `.agent/user_stories/mobile-hamburger-nav.md`
**Context:** The shared bar (phase 19/34 — the same header block on all six pages) carries brand + up to four text nav pills (Chat, `#nav-sources`, `#nav-git-sources`, `#nav-tuning` — the latter three ship `hidden`, revealed for admin by `header.js`) + four action controls (steering toggle, `#sync-btn`, `#new-chat-btn`, sign in/out). At ≤640px the phase-34/35 squeeze rules (0.72rem pills, 0.05rem gaps) leave a bar that is squished and hard to hit. The fix: on mobile the nav links move into an animated hamburger dropdown; the action pills stay in the bar.
## Objective
At ≤640px the nav links live in a `#nav-toggle`-opened dropdown menu (slide+fade, reduced-motion-still) with comfortable targets and the same auth visibility; at >640px the bar is byte-identical to today.
## Dependencies
- `45_agent_unlimited_tools` (todo) — sequential only (no shared files).
- `34_consistent_navbar` / `35_git_sources_admin` (complete) — the six-page bar contract, the admin-only link reveal, and the mobile squeeze rules being superseded.
- `07_story_responsive_polish` (complete) — the ≤640px conventions (44px targets, safe areas).
## Tasks
1. `01_hamburger_markup_all_pages.md` — `#nav-toggle` + `id="app-nav"` on all six pages; the mobile CSS (dropdown, animation, reduced-motion).
2. `02_toggle_behavior.md` — `header.js` open/close behavior (aria, Esc, link-close, resize-close) + source pins.
3. `03_hamburger_e2e_and_commit.md` — story E2E suite (mobile + desktop + reduced motion) + regressions + commit.
## Testing & Quality
- Unit: new `tests/unit/test_hamburger_nav.py` — `#nav-toggle` (with `aria-controls="app-nav"`, `aria-expanded`, `aria-label="Menu"`) present in **all six** pages and absent-visible on desktop (CSS `display: none` outside the media query); `<nav class="app-nav" id="app-nav">` in all six; the mobile CSS block carries the dropdown rules + `.is-open` state + the 180ms transition + the reduced-motion override; the old nav-pill squeeze rules are gone/superseded; `header.js` carries the toggle binding (click, Esc, delegated link close, matchMedia close).
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
- E2E (mandatory, A16): `tests/e2e/test_mobile_hamburger_nav.py`, run in isolation.
## Completion Criteria
- [ ] 375px: hamburger visible (44px target), inline nav hidden, no horizontal bar overflow; menu opens with animation, closes via link/Esc/outside; anonymous sees only "Chat" in the menu, admin sees all four links.
- [ ] `reducedMotion: "reduce"`: no transition, open/close still instant and correct.
- [ ] >640px: no hamburger, inline pills exactly as today (phase-34/35 contract intact).
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
- [ ] `uv run pytest tests/e2e/test_mobile_hamburger_nav.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_nav_consistency.py`, `test_header_consistency.py`, `test_shared_header.py`, `test_responsive_polish.py`, `test_tuning_nav_link.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] UI Structure Check (AGENTS.md rule 5): the toggle is a labeled 44px button with `aria-expanded`/`aria-controls`; the menu keeps `nav aria-label="Primary"`; focus-visible on the new control; contrast ≥4.5:1; no CDN.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **Owner-locked (2026-08-27, roadmap A5):** nav links only in the menu; action pills stay in the bar; slide-down + fade 180ms; `prefers-reduced-motion` stills it; the existing ≤640px breakpoint (no new one).
- **A11 untouched** — no new assets/CDN; the menu reuses the existing `<nav>` element (no duplicated links, so the whoami reveal keeps working unchanged).
- **Phase-34 contract kept** — the same bar on every page; the auth visibility rules apply inside the menu exactly as before.
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
@@ -0,0 +1,42 @@
# Task 01 — Hamburger markup (six pages) + mobile dropdown CSS
**Phase:** `46_mobile_hamburger_nav` · **Source:** `TODO.md:9` — "The navbar on mobile is way too squished. Make it a hamburger dropdown menu with a nice animation"
**Story:** `.agent/user_stories/mobile-hamburger-nav.md`
## Objective
All six pages carry the identical `#nav-toggle` button + `id="app-nav"`, and the ≤640px stylesheet turns the nav into an animated full-width dropdown — desktop untouched.
## Work
1. **Markup — all six pages** (`frontend/index.html`, `sources.html`, `document.html`, `git-sources.html`, `login.html`, `tuning.html`), each in its `<header class="app-header">` block:
- insert the toggle button **immediately before** the `<nav>`:
```html
<!-- Phase 46 (owner permission 2026-08-27, `TODO.md` L9): the
mobile hamburger — visible ≤640px only (CSS); opens the nav as
an animated dropdown. Behavior: assets/header.js. -->
<button type="button" class="nav-toggle" id="nav-toggle"
aria-expanded="false" aria-controls="app-nav" aria-label="Menu">
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M4 7h16M4 12h16M4 17h16"/></svg>
</button>
```
- give the existing nav the id: `<nav class="app-nav" id="app-nav" aria-label="Primary">` (every other attribute/child byte-identical — the links keep their `hidden` attributes; **no links duplicated**).
- keep the six pages visually/structurally identical (the phase-34 contract) — the toggle is part of the shared bar block, positioned the same on every page.
2. `frontend/assets/styles.css` —
- **global (outside media queries):** `.nav-toggle { display: none; }` (desktop: absent);
- **inside the existing `@media (max-width: 640px)` block:**
- `.nav-toggle { display: inline-flex; align-items: center; justify-content: center; width: 44px; height: 44px; padding: 0; color: var(--ink); background: none; border: 0; border-radius: var(--radius-sm); cursor: pointer; }` + `:focus-visible` inherits the global 3px outline + a hover state matching the other pills (the `.steering-toggle:hover` family);
- `.app-nav` becomes the dropdown: `position: absolute; top: 100%; left: 0; right: 0; flex-direction: column; gap: 0; background: var(--surface); border-bottom: 1px solid var(--line); box-shadow: <existing shadow token or 0 8px 24px rgba(0,0,0,.4)>; padding: 0.5rem 0; z-index: <above the header content — check the header's z-index and use header+1>;` — note the containing block is the sticky `.app-header` (`.header-inner` is not positioned), so the menu spans the header's full width, edge to edge — intended on mobile;
- **closed state (default):** `visibility: hidden; opacity: 0; transform: translateY(-8px); pointer-events: none; transition: opacity 180ms ease, transform 180ms ease, visibility 0s linear 180ms;`
- **open state:** `.app-nav.is-open { visibility: visible; opacity: 1; transform: none; pointer-events: auto; transition: opacity 180ms ease, transform 180ms ease, visibility 0s; }`
- **menu rows:** `.app-nav .nav-link { padding: 0.75rem 1.25rem; font-size: 1rem; }` (comfortable 44px+ targets, readable — this **supersedes** the ≤640px pill-squeeze rules for `.nav-link` and `.app-nav` gap in that block: delete/replace the `.nav-link { padding: 0.3rem 0.25rem; font-size: 0.72rem; }` and `.app-nav { gap: 0.05rem; }` rules, keeping the rest of the block);
- **reduced motion:** inside the file's existing `@media (prefers-reduced-motion: reduce)` block (the one covering the 640px rules — or a new one after it): `.app-nav { transition: none; }`;
- the **900px tablet block is untouched** (inline nav still in use at 641–900px); the action-pill rules in the 640px block are untouched; the header height (`--header-h: 58px`) is untouched.
- verify at 360px: brand (clipped clean as today) + hamburger + the four icon action pills fit without horizontal overflow (the old four text pills are gone from the bar — there is now room; if the bar still overflows, the brand ellipsis target absorbs it exactly as before).
## Testing & Quality
- Unit: the new `tests/unit/test_hamburger_nav.py` starts here with the markup + CSS pins (the JS pins land in task 02): toggle markup (attributes) in all six pages; `id="app-nav"` in all six; `.nav-toggle { display: none }` outside media queries; the mobile block carries the dropdown, `.is-open`, the 180ms transition pair, the reduced-motion override, and the superseded squeeze rules are gone; full suite green (behavior not yet wired — the menu is closed by default and CSS-inert without JS, so no E2E regressions at this checkpoint).
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] Six identical toggles + `id="app-nav"`; desktop rendering byte-identical (`.nav-toggle` hidden, nav inline as before).
- [ ] Mobile: closed dropdown is invisible and non-interactive; `.is-open` (added by task 02's JS) will be the only opener.
- [ ] Full suite green at this checkpoint.
@@ -0,0 +1,63 @@
# Task 02 — Toggle behavior in the shared header module
**Phase:** `46_mobile_hamburger_nav` · **Source:** `TODO.md:9` — "The navbar on mobile is way too squished. Make it a hamburger dropdown menu with a nice animation"
**Story:** `.agent/user_stories/mobile-hamburger-nav.md`
## Objective
The menu opens and closes with correct ARIA state, Esc/link/outside dismissal, and desktop-resize cleanup — one module-owned binding, like the existing sign-out/steering bindings in `header.js`.
## Work
1. `frontend/assets/header.js` — a new module-import binding (same pattern as the sign-out binding: look up at import, guard null-safe, no page-script involvement):
```js
/* ---------- mobile hamburger (phase 46; module-owned) ----------
* ≤640px only (CSS hides the button elsewhere): #nav-toggle opens the
* nav as a dropdown (#app-nav .is-open — the animated state, task 01
* CSS). One binding for all six pages; a page without either element
* is a no-op, like the rest of this module. The nav LINKS keep their
* ship-hidden whoami contract (hidden links stay hidden inside the
* menu) — this binding only toggles the container. */
const navToggle = document.querySelector("#nav-toggle");
const appNav = document.querySelector("#app-nav");
function setNavMenu(open) {
if (!appNav || !navToggle) return;
appNav.classList.toggle("is-open", open);
navToggle.setAttribute("aria-expanded", open ? "true" : "false");
}
if (navToggle && appNav) {
navToggle.addEventListener("click", () =>
setNavMenu(!appNav.classList.contains("is-open")));
// A link click navigates (or closes same-page) — shut the menu.
appNav.addEventListener("click", (e) => {
if (e.target.closest("a")) setNavMenu(false);
});
// Esc closes while open (document-level; the menu is the only
// document-level overlay this module owns).
document.addEventListener("keydown", (e) => {
if (e.key === "Escape" && appNav.classList.contains("is-open")) {
setNavMenu(false);
navToggle.focus(); // focus returns to the opener
}
});
// Resize back to desktop: the inline nav reappears — no stale open
// state (the .is-open class is scoped by the ≤640px CSS anyway, but
// dropping it keeps aria-expanded honest).
const mq = window.matchMedia("(max-width: 640px)");
const onMqChange = () => { if (!mq.matches) setNavMenu(false); };
if (mq.addEventListener) mq.addEventListener("change", onMqChange);
else mq.addListener(onMqChange); // older engines, defensive
}
```
- update the module docstring: add the hamburger bullet (phase 46, owner permission 2026-08-27, `TODO.md` L9).
2. `tests/unit/test_hamburger_nav.py` — extend (from task 01) with the JS pins: `header.js` contains the `#nav-toggle` binding, `setNavMenu` (or equivalent) syncing **both** `.is-open` and `aria-expanded`, the delegated `a`-click close, the `Escape` close (with focus return), and the `matchMedia("(max-width: 640px)")` change-close; assert the binding is null-safe (`navToggle && appNav` guard).
3. `uv run pytest` green at this checkpoint.
## Testing & Quality
- Unit: the extended pin file; full suite green.
- Coverage: **>90%** on `app/` (unchanged — frontend-only).
## Completion Criteria
- [ ] One module-owned binding: click toggles (aria-expanded in sync), Esc closes + refocuses the toggle, a link click closes, desktop resize closes; pages lacking the elements are a no-op.
- [ ] The auth visibility contract is untouched — the binding toggles the container only; `hidden` links stay hidden.
- [ ] Full suite green at this checkpoint.
@@ -0,0 +1,28 @@
# Task 03 — Hamburger E2E + regressions + commit
**Phase:** `46_mobile_hamburger_nav` · **Source:** `TODO.md:9` — "The navbar on mobile is way too squished. Make it a hamburger dropdown menu with a nice animation"
**Story:** `.agent/user_stories/mobile-hamburger-nav.md`
## Objective
Prove the full mobile contract in the browser (visibility, contents per auth state, navigation, dismissal, animation, reduced motion, desktop regression) and commit the phase.
## Work
1. `tests/e2e/test_mobile_hamburger_nav.py` (new) — mock-only, DB up. Mobile tests use a 375×812 page (new page per test, or `page.set_viewport_size` — the conftest `page` fixture is 1280×800, so create mobile pages via the `browser` fixture); per the story's Playwright Mapping Rule:
- `test_mobile_hamburger_visible_and_bar_roomy` — 375px: `#nav-toggle` visible (box ≥44px in both dimensions), `aria-expanded="false"`, the inline nav links are **not** visible in the bar (menu closed — bounding boxes outside the header band or opacity 0), and no horizontal overflow (`document.documentElement.scrollWidth <= window.innerWidth`);
- `test_anonymous_menu_contents` — anonymous at 375px: click `#nav-toggle` → `aria-expanded="true"`, exactly **one** visible link in `#app-nav` ("Chat"); `#nav-sources` / `#nav-git-sources` / `#nav-tuning` remain `hidden` inside the menu;
- `test_admin_menu_contents` — login (e2e.auth_helpers) at 375px: open → Chat / Sources / Git sources / Tuning all visible (the whoami reveal works inside the menu);
- `test_link_click_navigates_and_closes` — admin at 375px: open, click "Sources" → URL becomes `/sources.html` and on the arrival page the menu is closed (`aria-expanded="false"`, no `.is-open`);
- `test_esc_and_outside_close` — open, press `Escape` → closed **and** focus is back on `#nav-toggle`; open again, click a neutral point (e.g. the page footer/main) → closed. (If the outside-click close is not implemented per task 02's contract — it is not: only Esc/link/resize close — assert instead that the menu stays open on an outside click and **note the accepted behavior** in the test docstring; the story's AC 4 lists Esc + link + resize, not backdrop click. Do NOT add a backdrop-close — it is out of the locked scope.)
- `test_animation_and_reduced_motion` — motion allowed: computed `transition-duration` on `#app-nav` includes `0.18s` (opacity/transform pair); open → the class/aria flip; `reducedMotion: "reduce"` (new context via the `browser` fixture): computed transition is `none`/`0s` and open/close still works;
- `test_desktop_unchanged` (regression) — 1280×800: `#nav-toggle` not visible (`display: none`), the inline nav renders in the bar exactly as before (admin: all four links visible inline).
2. Regression pass (isolation runs): `test_nav_consistency.py`, `test_header_consistency.py`, `test_shared_header.py`, `test_responsive_polish.py`, `test_tuning_nav_link.py`.
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(header): hamburger dropdown nav on mobile (owner permission)`, staging this phase's files; move `.agent/phases/todo/46_mobile_hamburger_nav/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_mobile_hamburger_nav.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
## Completion Criteria
- [ ] The story E2E suite passes in isolation (all seven tests); the five regression suites pass in isolation.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,42 @@
# Phase 47 — Import quadlet + jinja files
**Source:** `TODO.md` L10–L11 — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
**Context:** A9 (LOCKED, revised 2026-08-21): default import formats `md, markdown, txt, yaml, yml, json, py`; `app/config.py::_ALLOWED_IMPORT_EXTENSIONS` bounds `BOR_IMPORT_EXTENSIONS` (narrow-only); `app/rag/chunker.py::_FORMAT_CHUNKERS` maps suffix → chunker (unknown suffix → `chunk_text` fallback). The owner-locked revision (2026-08-27, roadmap R1): ten new formats join the allowed **and** default set — the full Podman quadlet family (`container, network, volume, image, pod, kube, swap, os, endpoint`) plus `j2` — chunked as plain text.
## Objective
Quadlet unit files and Jinja templates are indexed like any other A9 format: allowed + default in config, dispatched to plain-text chunking, and provable end to end (import → catalog → viewer → retrieval).
## Dependencies
- `46_mobile_hamburger_nav` (todo) — sequential only (no shared files: this phase is `app/` + `scripts/` + tests + fixtures + docs).
- `38_local_directory_sources` / `28_git_based_sources` (complete) — the import path the new formats ride (`import_sources` / sync).
- `02_story_import_documents` (complete) — the A9 format machinery (walk, title, delta, prune) the new formats inherit unchanged.
## Tasks
1. `01_config_formats.md` — allowed + default extension sets, `.env.example`, config unit tests.
2. `02_chunker_dispatch_fixtures.md` — chunker dispatch for the ten suffixes + the fixture files.
3. `03_importer_integration.md` — importer walk/delta/prune parity + integration test.
4. `04_quadlet_e2e_and_docs_commit.md` — story E2E + README + PLAN A9 revision note + commit.
## Testing & Quality
- Unit: `test_config.py` (allowed set contains all ten; the default CSV carries them after the original seven; the env validator accepts the new names and still rejects unknown ones; `import_extension_set` dotted form); `test_chunker.py` (dispatch for **every** new suffix → `chunk_text` semantics: a quadlet TOML fixture and a jinja fixture chunk under `HARD_MAX_CHARS`, paragraph packing behaves); `test_importer.py` (a directory walk with the new files indexes them; hidden dirs + exclusions still filter).
- Integration: import over a temp tree with quadlet+j2 files → `documents` + `chunks` rows, delta re-import idempotent.
- Coverage: **>90%** on `app/` — config/chunker/importer changes fully covered.
- E2E (mandatory, A16): `tests/e2e/test_quadlet_jinja_import.py`, run in isolation.
## Completion Criteria
- [ ] A default-extensions import indexes `.container` / `.network` / `.volume` / `.image` / `.pod` / `.kube` / `.swap` / `.os` / `.endpoint` / `.j2` files (fixture-proven); no env configuration needed.
- [ ] The Sources table lists them; the viewer shows a `.container` file's TOML content with the stem as title.
- [ ] A question containing a `.j2` sentinel is non-deflected with the `.j2` doc as a source chip (A8 FTS-OR honesty gate).
- [ ] `BOR_IMPORT_EXTENSIONS` still rejects truly unknown extensions (validator intact).
- [ ] README + `.env.example` + PLAN.md A9 revision note record the extended set.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
- [ ] `uv run pytest tests/e2e/test_quadlet_jinja_import.py -v --no-cov` green in isolation (DB up).
- [ ] Regression E2E suites green in isolation: `test_import_documents.py`, `test_sync_button.py`, `test_git_sources_admin.py`.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
## Locked decisions
- **Owner-locked revision (2026-08-27, roadmap R1):** A9's format set is **revised** — ten formats added to the allowed + default set (the full quadlet family + `j2`); plain-text chunking (no TOML/Jinja-aware splitters); recorded as a PLAN.md A9 revision note with owner permission — a recorded revision, not a silent deviation (AGENTS.md rule 3).
- **A9 invariants kept:** hidden (dot) directories still skipped; the exclusion list unchanged; narrow-only `BOR_IMPORT_EXTENSIONS` validator; sha256 delta / prune unchanged; `HARD_MAX_CHARS` (1200) honored by the `chunk_text` dispatch.
- **A17 honoured** — one atomic commit.
@@ -0,0 +1,29 @@
# Task 01 — Config: the ten new formats (allowed + default)
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md:10–11` — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
## Objective
`app/config.py` allows and imports the ten new formats by default; the env validator keeps rejecting truly unknown extensions.
## Work
1. `app/config.py` —
- `_ALLOWED_IMPORT_EXTENSIONS`: add `"container", "network", "volume", "image", "pod", "kube", "swap", "os", "endpoint", "j2"` (with a comment: A9 revised 2026-08-27, owner permission — the full Podman quadlet family + Jinja templates);
- `import_extensions` default: `"md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2"` (the original seven first, the ten appended — order is cosmetic, the set is what matters; keep the field's docstring and the `mode="after"` validator **unchanged** — it already normalizes/lowercases/dedups and rejects unknowns, so it accepts the new names automatically);
- update the module-level comment on `_ALLOWED_IMPORT_EXTENSIONS` (it cites A9 revised 2026-08-21 — append the 2026-08-27 revision).
2. `.env.example` — the commented `BOR_IMPORT_EXTENSIONS` line updates to the new default CSV (it currently documents the old default).
3. `tests/unit/test_config.py` —
- the allowed set contains all seventeen formats;
- the default `import_extensions` includes the ten new names (assert each);
- `import_extension_set` returns the dotted lowercased set (`.container`, `.j2`, …);
- the validator **accepts** a `BOR_IMPORT_EXTENSIONS` containing the new names (e.g. `md,container,j2`) and **still rejects** an unknown one (e.g. `md,xyz`) — the never-widen contract with the widened base set.
4. `uv run pytest tests/unit/test_config.py -v` green; full unit suite green (the chunker/importer don't know the new suffixes yet — task 02; no test at this checkpoint imports a new-format file).
## Testing & Quality
- Unit: as above; coverage **>90%** on `app/` (config covered).
- No behavior change for existing formats (the default CSV only grows — every previously-imported file still matches).
## Completion Criteria
- [ ] Default settings import all seventeen formats; env narrowing/widening rules behave (new names allowed, unknowns rejected).
- [ ] `.env.example` documents the new default.
- [ ] Full suite green at this checkpoint.
@@ -0,0 +1,39 @@
# Task 02 — Chunker dispatch + fixture files
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md:10–11` — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
## Objective
Every new suffix dispatches to plain-text paragraph packing (`chunk_text`), and the fixture tree carries realistic quadlet + jinja files for the unit/integration/E2E layers.
## Work
1. `app/rag/chunker.py` —
- `_FORMAT_CHUNKERS`: add the ten entries (owner decision R1: plain-text packing — no TOML/Jinja-aware splitter):
```python
".container": chunk_text, ".network": chunk_text, ".volume": chunk_text,
".image": chunk_text, ".pod": chunk_text, ".kube": chunk_text,
".swap": chunk_text, ".os": chunk_text, ".endpoint": chunk_text,
".j2": chunk_text,
```
- update the `_FORMAT_CHUNKERS` comment (currently "A9, revised: md, markdown, txt, yaml, yml, json, py") and the module docstring's per-format list (add: "**container / network / volume / image / pod / kube / swap / os / endpoint / j2** (A9 revised 2026-08-27) — quadlet unit files (TOML) and Jinja templates; plain-text paragraph packing (`chunk_text`) — no format-specific splitter (owner decision).").
- `chunk_document`'s unknown-suffix fallback stays as-is (belt-and-braces).
2. **Fixture files** (under `tests/fixtures/docs/homelab/` — the tree the import E2E imports; hidden dirs are skipped, so no dot-dirs):
- `quadlet/compose.container` — realistic quadlet TOML (≥ ~1 500 chars to exercise sub-splitting past a single paragraph pack): `[Unit]` (Description/Wants), `[Service]` (Restart=always), `[Container]` (Image, Ports, Environment, Network, Volume mounts, Exec), comments. Include a unique sentinel token on its own line, e.g. `# RESE-QUADLET-SENTINEL-77aa`.
- `quadlet/lan.network` — small: `[Unit]` + `[Network]` (Driver=bridge, IPAMDriver, Subnets) + sentinel `# RESE-NETWORK-SENTINEL-11bb`.
- `quadlet/cache.volume` — small: `[Unit]` + `[Volume]` (Driver, Device) + sentinel `# RESE-VOLUME-SENTINEL-22cc`.
- `templates/deploy.j2` — a Jinja snippet with `{% for %}` / `{{ var }}` / `{# comment #}` constructs (realistic: an ansible-style service template) + sentinel `RESE-JINJA-SENTINEL-33dd` (no `#` prefix — it lives in a `{% set %}` line or a comment the importer keeps).
- keep the existing fixture files byte-identical (other E2E suites import this tree — `test_import_documents.py` asserts exact chunk counts: **run that suite's expectations check**: the tree grew by 4 files, so the phase-02 import E2E's document/chunk count assertions will change — update `tests/e2e/test_import_documents.py`'s count constants in this task, or fold that update into task 04's E2E work; whichever you choose, the full E2E regression pass in task 04 must be green. Prefer updating the constants here so task 03's integration test and task 04 share the same fixture state.)
3. `tests/unit/test_chunker.py` —
- dispatch: for **each** of the ten suffixes, `chunk_document(content, "x/<name>.<suffix>")` produces the same chunks as `chunk_text(content, …)` (parametrize over the suffix list);
- the `.container` fixture (read the file in the test, house pattern) chunks into ≥2 chunks, every chunk ≤ `HARD_MAX_CHARS`, and the sentinel token survives in some chunk;
- the `.j2` fixture chunks; Jinja braces are just text (no special handling — assert a `{{` line appears verbatim in a chunk);
- the unknown-suffix fallback is unchanged (a `.whatever` file still chunk-paragraphs).
4. `uv run pytest tests/unit/test_chunker.py -v` green; full unit suite green.
## Testing & Quality
- Unit: as above; coverage **>90%** on `app/` (chunker dispatch covered).
- No behavior change for the seven original formats (their chunker bindings are untouched).
## Completion Criteria
- [ ] Ten dispatch entries; docstrings/comments cite the A9 revision; fixtures exist with their sentinels and the ≥1 200-char container file.
- [ ] `test_import_documents.py` count constants updated (or explicitly deferred to task 04 — state the choice in the task's completion note).
@@ -0,0 +1,30 @@
# Task 03 — Importer parity: walk, delta, prune, titles
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md:10–11` — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
## Objective
Prove the new formats ride the existing import machinery unchanged: the default walk picks them up, delta detection re-imports on change, prune drops them on removal, and titles fall back to the file stem.
## Work
1. `tests/unit/test_importer.py` — extend (house style — the walk function `iter_importable_files` is pure and unit-tested against temp trees):
- a temp tree containing one file per new extension (all ten) + one unknown (`.xyz`) + one hidden-dir file (`.esphome/x.container`) + one exclusion (`node_modules/y.container`): the default-extensions walk returns exactly the ten new files — unknown/hidden/excluded filtered;
- the seven original extensions still walk (regression in the same test);
- title extraction: a `.container` file with no H1 gets the stem title (`extract_title` fallback — via the importer's title path, whatever the house test asserts titles through).
2. `tests/integration/` — a new `tests/integration/test_import_quadlet_jinja.py` (or extend `test_importer_e2e.py` if that file's harness fits better — choose and note):
- build a temp source dir with a `.container`, a `.volume`, and a `.j2` file (reuse the fixture files' content or small inline variants);
- run `import_sources([dir], fake_llm, prune=False)` with the house fake (`tests/fakes.py::FakeEmbedder` — it already implements `embed` + `chat`; give it an `embed_one` delegate if the import path calls one — check the `Embedder` protocol in `app/rag/importer.py` and satisfy exactly what it names):
- the three docs land in `documents` (source/path/title — stem titles) with non-zero `chunks` rows;
- **delta:** re-run with the `.j2` file's content changed → that doc `updated` (hash changed), the others `unchanged`;
- **prune:** delete the `.volume` file, re-run with `prune=True` → pruned count 1, the row gone, its chunks cascade-deleted.
- use the existing DB integration harness (the conftest app/db fixtures in `tests/conftest.py` — same pattern as `test_importer_e2e.py`).
3. `uv run pytest tests/unit/test_importer.py tests/integration/test_import_quadlet_jinja.py -v` green; full suite green (DB up for the integration part: `podman compose up -d db`).
## Testing & Quality
- Unit + integration as above; coverage **>90%** on `app/` (the importer is unchanged code — the new coverage comes from exercising it with the new formats; if TOTAL dips from task 01's config growth, add asserts — but no `app/` code change is expected in this task).
- No `app/` code change expected: if a gap is found (e.g. the walk already accepts any dotted suffix and the config set was the only gate), record that in the task completion note — the tests then prove the gate's location.
## Completion Criteria
- [ ] Default walk indexes the ten new formats; hidden-dir/exclusion/unknown filtering unchanged; stem titles.
- [ ] Delta + prune parity for the new formats (integration).
- [ ] Full suite green at this checkpoint.
@@ -0,0 +1,30 @@
# Task 04 — Quadlet/jinja E2E + docs (README, PLAN A9 revision) + commit
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md:10–11` — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
## Objective
Prove the story end to end (import → catalog → Sources table → viewer → FTS retrieval), record the A9 revision in the docs, run the regressions, and commit the phase.
## Work
1. `tests/e2e/test_quadlet_jinja_import.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule (the import happens out-of-band against the session mock, exactly like `test_import_documents.py`: truncate the KB, `import_sources([FIXTURES], LLMClient(settings))` in a thread — reuse that file's helpers/pattern; the task-02 fixtures are already in the tree):
- `test_quadlet_and_jinja_indexed` — after the module import, `GET /api/docs` lists `quadlet/compose.container`, `quadlet/lan.network`, `quadlet/cache.volume`, `templates/deploy.j2`, each with a non-zero chunk count and the stem as title;
- `test_sources_table_shows_them` — admin: `/sources.html` renders rows for the four files (path links present, `.doc-link`);
- `test_container_content_viewable` — open `compose.container` from the Sources table (modal, phase 26): the content area contains the `[Container]` section text and the `RESE-QUADLET-SENTINEL-77aa` sentinel; the title is the stem (`compose`);
- `test_jinja_retrievable_not_deflected` — ask a question containing `RESE-JINJA-SENTINEL-33dd` (the A8 gate: an FTS hit among the candidates keeps it honest-positive — LOW requires **zero** FTS hits): the brain bubble is **not** `.is-deflected` and a source chip names `templates/deploy.j2` (the mock's answer shape is deterministic; the assertion is on the gate + the chips, not the prose).
- module fixture: truncate `documents`/`chunks`/`query_log` per test module (the house E2E pattern) and re-import — note: this file's re-import changes the KB for the session; it is run in **isolation** (A16), so no cross-suite interference.
2. **Docs:**
- `README.md` — wherever the import format list is documented (the "Import & update" section mirrors PLAN §11 / A9), extend it with the ten new formats (the 2026-08-27 A9 revision, plain-text chunking);
- `.agent/PLAN.md` — **the only PLAN edit in this phase** (owner-locked revision R1): in the §2 anchors table, the A9 row's decision text gains the extension — append to the A9 row (keep the original wording, mark the revision in the row's notes/status or in a revision note under the table, the established style): "**A9 revision (phase 47, owner permission 2026-08-27):** the format set extends with the Podman quadlet family (`container, network, volume, image, pod, kube, swap, os, endpoint`) and `j2` (Jinja templates) — plain-text chunking (`chunk_text`), owner: `TODO.md` L10–L11. The narrow-only `BOR_IMPORT_EXTENSIONS` rule and the hidden-dir/exclusion invariants are unchanged." Update PLAN §5's chunking-policy format line and §11's workflow line to list the extended set (same note style). Touch **nothing else** in PLAN.md.
3. Regression pass (isolation runs): `test_import_documents.py` (task 02's count-constant update must hold — the tree now has four more files), `test_sync_button.py`, `test_git_sources_admin.py`.
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(import): index quadlet unit files and jinja templates (A9 revision)`, staging this phase's files **including the force-added `.agent/PLAN.md`** (AGENTS.md rule 8) and the phase dir move `.agent/phases/todo/47_quadlet_jinja_import/` → `.agent/phases/complete/`.
## Testing & Quality
- E2E: `uv run pytest tests/e2e/test_quadlet_jinja_import.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (tasks 01–03 carry it).
## Completion Criteria
- [ ] The story E2E suite passes in isolation (all four tests); the three regression suites pass in isolation.
- [ ] README + `.env.example` (task 01) + PLAN.md (A9 row + §5 + §11) record the extended set; no other PLAN change.
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
@@ -0,0 +1,103 @@
# Story: Agent makes as many tool calls as it wants
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md` L8 ·
**E2E:** `tests/e2e/test_agent_unlimited_tools.py`
## Bug report (verbatim, `TODO.md` L8)
> "Allow the LLM to make as many tool calls as it wants, remove the
> restrictions, they're causing problems getting correct answers"
## Narrative
As **the owner**, the phase-37 per-turn budgets — one `list_documents`
call and one `read_document` call — cap answers: complex questions need
several documents, and the "No reading budget left" refusal is where
correct answers die. The per-tool restrictions are being **removed**:
the model may call the tools as many times as it needs, within a single
configurable **round cap** that exists only to stop a pathological
infinite loop (and doubles as the no-tools kill switch).
- **Given** a grounded chat turn (retrieval found relevant docs)
- **When** the model calls `list_documents` / `read_document`
- **Then** every valid call is executed — re-lists included — until the
model answers or the round cap is reached, at which point a final
no-tools answer is forced.
## Acceptance criteria
1. **No per-tool budgets:** `BOR_AGENT_LIST_CALLS` /
`BOR_AGENT_READ_CALLS` are gone from `app/config.py`,
`.env.example`, and the agent loop; the `LIST_EXHAUSTED` /
`READ_EXHAUSTED` refusals no longer exist.
2. **Round cap only:** new `agent_max_rounds`
(`BOR_AGENT_MAX_ROUNDS`, default **10**) counts tool rounds; at the
cap the loop forces one final `tools=None` answer. `0` disables the
tools entirely — the request goes out with `tools=None`,
byte-identical to the pre-phase-37 path (the kill switch survives,
per the owner-locked revision).
3. **Non-budget rejections kept:** unknown tool → `"Unknown tool."`,
missing args → the `MISSING_READ_ARGS` refusal, already-in-context
document → `"Already in your context."` — none of these consume a
round's *budget* (there is none) but the round cap still bounds a
stream that keeps emitting rejected calls.
4. **Everything downstream unchanged:** the `tool` SSE event shape, the
per-turn `tool_calls=N` log field (budget-consuming → now: executed),
`done.sources` extension, and the UI tool lines are untouched.
5. **PLAN recorded:** the phase-37 locked decision
("budgets-as-kill-switch") is revised in `.agent/PLAN.md` with an
owner-permission note (2026-08-27, `TODO.md` L8), following the
established revision-note pattern (phases 16/19/24/37).
## Owner-confirmed (2026-08-27, roadmap R2)
1. **Both budget env vars removed.** New single guard
`BOR_AGENT_MAX_ROUNDS` (default **10** tool rounds, then forced final
answer); **`0` = no-tools kill switch** (pre-phase-37 behavior).
2. **Re-listing is allowed** (a second `list_documents` is a normal
executed call — it even counts in `tool_calls=N`).
## UI Visualization & Structure
- **Config** (`app/config.py`): delete `agent_list_calls` /
`agent_read_calls`; add `agent_max_rounds: int = 10` with a docstring
(0 disables the tools entirely — the loop makes exactly one request
with `tools=None`). `.env.example`: the two `BOR_AGENT_*_CALLS` lines
are replaced by `BOR_AGENT_MAX_ROUNDS=10` with an updated comment.
- **Agent loop** (`app/rag/agent.py`): `run_agent` —
`max_rounds = settings.agent_max_rounds`; `tools = AGENT_TOOLS if
max_rounds > 0 else None`; the loop drops `list_left` / `read_left`
and the budget-driven `tools = None` transition; after each executed
call `rounds += 1` and at `rounds >= max_rounds` the existing forced
final answer path runs (now the *only* exit besides "no calls").
`_execute_tool` loses its budget parameters + the two exhaustion
constants; `AGENT_TOOLS`' `read_document` description drops "exactly
one more"; module/docstrings and the probe reference updated.
`AgentHolder` unchanged (`tool_calls` still counts executed calls).
- **Tests:** `tests/unit/test_agent.py` rewritten around the round cap
(always-listing mock LLM: N rounds then forced answer; `max_rounds=0`
→ single `tools=None` request; rejected-call spam bounded by the cap);
`tests/unit/test_config.py` (default 10, env override, 0);
`tests/integration/test_chat_api.py` budget fixtures →
`agent_max_rounds`; `tests/e2e/mock_llm.py` `_tool_flow` extended: the
existing deterministic 3-step flow stays, plus a multi-read variant
triggered by a marker (e.g. the user message containing
`"read two documents"`) — stateless classification by counting
`Document …:` tool messages (list → read #1 → read #2 → answer).
- **Non-goals:** no per-call cost cap, no streaming change, no new
endpoint.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_agent_unlimited_tools.py`** (mock
LLM; DB up):
1. `test_multi_read_turn` — a grounded question carrying both
`TOOLS_TRIGGER` and the multi-read marker: the turn streams tool
lines for `list_documents` **and two** `read_document` calls, then a
final answer; the bubble is not deflected.
2. `test_done_sources_include_reads` — the final `done` (observed via
the source chips) lists the retrieved doc(s) plus **both** read
documents, deduped.
3. `test_relist_allowed` (unit-level via integration, plus E2E
observable state) — a re-listed catalog does not produce a refusal
line; the UI shows a tool line per executed call.
4. `test_single_tool_flow_regression` (phase 37) — the original
3-step flow still answers with exactly one read (runs against the
unchanged `tests/e2e/test_agent_document_tools.py` in the
regression pass, not duplicated here).
+100
View File
@@ -0,0 +1,100 @@
# Story: Markdown tables in chat (and everywhere the renderer runs)
**Phase:** `44_markdown_tables` · **Source:** `TODO.md` L6 ·
**E2E:** `tests/e2e/test_markdown_tables.py`
## Bug report (verbatim, `TODO.md` L6)
> "Certain markdown formatting isn't working - tables for example don't
> get rendered as tables in the chat response."
## Narrative
As **a user**, when Brain answers with a markdown table (services and
ports, versions, schedules — the concrete specifics the persona is built
around), I expect a real table: aligned columns, borders, readable.
Today the shared renderer (`frontend/assets/markdown.js`, ~60 lines,
no-CDN by A11) has no table support — a pipe table renders as one
paragraph of raw `|` text.
- **Given** Brain's answer (or a document / thinking block) contains a
GFM pipe table
- **When** the renderer runs
- **Then** it renders a semantic `<table>` with a `<thead>` header row
and `<tbody>` body rows, XSS-safe (escape-first, as the rest of the
renderer).
## Acceptance criteria
1. **Pipe tables render:** header row + `|---|` separator row + body
rows → `<table class="md-table">` with `<th scope="col">` header
cells; leading/trailing pipes and in-cell whitespace are handled;
cells keep their inline markdown (bold/em/code).
2. **XSS-safe:** cell content is escaped before any transform — a cell
containing `<img onerror=…>` renders inert (the escape-first
guarantee, same as all other content).
3. **Code fences win:** a `|`-heavy block inside a ``` fence is never
parsed as a table (fence protection runs first, as today).
4. **Non-tables stay put:** a single `|` in prose, a lone separator
without a header, or a 1-line "table" is left as text.
5. **Wide tables:** the table sits in an `overflow-x: auto` wrapper so a
wide table scrolls horizontally instead of breaking the 46rem chat
column (PLAN §7.1).
6. **Shared everywhere:** the same renderer serves the chat answer, the
document viewer, and the thinking block — all three render tables.
7. **Style:** `.md-table` uses the existing dark-tech palette
(PLAN §7.2 tokens, contrast ≥4.5:1); alignment colons in the
separator are parsed but all cells render left-aligned (owner
decision).
## Owner-confirmed (2026-08-27, roadmap A3)
1. **Scope = GFM pipe tables** (header + separator + body rows). Links,
blockquotes, and hr are **not** in scope for this story.
2. **Alignment colons parsed, rendered left.**
3. **Wide tables get a horizontal scroll wrapper** inside the bubble.
## UI Visualization & Structure
- **Renderer** (`frontend/assets/markdown.js`): a table-protection pass
between the existing fence pass and the escape pass — consecutive
lines forming a table (every line contains `|`; line 2 matches the
separator `^\s*\|?(\s*:?-{3,}:?\s*\|)+\s*:?-{3,}:?\s*\|?\s*$`-style
rule with ≥1 cell) are pulled out, each cell is escaped + inline-
transformed, and the block is reinserted as protected HTML (same
`\u0000CODE…\u0000` placeholder mechanism as code fences, or a
sibling placeholder — the existing restore step is the only place
placeholders are re-expanded). Output shape:
`<div class="md-table-wrap"><table class="md-table"><thead><tr>
<th scope="col">…` / `</table></div>`.
- **CSS** (`frontend/assets/styles.css`): `.md-table-wrap
{ overflow-x: auto; }` (the wrapper is the scroller — the table keeps
natural width); `.md-table` border-collapse, `th`/`td` borders from
`--line`, padding ≈0.4rem 0.6rem, `thead` tinted from the surface
palette; fits the 46rem column without a new container.
- **Mock** (`tests/e2e/mock_llm.py`): new `TABLE_TRIGGER` (a substring
like `"show me a table"`) → `compose_answer` returns a fixed
deterministic GFM table answer (checked before the default tail-echo
branch), including one deliberately wide table for the overflow
assertion.
- **Non-goals:** no new library (A11), no CDN, no change to the escape-
first architecture, no table editing in the steering/tuning UI.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_markdown_tables.py`** (mock LLM; DB
up):
1. `test_chat_table_renders` — ask a question containing `TABLE_TRIGGER`;
the brain bubble contains `<table class="md-table">` with a
`<thead>` (header cells as `<th scope="col">`) and the expected
body-cell texts; no raw `|---|` separator text in the bubble.
2. `test_wide_table_scrolls` — the mock's wide table: the bubble's
`.md-table-wrap` has `scrollWidth > clientWidth` and horizontal
wheel/scroll moves it; the 46rem column itself does not overflow the
page.
3. `test_table_xss_safe` — a table whose cell contains an HTML tag
(mock variant or document content) renders the tag as text
(no injected element).
4. `test_viewer_table_renders` (shared renderer) — an indexed fixture
document containing a pipe table opens in the document modal and
renders the same `<table class="md-table">`.
5. `test_fence_not_a_table` (regression) — a fenced code block full of
`|` pipes renders as `<pre><code>`, no `<table>`.
6. `test_plain_pipe_stays_text` (regression) — a prose answer with a
single `|` renders as text, no `<table>`.
+110
View File
@@ -0,0 +1,110 @@
# Story: Mobile hamburger nav
**Phase:** `46_mobile_hamburger_nav` · **Source:** `TODO.md` L9 ·
**E2E:** `tests/e2e/test_mobile_hamburger_nav.py`
## Bug report (verbatim, `TODO.md` L9)
> "The navbar on mobile is way too squished. Make it a hamburger
> dropdown menu with a nice animation"
## Narrative
As **a mobile user**, the 58px header currently crams up to four text
nav pills (Chat / Sources / Git sources / Tuning) next to the brand and
four icon action pills — the phase-34/35 squeeze at 360–375px leaves
0.72rem-font pills that are hard to hit and hard to read. The nav links
move into a **hamburger dropdown menu** on small screens: one
`#nav-toggle` button in the bar, and the links open as an animated
panel below the header with comfortable touch targets.
- **Given** a viewport ≤640px
- **When** I tap the hamburger
- **Then** the nav menu drops down with a short slide+fade animation and
full-size links; tapping a link navigates and closes the menu.
- **Given** a viewport >640px
- **When** the page loads
- **Then** nothing changes — the inline nav pills render exactly as
today.
## Acceptance criteria
1. **Mobile (≤640px):** the inline nav pills are hidden from the bar; a
hamburger button (`#nav-toggle`, `aria-label="Menu"`,
`aria-controls="app-nav"`, `aria-expanded`) appears — 44px touch
target, icon-only.
2. **Menu:** `.app-nav` (now `id="app-nav"`) becomes a dropdown panel
below the header — vertical full-width rows, ≥44px targets, readable
font size; the **auth visibility contract is preserved inside the
menu** (anonymous: only "Chat"; admin: Chat / Sources / Git sources /
Tuning — the same ship-hidden `hidden` attributes the whoami gate
already drives).
3. **Animation:** opening/closing animates (slide-down + fade, ≈180ms);
`prefers-reduced-motion: reduce` stills it (no transition).
4. **Behavior:** toggle flips `aria-expanded`; `Esc` closes while open;
a link click navigates **and** closes the menu; resizing back to
>640px closes it (the inline nav reappears, no stale state).
5. **Bar layout:** the action pills (Tuning toggle, Sync, New chat,
Sign in/out) stay in the bar icon-only — the phase-35 tightest
squeeze rules on `.nav-link` / `.app-nav` gaps are replaced by the
roomier menu; the bar fits 360px with the brand intact or clipped as
today.
6. **All six pages** get the identical toggle + menu (the phase-34
"same bar on every page" contract).
## Owner-confirmed (2026-08-27, roadmap A5)
1. **The hamburger contains the nav links only** (Chat / Sources / Git
sources / Tuning). The action pills stay in the bar.
2. **Animation:** slide-down + fade, 180ms; `prefers-reduced-motion`
stills it.
3. **Breakpoint:** the existing ≤640px mobile block (no new breakpoint).
## UI Visualization & Structure
- **Markup (all six pages — `index.html`, `sources.html`,
`document.html`, `git-sources.html`, `login.html`, `tuning.html`):**
a `#nav-toggle` button inserted before `<nav class="app-nav"
aria-label="Primary">` (which gains `id="app-nav"`); the nav keeps
its existing links and `hidden` attributes byte-identically.
- **CSS** (`frontend/assets/styles.css`, the `@media (max-width: 640px)`
block): `.nav-toggle { display: none }` outside, `display:
inline-flex` + 44px target inside; `.app-nav` becomes the dropdown:
absolute below the header, `flex-direction: column`, surface background
+ bottom border/shadow, full-width row links; closed state
(`visibility: hidden; opacity: 0; transform: translateY(-8px);
pointer-events: none`) → `.is-open` (`visible; opacity: 1; transform:
none`), `transition: opacity/transform 180ms ease`; a
`prefers-reduced-motion` override kills the transition. The old
nav-pill squeeze rules (`.nav-link` 0.72rem, `.app-nav` gap 0.05rem)
are superseded for the menu rows.
- **JS** (`frontend/assets/header.js`, module-import binding like
sign-out): null-safe `#nav-toggle` / `#app-nav` — click toggles
`.is-open` + `aria-expanded`; delegated click on nav links closes it;
`document` keydown `Esc` closes while open; a
`matchMedia("(max-width: 640px)")` change listener closes on
desktop. No other header.js behavior touched.
- **Non-goals:** no change to the 900px tablet rules, the action pills,
the viewer's title bar height, or the no-CDN/A11 constraints.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_mobile_hamburger_nav.py`** (mock
LLM; DB up):
1. `test_mobile_hamburger_visible_and_bar_roomy` — 375×812: `#nav-toggle`
visible with `aria-expanded="false"`; the inline nav links are not
visible in the bar (menu closed); the header has no horizontal
overflow.
2. `test_anonymous_menu_contents` — anonymous: open the menu → exactly
"Chat" is visible; open/close flips `aria-expanded`.
3. `test_admin_menu_contents` — login: open the menu → Chat / Sources /
Git sources / Tuning all visible (auth contract inside the menu).
4. `test_link_click_navigates_and_closes` — open, click "Sources"
(admin): navigates to `/sources.html`, and the menu on the arrival
page is closed.
5. `test_esc_and_backdrop_close` — open, `Esc` closes (aria-expanded
false); open again, click outside the panel closes.
6. `test_animation_and_reduced_motion` — with motion allowed, the menu
has a transition (computed `transition-duration` ≈180ms on the
opacity/transform pair); with `reducedMotion: "reduce"` emulated,
the transition is none/0s and the menu still opens/closes.
7. `test_desktop_unchanged` (regression) — 1280×800: no hamburger,
inline nav pills exactly as before (the phase-34/35 bar contract,
`test_nav_consistency` / `test_header_consistency` pass in the
regression pass).
@@ -0,0 +1,91 @@
# Story: No reply autoscroll
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md` L5 ·
**E2E:** `tests/e2e/test_no_reply_autoscroll.py`
## Bug report (verbatim, `TODO.md` L5)
> "Get rid of the chat reply autoscroll, it's breaking things like
> making it impossible for the user to scroll while a reply generates."
## Narrative
As **a user reading a long answer**, I want full control of the
viewport while Brain replies. Today the page auto-scrolls to follow the
stream (phase 18 "follow-the-bottom"): while I'm in the 200px
near-bottom band the page is yanked down on every thinking / tool /
delta frame, which fights my own scrolling mid-answer. The reply
autoscroll is being **removed** — the page only scrolls when I
explicitly cause it.
- **Given** a reply is streaming (thinking, tool calls, or answer text)
- **When** I scroll up to read earlier context
- **Then** the viewport stays exactly where I put it for the rest of the
turn — no frame yanks it back.
## Acceptance criteria
1. **No streaming autoscroll:** during a long thinking stream, a tool
call, and a long answer, the page never auto-scrolls — sampled
`window.scrollY` is stable (within 1px) across frames while the
viewport is away from the bottom.
2. **Submit reveals my message:** sending a question while scrolled up
still scrolls the viewport down so my own message is visible
(user-initiated — kept by owner decision).
3. **Restore landing kept:** reloading a persisted conversation
(phase 14) still lands one-shot on the latest message.
4. **The phase-18 gate is gone:** `NEAR_BOTTOM_PX` /
`isNearBottom()` and the per-frame `scrollReveal` calls in the
thinking / tool / delta handlers are removed from `app.js`; the unit
pin (`tests/unit/test_frontend_scroll.py`) is rewritten for the new
contract (scrolls happen only on submit + restore landing).
5. **Everything else unchanged:** the thinking window's *internal*
bottom-pin (phase 17 — `textEl.scrollTop`, not the page) is untouched
in this phase (phase 43 reworks it separately); message rendering,
persistence, UI states, and the 120 s guard are unchanged.
## Owner-confirmed (2026-08-27, roadmap A1)
1. **"Reply autoscroll" = the phase-18 follow-the-bottom auto-follow on
thinking / tool / delta frames.** Removed.
2. **Kept:** scroll-on-submit (reveal the user's own message) and the
one-shot restore landing on page load.
## UI Visualization & Structure
- **The functional change is in `frontend/assets/app.js` only:**
- delete `export const NEAR_BOTTOM_PX = 200`, `isNearBottom()`, and
the `force`-optional gating in `scrollReveal` — the helper becomes an
unconditional `scrollIntoView` (still smooth, still still under
`prefers-reduced-motion` via the existing `SCROLL` constant);
- `addMessage(...)` gains an explicit "scroll" intent: the **user
submit** path scrolls (shows my message), brain bubble creation and
the typing indicator do **not**;
- the thinking / tool / delta handlers drop their `scrollReveal(wrap)`
calls (the thinking handler keeps its `textEl.scrollTop` window-pin —
phase 17, reworked in phase 43);
- the phase-14 restore landing keeps its one-shot forced scroll;
- module docstrings updated (the phase-18 contract block is replaced
by the new "no reply autoscroll (owner direction 2026-08-27)"
contract).
- **Non-goals:** no new UI element, no "↓ new content" pill (the owner
wants silence, not a substitute affordance), no change to the
composer / safe-area layout.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_no_reply_autoscroll.py`** (mock LLM;
DB up; the phase-18 suite `tests/e2e/test_follow_bottom_scroll.py` is
**deleted** in this phase — its behavior is intentionally removed):
1. `test_no_autoscroll_during_long_answer` — `LONG_ANSWER_TRIGGER`
question; once the answer starts, scroll the viewport up ~2× the
answer height; sample `window.scrollY` across ≥10 streaming frames:
stable within 1px; after `done` the viewport is still where it was.
2. `test_no_autoscroll_during_thinking` — `THINKING_TRIGGER` question;
scroll up during the ~4.5 s thinking stream; the viewport stays
pinned (no per-chunk page follow).
3. `test_submit_reveals_user_message` — scroll to the very top of a
populated conversation, send a question; the viewport ends with the
user's message visible (bottom in view).
4. `test_restore_landing_one_shot` (phase 14 regression) — settle a
conversation, reload; the page lands on the latest message one-shot
and stays there while no stream is active.
5. `test_answer_content_intact` (regression) — the long answer streams
to completion with sources and (for thinking) the collapsed block,
persisted and restorable.
@@ -0,0 +1,98 @@
# Story: Import quadlet + jinja files
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md` L10–L11 ·
**E2E:** `tests/e2e/test_quadlet_jinja_import.py`
## Bug reports (verbatim, `TODO.md` L10–L11)
> "Add \".container\", \".network\", \".volume\" and other quadlet files
> to the list of allowed/parsed files"
> "Add \".j2\" jinja files to the list of allowed/parsed files"
## Narrative
As **the owner**, my homelab notes increasingly live in Podman quadlet
unit files (`.container`, `.network`, `.volume`, …) and Jinja templates
(`.j2`). Neither is in the A9 import format list, so the KB is blind to
exactly the config files I ask questions about. Both families join the
allowed + default import formats and are parsed (chunked) by the
importer.
- **Given** a source directory containing quadlet and/or `.j2` files
- **When** the importer (CLI or sync) runs
- **Then** those files are indexed (chunked, embedded, upserted) like any
other A9-format file, and their content is retrievable.
## Acceptance criteria
1. **Allowed set:** `_ALLOWED_IMPORT_EXTENSIONS` in `app/config.py`
gains `container, network, volume, image, pod, kube, swap, os,
endpoint` (the full Podman quadlet family) and `j2`; a
`BOR_IMPORT_EXTENSIONS` env value may name any of them (the
never-widen validator keeps rejecting truly unknown extensions).
2. **Default set:** the default `import_extensions` CSV includes all ten
new formats after the existing seven — a default import now picks
them up with no env configuration.
3. **Parsing:** `chunker.py` dispatches every new suffix to
plain-text paragraph packing (`chunk_text`) — quadlet files are TOML
unit files and `.j2` files are templates; no format-specific
splitter (owner decision). Every chunk still honors
`HARD_MAX_CHARS` (1200).
4. **Title:** no H1 → `extract_title` falls back to the file stem, as
with other non-markdown formats (no change needed, verified).
5. **Behavior parity:** hidden (dot) directories are still skipped, the
exclusion list is unchanged, sha256 delta detection / prune work
unchanged for the new formats.
6. **Docs:** `.env.example`'s `BOR_IMPORT_EXTENSIONS` comment, the
README's format list, and an **A9 revision note** in
`.agent/PLAN.md` (owner permission 2026-08-27, `TODO.md` L10–L11)
record the extended set.
## Owner-confirmed (2026-08-27, roadmap R1)
1. **Full quadlet family:** `container, network, volume, image, pod,
kube, swap, os, endpoint` — plus `j2`.
2. **Chunked as plain text** — no TOML/Jinja-aware splitting.
3. **A9 is revised** with a PLAN.md revision note (the established
owner-permission pattern).
## UI Visualization & Structure
- **Config** (`app/config.py`): extend the `_ALLOWED_IMPORT_EXTENSIONS`
frozenset + the `import_extensions` default string (comment cites the
A9 revision 2026-08-27). No validator change — it already
normalizes/dedups and rejects unknowns.
- **Chunker** (`app/rag/chunker.py`): ten new `_FORMAT_CHUNKERS` entries
→ `chunk_text`; module docstring's format list updated.
- **Fixtures:** `tests/fixtures/docs/homelab/quadlet/compose.container`
(realistic quadlet TOML: `[Unit]` / `[Service]` / `[Container]`
sections, one unique sentinel token, >1200 chars to exercise
sub-splitting), `…/quadlet/lan.network`, `…/quadlet/cache.volume`,
and `tests/fixtures/docs/homelab/templates/deploy.j2` (Jinja snippet
with `{{ … }}` / `{% … %}` tags + its own sentinel).
- **Tests:** unit (`test_config.py` allowed-set/default/validator;
`test_chunker.py` dispatch for every new suffix + fixture chunking;
`test_importer.py` directory walk picks the new files up);
integration (import_sources over a temp dir with quadlet+j2 files →
documents + chunks rows).
- **Non-goals:** no new DB column, no format badge change (the viewer
shows the extension it already shows), no summary-model changes.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_quadlet_jinja_import.py`** (mock
LLM; DB up):
1. `test_quadlet_and_jinja_indexed` — truncate + import the fixture
tree (the `test_import_documents.py` pattern): `GET /api/docs` lists
the `.container` / `.network` / `.volume` / `.j2` files with
non-zero chunk counts.
2. `test_sources_table_shows_them` (admin) — the Sources table renders
rows for the new files; their path links open the document modal.
3. `test_container_content_viewable` — the viewer modal shows the
`.container` file's TOML content (sentinel token present) with its
stem as the title.
4. `test_jinja_retrievable_not_deflected` — ask a question containing
the `.j2` file's sentinel word: the FTS hit keeps the honesty gate
honest-positive (A8) — the answer bubble is **not**
`.is-deflected` and the source chip names the `.j2` document.
5. `test_default_walk_includes_new_formats` (regression, unit-backed) —
a default-extensions walk over a temp tree with all ten new
extensions indexes every file; hidden directories + the exclusion
list still filter (covered by `test_importer.py` in the regression
pass).
@@ -0,0 +1,97 @@
# Story: Sync fails fast + modal when a model is down
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md` L4 ·
**E2E:** `tests/e2e/test_sync_model_down.py`
## Bug report (verbatim, `TODO.md` L4)
> "If the embedding or lite model is not accessible the sync button
> should fail fast and there should be a modal error popup explaining
> that the model isn't available."
## Narrative
As **the admin**, when I press "Sync sources" with the aipi models
(`embed` or `lite`) unreachable, I don't want to wait through git clones
and a partial import to discover the KB can't be updated — and a tooltip
on a button is not a readable error. The sync should **fail fast**
(before any expensive work) with a clear "the model isn't available"
message, shown in a **modal dialog** I can read and dismiss.
- **Given** the `embed` or `lite` model endpoint is unreachable
- **When** I press "Sync sources"
- **Then** the run fails within a couple of seconds (before any git
clone), the button settles retry-ready, and a modal dialog explains
which model isn't available.
## Acceptance criteria
1. **Server fail-fast:** `POST /api/sync` with a dead LLM endpoint
reaches `state: "failed"` with a message naming the unavailable model
(embedding first, then summary/lite) **without** cloning any source —
the probe (one small embedding + one tiny completion against
`BOR_LLM_SUMMARY_MODEL`) runs before source resolution and before any
`clone_or_pull`.
2. **Modal:** on a failed sync the page shows a modal error dialog
(`role="alertdialog"`, `aria-modal="true"`) with a title, the
sanitized error text (rendered via `textContent` — XSS-safe), and a
close control; it closes on the close button, `Esc`, or backdrop
click; focus moves into the dialog on open and returns to `#sync-btn`
on close.
3. **Every page:** the modal is built by the shared header module
(`frontend/assets/header.js`), which owns the sync state machine — so
it appears wherever `#sync-btn` exists (all six pages from phase 34).
4. **Existing surfaces kept:** the button's failed-state `title` /
`aria-label` / `.is-error` affordance and the Sources page's
`#sync-error-banner` (via `bor:sync-status`) are unchanged — the modal
is the primary, readable surface.
5. **Success path unchanged:** a healthy model still runs clone → import
→ overview exactly as phase 32/35/38 define it (regression).
## Owner-confirmed (2026-08-27, roadmap A4)
1. **The probe runs before git clones** — the fastest possible failure;
it costs one small embedding request and one ~1-token completion.
2. **The modal is the primary failure surface on every page;** the
button-title affordance and the Sources banner stay as secondary
surfaces.
## UI Visualization & Structure
- **Server** (`app/rag/llm.py`, `app/api/sync.py`): a new
`ModelUnavailableError` (subclass of `LLMError`) + an `async
check_models(llm)` probe: `embed_one("sync model check")` then a tiny
`chat([...])` against the summary model; each failure mode maps to a
message naming the model and that it isn't available (the sync
sanitizer's credential masking still applies downstream). `_run_sync`
calls it first, after `LLMClient()` construction — before
`effective_sources`, before any clone.
- **UI** (`frontend/assets/header.js`, `frontend/assets/styles.css`):
`applySyncFailure(status)` additionally opens `showSyncModal(status)`:
a lazily-created backdrop + `role="alertdialog"` panel appended to
`<body>` (so no page markup changes), error text via `textContent`,
close button + `Esc` + backdrop-click dismissal, focus management as
in AC 2. Styled with the existing dark-theme error palette
(PLAN §7.2: `#fca5a5` on `#2d1318` class, error border), `:focus-visible`
per the global rule, no motion under `prefers-reduced-motion`.
- **Non-goals:** no new endpoint, no retry-from-modal button (the button
itself is retry-ready), no change to the 2 s poll lifecycle.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_sync_model_down.py`** (mock LLM; DB
up). The suite boots its **own module-scoped app** on a distinct port
(conftest pattern used by `test_sync_button.py`) with
`BOR_LLM_BASE_URL=http://127.0.0.1:9/v1` (dead port — connection
refused) and a local `file://` fixture repo as the configured source, so
a (regressed, non-fail-fast) run would spend time cloning before failing:
1. `test_model_down_fails_fast_with_modal` — admin login, click
`#sync-btn`; within a short wall-clock budget (≤ ~10 s, vs the 60 s
generous budget of the healthy-run suite) the button settles
retry-ready **and** the modal is visible with an error naming the
model; assert the dialog role/aria contract.
2. `test_modal_dismissal` — close via button, `Esc`, and backdrop click
(one fresh failure per path); focus returns to `#sync-btn` each time.
3. `test_sync_error_surfaces_unaffected` (phase 32 regression) — after
the failure the button keeps its `title` / `.is-error` affordance; on
`/sources.html` the `#sync-error-banner` still renders off
`bor:sync-status`.
4. `test_healthy_sync_still_succeeds` (phase 32/35 regression) — the
session mock-backed app (or a second healthy module app) still runs
the full clone → import → overview pipeline to "Synced HH:MM".
@@ -0,0 +1,98 @@
# Story: Thinking scroll back (user scroll + generate-time autoscroll)
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md` L7 ·
**E2E:** `tests/e2e/test_thinking_scroll.py`
## Bug report (verbatim, `TODO.md` L7)
> "Add scrolling back to the thinking block, but have it autoscroll
> while thinking content is generating."
## Narrative
As **a user watching Brain reason**, the Thinking block should work like
a well-behaved live console: it **follows the tail while the reasoning
is generating** — but the moment I scroll up to re-read an earlier line,
it must **stop yanking me down**, and it must let me scroll the window
freely (phase 21's no-scroll clip is being reversed by owner direction).
- **Given** the Thinking block is streaming reasoning content
- **When** I'm at the bottom of the 320px window
- **Then** each new chunk keeps the window pinned to the live tail.
- **When** I scroll up to read earlier reasoning
- **Then** the window stays where I put it (no more re-pinning);
- **When** I return to the bottom
- **Then** tail-following resumes on the next chunk.
## Acceptance criteria
1. **User scroll restored:** computed `overflow-y` of
`.thinking-text` is `auto`; wheel / mouse-drag / keyboard move the
window (phase 21's `overflow-y: hidden` is gone).
2. **Follow while generating:** with the user pinned at the window's
bottom (within a small near-bottom band — the phase-18 pattern,
now applied to the *window* instead of the page, exported constant
`THINKING_NEAR_BOTTOM_PX = 32`), each streamed chunk re-pins the
window to the tail (within 1px).
3. **Paused on scroll-up:** scrolled up, the window stops being re-pinned
— `scrollTop` stays stable across subsequent chunks (within 1px).
4. **Resumes on return:** scrolling back to the bottom (within the band)
resumes tail-following on the next chunk.
5. **Kept from phases 17/21:** the fixed 320px `max-height` window, the
auto-collapse on the first answer token, the reduced-motion stillness,
and the answer-bubble scroll behavior (phase 11) are all unchanged.
## Owner-confirmed (2026-08-27, roadmap A2)
1. **The 320px window stays** — only the overflow mode and the pinning
logic change.
2. **Follow-the-bottom for the window:** autoscroll only while the user
is pinned near the window's bottom (≈32px band); scroll-up pauses,
return-to-bottom resumes. (The phase-18 page-level band is removed in
phase 42; this is its window-level successor.)
## UI Visualization & Structure
- **CSS** (`frontend/assets/styles.css`): `details.thinking
.thinking-text` — `overflow-y: hidden` → `overflow-y: auto`; the
owner-choice comment is replaced with the 2026-08-27 direction
(user-scrollable window; JS follows the tail only while pinned).
`max-height: 320px` and all other declarations untouched.
- **JS** (`frontend/assets/app.js`, the `thinking` SSE handler):
- new exported `const THINKING_NEAR_BOTTOM_PX = 32` +
`isThinkingNearBottom(textEl)` (`scrollHeight - scrollTop -
clientHeight <= band`);
- the phase-17 unconditional pin
(`textEl.scrollTop = textEl.scrollHeight`) becomes gated:
`if (block.open && isThinkingNearBottom(textEl)) { textEl.scrollTop
= textEl.scrollHeight; }` — a scrolled-up user is never re-pinned,
and returning to the bottom re-arms the pin automatically (the check
runs on every chunk).
- **Non-goals:** no "↓ more" affordance, no auto-height growth, no change
to the summary/chevron, the tool-call lines, or the answer bubble.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_thinking_scroll.py`** (mock LLM; DB
up; phase 21 lengthened `mock_llm.compose_thinking` to ~2 700 chars ≈
4.5 s of paced frames so the scratchpad overflows the 320px window by
~2×, and the phase-20 hesitation trigger gives a deterministic 4 s
frozen-tail state with the block open). The phase-21 suite
`tests/e2e/test_thinking_no_scroll.py` is **deleted** in this phase:
1. `test_thinking_window_user_scrollable` — frozen live tail (4 s
hesitation): focus `.thinking-text`, wheel up, `Home`, mouse-drag up
— `scrollTop` moves; the window shows earlier content.
2. `test_thinking_window_follows_while_pinned` — during the live stream:
with the user at the bottom, after the 2nd-to-last and the last chunk
the window is pinned to the tail (within 1px) and the last chunk's
text renders inside the visible rectangle.
3. `test_thinking_window_stops_on_scroll_up` — mid-stream: scroll up
~half the window; over the next ≥5 chunks `scrollTop` stays stable
(within 1px) — no re-pin.
4. `test_thinking_window_resumes_on_return` — from the paused state,
scroll the window back to its bottom; on the next chunk the window is
re-pinned to the tail (within 1px).
5. `test_thinking_window_css_contract` — computed `overflow-y: auto`,
`max-height: 320px`, and the clip is real (`scrollHeight >
clientHeight` for the long scratchpad).
6. `test_answer_bubble_still_scrollable` (phase 11 regression) — a long
answer: the page scrolls, the bubble's overflow is untouched.
7. `test_restored_collapsed_thinking_unaffected` (phase 17 regression) —
a settled thinking turn reloads as a collapsed block with its full
text.
@@ -0,0 +1,87 @@
# Story: Tuning toggle anonymous flash
**Phase:** `40_tuning_toggle_flash` · **Source:** `TODO.md` L3 ·
**E2E:** `tests/e2e/test_tuning_toggle_flash.py`
## Bug report (verbatim, `TODO.md` L3)
> "Loading the page briefly shows the 'Tuning' button in the header even
> when the user isn't authenticated. Only show that if the user is
> authenticated."
## Narrative
As **an anonymous visitor**, the header must never show admin-only
controls — not even for a frame. Today the tuning-notes toggle
(`#steering-toggle`, the header button labeled **"Tuning"**) ships
*visible* in all six pages' markup and is only removed from the DOM after
`/api/whoami` resolves — so every anonymous page load flashes the button
for the length of the whoami round-trip. The admin-only *nav links*
(`#nav-sources`, `#nav-git-sources`, `#nav-tuning`) already ship `hidden`
(phase-19 "absent, not hidden" contract) and are not the issue.
- **Given** an anonymous visitor loads any page
- **When** the page renders (before `/api/whoami` resolves)
- **Then** no "Tuning" control is ever visible — not for a single frame.
- **Given** a signed-in admin loads any page
- **When** whoami resolves
- **Then** the toggle is revealed (and the note list refreshes, as today).
## Acceptance criteria
1. Anonymous load of **every** page: `#steering-toggle` is never
attached-visible — a MutationObserver installed via `addInitScript`
records zero visible frames of the toggle from first paint to settled
state; after load the toggle is absent from the DOM (the existing
remove-from-DOM behavior).
2. Admin load: the toggle is visible after whoami, `aria-expanded`
works, the count badge refreshes — identical to today's admin
behavior (phase 15/34 contract).
3. No-JS visitors: the toggle is hidden (the control is JS-gated by
design — whoami decides).
4. No regression to the shared-header contract (phase 19/34): nav links
ship hidden, sign-in/sign-out pair, sync button, new-chat binding
unchanged.
## Owner-confirmed (2026-08-27, roadmap A — confirmed with the
conversion interview)
1. **The flashing control is the steering toggle**, not the admin
"Tuning" nav link (which already ships `hidden`) — confirmed by code
inspection: `#steering-toggle` ships visible in all six pages
(`index.html`, `sources.html`, `document.html`, `git-sources.html`,
`login.html`, `tuning.html`) and is removed post-whoami.
2. **Fix = ship `hidden`, reveal for admin, keep anonymous removal** —
the same ship-hidden / reveal-for-admin contract the admin-only nav
links already use; anonymous still gets "absent, not hidden".
## UI Visualization & Structure
- **The whole functional change is one attribute + one JS line:**
- `#steering-toggle` gains `hidden` in all six pages' header markup
(the button element only — the `#steering-panel` region already ships
`hidden`).
- `frontend/assets/header.js` `initSharedHeader()`: in the admin branch,
unhide the toggle (`steeringToggle.hidden = false`) before
`refreshSteering()`; the anonymous branch (`steeringToggle?.remove()`)
is unchanged.
- **Non-goals:** no change to the nav links, the panel, the steering
API, or any other shared-header control.
## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_tuning_toggle_flash.py`** (mock LLM;
DB up):
1. `test_anonymous_never_sees_toggle` — `addInitScript` a MutationObserver
that records every frame in which `#steering-toggle` exists in the
DOM and is not `[hidden]`; load `/` anonymously; after load assert the
observer recorded **zero** such frames, and the toggle is absent from
the DOM (removed, per the phase-16 contract).
2. `test_anonymous_other_pages_never_flash` — same observer assertion on
`/sources.html`, `/tuning.html`, `/login.html` (the page set the
contract must hold on).
3. `test_admin_toggle_revealed_and_working` — login via
`e2e.auth_helpers.login`; reload `/`; the toggle is visible
(`hidden` removed), clicking opens `#steering-panel`
(`aria-expanded="true"`), and the count badge matches the panel.
4. `test_nav_contract_regression` (phase 19/34) — anonymous: nav links
`#nav-sources` / `#nav-git-sources` / `#nav-tuning` stay hidden and
absent from the visible header; admin: they are revealed — the
ship-hidden contract this phase relies on is intact.