All verification complete. Final report:
**Phase 85 final verification pass — all green** (tasks 01–03 already complete; independently re-verified from scratch)
- Verified fix in tree: `styles.css` `.auth-gate` z-index 500→15 + rewritten stacking comment (no `z-index: 500` left); `index.html`/`document.html` comment-only; `token-gate.js` docstring-only (logic byte-identical); `test_mobile_hamburger_nav.py` real-click conversion + new `test_anonymous_toggle_tappable_with_gate_up`; new `tests/unit/test_gate_header_stacking.py` (3 pins); `test_api_tokens.py` untouched
- `uv run pytest` → 1717 passed, 1 warning (exit 0)
- `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%)
- E2E in isolation: `test_mobile_hamburger_nav.py` **8 passed**; `test_api_tokens.py` **9 passed** (unchanged); `test_smoke.py` **3 passed**; `test_document_viewer.py` **7 passed**
- `uv run ruff check . && uv run pyright` → clean / 0 errors
- Live probe (375×812, anonymous, fresh server): on `/` and `/history.html` `elementFromPoint` at toggle → SVG `path`, never `#auth-gate`; real click opens menu (`aria-expanded=true`); exactly one visible nav link (Chat) + sign-in copy; Chat link topmost over gate; `#main` inert — criteria 1, 2, 3 confirmed directly
- Criteria: (1) real-click menu on / + /history ✅ (2) dropdown above gate ✅ (3) anonymous contents + #main inert ✅ (4) admin byte-identical (phase-46 tests green) ✅ (5) doc-viewer gate under bar (CSS pin + doc-viewer suite) ✅ (6) full gate green ✅ (7) diff scoped to the 5 code files + new unit test + phase files, nothing in `app/` ✅ (8) commit + todo→complete move left to harness per executor rules (task files already in `complete/`)
- Note: `.agents/remediation_plan.md` (untracked) is a pre-existing artifact of the earlier security audit — left untouched; a missing "N passed" line seen mid-pass was my own `-q`+addopts `-q` = `-qq` quirk, not a defect
- Next pending phase: `86_history_page_width`
TODO.md L3-L5 converted to executable phases (protocol B - append):
- 85_mobile_menu_gate_overlap (L3): the phase-79 token gate (z 500, fixed
full-viewport) sits above the sticky header (z 20), so an
unauthenticated visitor's tap on the mobile hamburger hits the gate
overlay and the menu is unreachable until login. Reproduced:
elementFromPoint at the toggle resolves to #auth-gate on every shell
view; real clicks are intercepted. Fix: gate at z 15 (below the
header + its mobile dropdown), #main stays inert-locked; the
phase-46 E2E's programmatic-click workaround becomes a real click +
a new TODO-regression pin.
- 86_history_page_width (L4): at 375px the History page panned ~250px
into a blank region (document scrollWidth 626) although the table
scrolled fine inside its card. Root cause: the .visually-hidden
Actions header span is position:absolute with no positioned ancestor,
so its 1px box (at the 640px table's right edge) leaks into the
document's scrollable overflow. Fix: position:relative on the shared
.table-wrap card (the identical Tokens-view defect, measured 618, is
fixed by the same rule; RAG is clean and pinned). New E2E story.
- 87_big_read_progress (L5): after a tool read the UI sat on a static
'Reading <path>' line while the model prefilled the big context -
the turn looked frozen. Frontend-only: a ticking '(Ns)' suffix on
the latest tool line after 5s of frame silence (settle on the next
frame, live-only - restored lines stay timer-free) + the existing
10s aria-only typing clock promoted to a visible 'Ns' hint. New
source-level unit pins + slow-proxy E2E story (deterministic >=6s
gaps via the mock tool flow).
TODO.md cleared (items now live in .agents/phases/todo/). The pre-existing
uncommitted 81-84 phases + remediation_plan.md are a separate workstream
and are NOT part of this commit.
At <=640px the RAG "Sync sources" pill and the History "Refresh"
pill squeezed down to tiny icon-only buttons — hard to discover and
tap on a phone. They are now full-width labeled pills:
- the RAG page-head row wraps so the Sync pill drops below the
"Knowledge base" title at full width; the History page-head
already wrapped the pill below its title block
- the Sync label's min(16rem, 40vw) cap lifts on mobile (min-width: 0
engages the ellipsis) so the live-file text truncates against the
full width instead of the 40vw cap
- the Refresh glyph joins its visible label (it stays hidden on
desktop, where the label carries the pill)
This matches the established mobile full-width pill language (New
chat / Share / stale-ban Regenerate). The three unit tests that
pinned the old icon-only CSS are updated to pin the new behavior.
Single consolidated commit for four completed, validated phases (77, 78,
79, 80). The pipeline run left all work uncommitted because the harness
commits only with PHASE_COMMIT=1 while child executors are forbidden from
committing; the phases themselves all passed validation and moved to
.agents/phases/complete/.
Phase 77 — navbar view refresh
- router.js dispatches bor:view-refresh on re-show / active re-click /
popstate (gated on wasMounted; first show and boot exempt)
- History / RAG / Sources / Tuning re-fetch on refresh (admin branch);
Chat deliberately excluded (stream survival)
- History "Refresh" button (admin-only, in-flight disable + status line)
- New story suite tests/e2e/test_navbar_refresh.py (7 tests)
Phase 78 — static background
- Removed the animated glow layers; static 44px grid over the flat --bg
canvas; default and reduced-motion renders byte-identical
- Updated background/theme E2E suites; removed bg-glow test pins
Phase 79 — API tokens
- api_tokens model + migration 0012; hash-only token service
- Admin tokens API + Tokens admin view; POST /api/token-auth;
live-revoking require_user on chat / suggestions / document content
- Frontend token gate with localStorage cache; anonymous E2E suites
migrated to token login
- New story suite tests/e2e/test_api_tokens.py (9 tests)
Phase 80 — history suggestion chips
- last_questions() endpoint with SEED fallback; startNewChat() refetch
- Seed-semantics docs (config.py, .env.example, README)
- Integration state matrix + E2E suite rewritten to the 4 chip states
Also included: phase-76 report artifacts and the repo restore-test-db
skill (previously untracked), scripts/* ruff fixes from phase 77.
Final gate state (phase 80 final pass, covers everything above):
- uv run pytest --cov=app → 1637 passed, 0 failed, app/ coverage 99%
- uv run ruff check . && uv run pyright → clean, 0 errors
- Per-phase story E2E suites green in isolation
Protocol B append: navbar refresh + History refresh button (77, TODO L3),
static background — glow layers removed (78, TODO L4), admin-issued API
tokens with the in-app gate + browser caching, only shared chats stay
anonymous (79, TODO L5), onboarding chips as the last 3 questions asked
with the env seed only before the first (80, TODO L6).
TODO.md cleared — its items now live in .agents/phases/todo/.
Owner-confirmed assumptions recorded in each phase overview
(A1–A7, chat 2026-09-06).
New files:
- scripts/model_benchmark.py — shared CSV recorder for all model tests
- scripts/test_summary_model.py — summary model quality benchmark (coherence, coverage, brevity, hallucination)
- scripts/test_embed_model.py — embedding model benchmark (dimension, cosine accuracy, speed)
- .agents/skills/test-summary-model/SKILL.md — skill for testing summary models
- .agents/skills/test-embed-model/SKILL.md — skill for testing embedding models
- benchmarks/README.md — schema documentation
Updated:
- .agents/skills/test-chat-model/SKILL.md — now also records to CSV
All three scripts write to benchmarks/model_benchmarks.csv with one row
per run per check. The CSV accumulates results across runs for comparison.
2026-09-06 fixture runs: contract 100 %, executed 100 %, wall ~113 s (2 runs). Derived battery: FAIL only on usage floor (5/10 tool-turns) — answers seeded questions from context, which is ideal grounded behavior. Wall time ~2.8× lite (113 s vs 40 s). Model is clean.
2026-09-06 fixture runs: contract 92–93 %, executed 64–75 %, wall ~40.5 s (2 runs). Derived battery: FAIL, 36 % executed (38.3 s). Same pattern — copy-invariant re-read habit blocks the ≥90 % executed bar under current ALREADY_IN_CONTEXT refusal semantics. Model is working correctly; the bottleneck is the app's dedupe refusal, not the model.
Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).
Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.
Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
five `<section class="view">` blocks; hidden views carry BOTH
`hidden` and `inert` (WCAG — no focus/keyboard traversal). The
shared header, the single `doc-modal-*` skeleton, and the
`#app-version` footer each exist exactly once; the per-view copies
from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
bundler, No-CDN rule intact): lazy-imports a view module on FIRST
show only (mount-once, hide-forever — the chat view's in-flight SSE
reader persists across switches; that persistence IS the fix);
intercepts same-shell navbar links with preventDefault +
history.pushState (never a document load); handles popstate; single
writer of `.nav-link` active state (is-active + aria-current),
document.title, and the per-view meta description (values carried
over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
root-scoped queries; `initSharedHeader()` dropped (the header boots
once in the shell via the chat module; the admin flag comes from the
same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
/tuning.html, /sources.html, /git-sources.html, /history.html —
registered AFTER the API routers and BEFORE the static catch-all
(routes-first). The phase-33 caching middleware applies no-cache +
`?v=` rewriting unchanged; app/core/caching.py needed NO change
(the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
deep links to the old URLs keep working (the router picks the view
from the pathname); `/?chat=<id>` is unaffected; the Containerfile
bundles router.js (inlining the lazy view modules) and drops the
folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
keeps long saved answers in the chat, and the old cap (stricter than
the 24_000-char total history budget) 422-rejected any second turn
in such a chat (found by the phase-42 E2E suite on the shell).
Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).
Tests:
- Phase-20 suite REWRITTEN to the new semantics
(tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
cancels — the stream survives the switch and the FULL answer
settles; the pagehide partial persist REMAINS for real departures
(the partial's exact shape — first streamed chunk prefix, no done
metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
LLM): the owner repro (send -> RAG mid-stream -> Chat: window
sentinel survives = same document, FULL answer, exactly one brain
turn in bor.chat.v1, exactly one settled query_log row, auto-saved
row matches) + the same mid-stream switch against the other three
views + the real-departure-still-cancels control + the no-switch
baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
invariants (click interceptor targets ONLY same-shell view paths,
pushState-only switches, mount-once guard, hidden+inert pair,
single-writer active state/title); shell-route integration tests
(each folded path serves the shell with no-cache + `?v=` body; a
non-view path still 404s); the file-reading unit pins re-pointed at
the shell (the four view files are gone — the shell is the source
of truth).
Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.
Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.
Phase 75 (TODO.md L4): "Save as doc" now drafts a document from the
ENTIRE chat session — every question and answer up to the click, in
order — instead of only the clicked bubble's answer; the existing
doc-edit screen's free-form body editing is how the user edits out
anything they don't want to keep from previous replies (no new UI
surface).
Task 01 (frontend):
- app.js buildSessionTranscript(): walks the bor.chat.v1 conversation
record in order — a numbered section per user turn ("## N.
<question, raw>" + blank line + the raw answer text; more answers
join under the same heading), sections blank-line separated, all
trailing whitespace collapsed to one final newline. Only the raw
persisted text travels (m.who + m.text — no thinking blocks, no
source chips, no tune metadata); a brain record before the first
user record is skipped; a heading-only section marks a user turn
whose answer never landed (A6, owner-confirmed 2026-09-08).
- saveAsDoc(btn): the draft body is buildSessionTranscript(); the
dead single-bubble markdown parameter is dropped (the button's
appendSaveAsDocButton signature is unchanged — one button per
bubble). Title/path/double-click guard/hand-off are unchanged
(defaultDocTitle: the last question, whitespace-collapsed,
<=120 chars; docs/<slug>.md).
- Unit: the app.js source pins move to the transcript shape (whole
session, no thinking, no dead parameter).
Task 02 (E2E):
- tests/e2e/test_save_doc_session.py (bare-repo fixture, the
phase-59 convention — git as source of truth): three DISTINCT
on-topic turns in one session (turn 1 carries the phase-17
"think out loud" trigger so its record has a thinking block the
transcript must exclude) -> save on the LAST bubble -> the
prefilled body is ## 1./## 2./## 3. in order, byte-exact against
the deterministic mock, thinking-free -> edit the whole
section-2 block out of the body -> push -> git show
bor-docs:<path> equals the EDITED body byte-for-byte (section 2's
question and answer provably absent; sections 1 and 3 byte-exact;
the UI's sha prefix is git rev-parse bor-docs). Second test:
the button on the FIRST bubble still drafts the whole session
(A6 — the transcript is the session at click time, title stays
the last question); canceling leaves the branch tip untouched.
- tests/e2e/test_response_to_docs.py: the phase-59 single-turn body
expectation moves to the transcript shape ("## 1. <question>" +
the answer's markdown) — the rest of the suite unchanged.
Also lands the phase-74 file moves (00_phase.md /
03_mock_marker_e2e.md -> complete/) and the phase reports — the
house convention of committing .agents/ with the phase.
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH
the conversation so far — every prior user/brain turn and the prior
thinking blocks on brain turns (preserve-thinking) — while
POST /api/chat stays stateless (A10): the client provides the history
in the request body and the server stores nothing new.
Server (task 01):
- ChatRequest.history: optional list[HistoryTurn] (who: user|brain,
text, optional thinking) — absent/empty keeps the request
byte-identical to pre-phase-74 (the two-message [system, user]
request; the kill-switch semantics are pinned in the integration
suite).
- app.rag.prompts.history_to_messages: pure mapper — walks the turns
newest-first against the settings budgets (history_max_turns=40 /
history_max_chars=24000, BOR_HISTORY_MAX_TURNS /
BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut
mid-answer); the kept window is returned oldest-first; brain turns
carry their thinking as reasoning_content (A4) only when
non-empty.
- Both branches feed it: the deflected path splices it between the
system prompt and the current user message (the phase-71 recovery
still rebuilds from messages[1:]), the grounded agent receives
run_agent(..., history=hist); llm.py's message params widen to
list[dict[str, Any]] (string-only messages stay byte-identical on
the wire — the SDK passes message dicts through verbatim).
- The per-turn log line (PLAN §9) gains history_msgs=N after
kb_chars=N.
- Pins: tests/unit/test_history.py (mapper: mapping, reasoning
gating, both budgets, drop-whole, ordering, empty default),
tests/unit/test_config.py (the two settings + env overrides),
tests/unit/test_agent.py (the history splice + the default),
tests/integration/test_chat_api.py (deflected AND grounded forward
the history incl. reasoning_content, no-history byte-identity, 422
pins, the log field).
Client (task 02):
- runTurn — the single funnel for fresh send / phase-49 retry /
phase-53 stale-regen — sends history = the conversation record
minus the current question, with thinking only on brain records
that streamed one (undefined drops the key from the JSON, the
record's convention); the question is never duplicated into the
history.
Wire proof (task 03):
- The mock's echo my history marker (HISTORY_TRIGGER) answers with
the deterministic history echo — history: N prior messages; last
answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE
the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both
turn branches whatever the gate says; the module docstring records
the user/assistant-only history invariant that keeps every
existing (tool-result-classified) marker flow unaffected.
- tests/e2e/test_llm_history.py (isolated): a grounded follow-up and
a deflected follow-up both receive history: 2 prior messages +
thinking: yes + the byte-exact tail of turn 1's answer (derived
from the persisted bor.chat.v1 record — the same array the client
maps into the body); a cold start receives history: 0 prior
messages / last answer tail: none / thinking: no.
- Regressions green in isolation: chat_rag, chat_history (phase 50),
agent_document_tools, harness_aligned_tools, stop_generation,
retry_answer, response_to_docs.
Root cause (task 01): none of C1-C3 - in Chromium 151 (real mode) a
merely-hidden tab neither stops the stream (frames arrive at full rate;
turn completes) nor fires pagehide on tab switch; C1's double-record
path was proven latent via a synthetic pagehide (trigger is
browser-dependent, e.g. Safari) and C2 (the 120s pre-token guard) was
confirmed to fire while hidden.
- C1: the pagehide partial-persist is correlated with the turn's settle
(leavePartialIndex) - the done/stop settle REPLACES it in place
(identity-guarded rememberBrainTurn in-place mode), so bor.chat.v1
and the auto-saved saved_chats row keep exactly ONE brain turn per
question; a real navigation never runs a settle, so the leave-save
is unchanged.
- C2: the visibility re-arm gives the still-armed pre-token guard a
fresh TURN_TIMEOUT_MS when the tab returns to visible - hidden time
no longer counts toward the 120s guard.
- Phase-48 teardown contract untouched: Stop / tab close / real
navigation still cancel the fetch and stop the model.
- Unit pins: tests/unit/test_frontend_hidden_tab.py (the app.js
mechanisms without a browser).
- E2E pins: tests/e2e/test_hidden_tab_stream.py - synthetic pagehide
mid-stream completes exactly once with one brain turn (localStorage
+ auto-saved row), reload restores one bubble, no-event baseline,
and the fake-clock pre-token guard re-arm (discriminating: fails
with the re-arm disabled).
Convert the two unchecked TODO items into executable phases (Protocol B,
appended after the 72 completed phases):
- 73_hidden_tab_stream (TODO L3): a merely-hidden browser tab must never
stop a generating answer; repro/root-cause decision tree + the pagehide
partial-correlation fix + the hidden-tab E2E pin.
- 74_llm_chat_history (TODO L4, history): client-provided history in
POST /api/chat (stateless, A10) mapped through both the deflected and
grounded agent paths, prior thinking blocks preserved via
reasoning_content, capped oldest-first; mock echo marker + E2E.
- 75_save_doc_full_session (TODO L4, save-as-doc): the Save-as-doc draft
body becomes the full session transcript; edit-out happens in the
existing doc-edit body; multi-turn git-verified E2E.
Owner-confirmed assumptions A1-A7 are recorded as ASSUMPTION lines in the
task files. TODO.md is cleared (items now live in .agents/phases/todo/).
Standardize on the .agents/ directory (shared with project skills):
phases/, user_stories/, reports/, screenshots/, validate.sh, and
phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves
history; runtime artifacts move alongside).
Updates every reference in AGENTS.md, README.md, .gitignore, app
docstrings, and test story headers. Historical KB content in data/
and the runtime pipeline.log transcript are left untouched.
Codifies the 2026-09-05 turbo comparison workflow as a project skill under
.agents/skills/: switch BOR_LLM_CHAT_MODEL in .env, run the fixture gate
(twice, for variance) + the locked derived gate with per-turn wall timing,
interpret the two metrics against the reference model rates (re-read habit:
lite ~100%, turbo ~12%; usage-floor MISS as test artifact; caps as real
regression), record the verdicts byte-exact in TOOL_CALLING_TESTING.md, and
commit the doc. Rules baked in: never touch the battery/thresholds/fixtures,
never edit app code, never commit .env.
turbo (2026-09-05, same fixture KB): fixture gate PASS 100%/100% on both
metrics, two runs (wall 105-135s vs lite 43-55s); the redundant re-read
of seeded documents that capped lite's executed ratio at 58-73% is
model-specific (turbo re-read rate ~12% vs ~100% in-sample), corroborating
section 7's framing. Locked derived battery: turbo fails only the >=6/10
tool-turn usage floor (it answers seeded read-target questions from
context instead of making the refusable read call) - accuracy on all
emitted calls still 100%/100%.
Phase 72 (72_teaching_refusals) — completed under the 2026-09-04 controlled
methodology (owner directive: stop clearing/re-importing the homelab KB per
iteration; measure tool-calling accuracy on a controlled fixture KB, target
>90%).
Real-model gate verdicts (live, configured chat model 'lite', fixture KB):
- Controlled fixture battery (the new methodology's pass condition —
contract accuracy >= 90%): PASS, 4 consecutive runs:
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/11 executed (73%) contract 11/11 (100%) 2026-09-04 (wall 43.4s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/13 executed (62%) contract 12/13 (92%) 2026-09-04 (wall 50.6s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 7/11 executed (64%) contract 11/11 (100%) 2026-09-04 (wall 46.8s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 9/15 executed (60%) contract 14/15 (93%) 2026-09-04 (wall 54.8s)
- Locked derived battery (phase-72 task 05, executed >= 90% bar, run
unchanged on the same fixture KB):
gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 5/15 executed (33%) contract 12/15 (80%) 2026-09-04 (wall 47.7s)
The teaching works — every bare-path trap self-corrects in exactly one
round, zero cap hits, zero repeat loops, 10/10 answered. The locked
executed bar is blocked by ALREADY_IN_CONTEXT dedupe refusals on the
corrected re-reads (the trap question seeds its target, so the correct
combined-form read is refused for redundancy) — a copy-invariant model
behavior (five copy variants, 0/15 re-reads flipped, 2026-09-03 -> 04)
and an app-semantics decision for the owner (TOOL_CALLING_TESTING.md
sections 5 and 7), not a copy lever.
Copy changes this phase owns (unit pins updated to follow):
- app/rag/agent.py: ls teaching refusals (path-like scope -> document-path
line; unknown source -> no-source line with the source-name
parenthetical), read/grep 'did you mean source/path?' teaching
(find_path_candidates: exact or suffix path match, catalog order, cap 3),
ALREADY_IN_CONTEXT naming the correct action (answer from the text
already in the prompt), read tool description front-loaded with the
do-not-read rule (the 2026-09-04 controlled telemetry: the re-read is
the only remaining refusal class; contract accuracy 92-100% across runs)
- app/rag/prompts.py: TOOLS_SECTION states the document-identity contract
up front (ls path = source name; read/grep = combined source/path
including the source name; do-not-read for <documents> documents placed
next to the read teaching; one-call-per-reply and never-repeat rules)
- tests: refusal pins (unit + integration), new dedicated E2E suite
tests/e2e/test_tool_path_teaching.py (mock misuse flow, green in
isolation), regression suites green in isolation (harness_aligned_tools,
agent_document_tools, agent_unlimited_tools, search_tool, chat_rag).
Gates: uv run pytest green (1501); coverage TOTAL 99% (>90%); ruff +
pyright clean. Carries the still-uncommitted phase-71 todo/ -> complete/
move and both phases' .agent/reports/ (AGENTS.md 8).
The phase-72 iteration loop cleared the database, git-cloned the homelab repo, re-imported 38-51 documents and re-embedded per run — many minutes per iteration against a different KB every time (owner directive 2026-09-04: stop importing the homelab repo on every test run). Replace it with:
- tests/fixtures/agent_kb/: 8 hand-written markdown docs (sources 'deployments'/'homelab') whose specifics (rack7, 10.77.42.0/24, VLAN 130, rbm-8842, 17 2 * * *, obsidian-bor:2026.7.14, 18765, 18443, ...) no model can guess; read targets carry non-topical filenames so their questions do not lexically seed them (the read must actually happen)
- tests/fixtures/test_kb.dump.sql: data-only snapshot (TRUNCATE + INSERTs incl. embeddings, self-contained git_sources rows, static KB overview) — verified by round-trip checksum at build time
- scripts/load_test_kb.py: one-off rebuild (real pipeline + embeddings, ~2s) that also prints the per-question retrieval report (all 10 battery questions must be grounded)
- scripts/restore_test_kb.py: sub-second one-transaction restore (no git clone, no re-embedding)
- scripts/agent_realmodel_check.py: the gate gains --restore / --mode fixture (curated 10-question battery with one unambiguously correct tool behavior per question) / --turns N (12s micro-loop) / --concurrency / per-turn + total wall timing, and a second accuracy metric (contract accuracy: well-formed calls targeting resolvable entities) alongside the phase-72 locked executed ratio — the re-read of a seeded doc is a copy-invariant model behavior (5 variants, 0/15 flipped) that the dedupe refusal counts as a failure
- TOOL_CALLING_TESTING.md: the human-readable methodology (fast loop, design rules, metrics, copy levers + tried-and-reverted table, current standing, open design question)
Measured: restore 0.03s; micro-loop ~12s; full loop ~43-55s; concurrency 2/3 gives no gain (endpoint serializes).
The model treated the combined 'source/path' string (as printed in
search result lines, read-result headers and refusals) as the
document's identity and passed it as 'source' — e.g.
source='homelab/active/container_caddy/caddy.md' instead of
source='homelab', path='active/container_caddy/caddy.md'.
- Rewrite the read_document description with the split rule (source =
before the FIRST '/', path = after it) and a worked example; share
the source/path parameter descriptions between read_document and
search_documents; map search result lines back onto the split.
- New _resolve_document: on a lookup miss with a '/' in source, retry
at the first slash (source names are directory basenames and can
never contain '/'), plus a continuation candidate for a split at a
later slash; a self-corrected combined form for an already-in-context
document is still rejected as ALREADY_IN_CONTEXT.
- A slash-carrying source that matches nothing gets an educational
refusal naming the corrected arguments instead of the generic line
that repeated the combined form.
Verified live against aipi (lite) + the imported homelab KB: A/B on
the exact failure scenario (5 runs each, right after a
combined-source search result) — old descriptions 5/5 combined, new
descriptions 5/5 clean; two live UI turns (Playwright) produced only
clean split arguments, including a multi-hop read of
install_caddy_deskwork.yaml that landed in done.sources. Full suite:
1376 passed, app coverage 99% (agent.py 100%), ruff + pyright clean,
agent/search E2E green in isolation.
meta description -> locked (A3) auto-save string (history.html L6).
page-sub -> locked (A3) string, the <strong>Save</strong> emphasis retired with the button (L106-109).
empty row -> locked (A3) string; colspan=6, hidden, and row id untouched (L163).
h1, the anonymous gate section, and history.js are byte-identical — state language verified accurate.
- task 01: relocate the .chat-actions row (New chat + Share, comments byte-identical with a Phase 65 note) from the top of the column to the bottom of .chat-shell, directly above the composer
- task 02 (owner-locked A1): wrap the row + #composer in ONE sticky .chat-bottom unit (position: sticky; bottom: env(safe-area-inset-bottom, 0), no z-index) — the pills stay at the bottom of the screen at every scroll position and settle into flow above the footer
- task 03 (owner-locked A2): right-align the bottom row to the column's right edge (justify-content: flex-end), mirroring the right-aligned Save-as-doc corner; the five action pills share one 44px / 999px-pill geometry
- task 04: dedicated Playwright suite tests/e2e/test_bottom_chat_actions.py (resting geometry, the A1 pin across the sticky range, A2 alignment + DOM order + mobile stack + 360px overflow bound + 44px touch targets, New chat / Share click-through) — green in isolation
- task 05: regression matrix green in isolation (pinned_composer 4, save_share_ux 5, chat_persistence 4, share_chat 4, chat_history 5, smoke 3); full gate green — unit + integration pass, app/ coverage 99% (>90%), ruff + pyright clean
Phase 65 (TODO.md L3): move the New chat + Share cluster to the pinned
bottom of the chat column (sticky .chat-bottom unit with the composer)
and right-align the row to the Save-as-doc action corner.
Phase 66 (TODO.md L4): History tab copy — every chat saves
automatically; retire the Save-button references.
Fixed: index.html meta description, empty-state sub and composer
placeholder (A1); app/config.py default suggestion chips → the four
neutral A2 defaults (BOR_SUGGESTIONS override unchanged); sources.html
KB page-sub → the current source model (git repos + local dirs +
uploaded archives, Sync pulls/imports); git-sources.html example URL
→ your-repo.git (A3); all 9 footers → neutral default in
span.footer-text (the phase-62 hook); E2E/unit conftests force the
code defaults so a local .env cannot leak corpus copy into tests;
new unit text pins + dedicated E2E suite.
Task 02 verification read-through — no change needed:
- sources.html sync result/error copy (matches the real sync behavior)
- tuning.html page-sub (accurate as written)
- history.html page-sub (accurate as written)
- doc-edit.html page-sub (accurate as written)
- git-sources.html page-sub (accurate as written)
- #sources-gate anonymous copy (accurate as written)
Remove the blanket .agent/ gitignore so the phase roadmap, user
stories, reports, and PLAN.md are versioned with the code. Only
runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log)
remain ignored. Update AGENTS.md git protocol rule to match.
Phase 52's first pass shipped `position: sticky; bottom` on `.composer` and
called the phase done, but the owner's requirement — "the chat message-input
textarea should be at the bottom of the screen" — still failed in the browser:
on an empty/short chat the input rested just under the empty state (~57% of
the viewport) with a dead band down to the footer.
`position: sticky` can only pull a box UP toward the scrollport's bottom edge;
it can never push a box DOWN to meet it, so on a page that does not overflow
it is a no-op. The old story suite only exercised an overflowing conversation
(one test even asserted the buggy resting position as expected), which is why
the half-fix passed.
- `.messages { flex: 1 1 auto }` — absorbs a short page's free space so the
composer's resting in-flow position is the bottom of the full-height column
(body min-height:100dvh -> .app-main flex:1 -> .chat-shell flex:1); basis
stays `auto`, no height cap, no overflow — the document stays the scroller
- `.composer { bottom: env(safe-area-inset-bottom, 0) }` — the explicit 0
fallback replaces the env()-only offset, which degraded to `auto` (no pin)
wherever env() is unsupported
- E2E: `test_empty_chat_composer_sits_in_normal_flow` ->
`..._at_the_screen_bottom` (chrome-only band below the resting composer);
the phone suite now checks the resting position as well as the pinned one
- Unit pins: the flex-grow half and the full-height column are pinned, so the
fix cannot silently regress to sticky-only
Still CSS-only — no DOM change, no JS, no new scroll call site (phase 42
never-auto-scroll contract intact), no z-index.
Verified: 1019 unit/integration tests pass (app/ coverage 99%), ruff and
pyright clean; tests/e2e/test_pinned_composer.py green in isolation (4), plus
the stop/autoscroll/persistence/mobile-nav suites and 14 layout/scroll
neighbours green in isolation.
- scrollReveal lands at the document bottom (window.scrollTo) instead of
scrollIntoView({ block: 'end' }): the old alignment sat above the
in-flow composer, so every Enter hopped the page up by the
composer+footer height and pushed the composer below the fold.
- The thinking window's pin state is now captured BEFORE the re-render
(const pinned = block.open && isThinkingNearBottom(textEl)): the
post-render distance read the new chunk's rendered height, not the
user's position, so any chunk taller than the 32px band (real-model
deltas, '\n\n' paragraph breaks) killed the follow at the first
2-newline gap.
- Mock LLM: new 'think in paragraphs' trigger (scratchpad with real
blank-line breaks, 60-char frames) — the 12-char mock frames never
rendered past the band, which is why the bug survived the E2E gates.
- E2E (both verified red against the old code):
test_submit_does_not_hop_up, test_thinking_window_follows_across_paragraph_breaks.
- Unit source-marker tests updated to the new contracts.
Phase 49 (owner request, chat 2026-08-28: "The git sources page should
remove local directory and should instead accept a tarball or zipfile
upload which it will unpack and scan … reuploading the same tarball
should not create a new folder, but should unpack and overwrite the
previously unpacked content" — design confirmed in the same
conversation):
* POST /api/git-sources/upload (admin-only, require_admin): accepts
.tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap
(bounds BOTH the compressed upload and the total extracted bytes —
zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink
escape and device/FIFO members rejected), and atomically swaps the
content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the
archive suffix — no missing window, a failed upload never touches the
existing folder/row/KB). The git_sources row is upserted by path
(kind='local', no duplicates, added_at preserved), the models are
checked fail-fast (503 sanitized when down — the folder/row stay
committed and the next sync/re-upload retries idempotently), and the
source is scanned synchronously in the request (single-source
import_sources prune=True + change-gated KB overview), answering 200
with the sync-style counts. One upload at a time (409); the request
session is released before the scan so a concurrent TRUNCATE cannot
deadlock against it.
* app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES,
archive_source_name (safe-name derivation), unpack_archive (guarded
zip/tar extraction with the extracted-byte cap, no partial state),
swap_in (atomic replace with restore-on-failure) — fully unit-tested.
* app/config.py + .env.example: BOR_UPLOAD_DIR (default
~/bor-sources/uploads, deliberately separate from the git checkouts)
and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at
startup on <= 0).
* python-multipart added to the dependencies — FastAPI's required
multipart parser (an A2 implementation detail, phase locked decision).
* The Sources page: the phase-38 "Add a local directory" form is
removed; #archive-upload-form takes its place (labeled file input,
"Upload & scan" button, the §7.4 never-stale lifecycle, inline
role=alert error, role=status count line); hint + table caption
updated. The POST /api/git-sources kind=local API contract is
UNCHANGED — a plain directory is still registrable via the API, and
existing Local rows list/remove/sync exactly as before.
* The phase-38 story E2E (test_local_directory_sources.py) is rewritten
API-driven — the form it drove is gone; its acceptance stands.
* The story E2E (test_archive_upload_sources.py): the swap,
upload→scan→list (the deterministic "Uploading…" in-flight state, the
Local row, /api/docs + the RAG catalog), same-filename re-upload
(in-place replace, prune, no duplicate row, v2-only folder), the
422 inline error + recovery (the form is not wedged), and the
anonymous gate + 403.
* README: the archive-upload section (formats, naming rule, in-place
replace, both new settings), the local-directory form removal noted,
config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB.
Gates: unit+integration green, app/ coverage 99%, the story E2E green
in isolation, the regression suites (git sources admin, local
directory sources, sync button, import documents, nav rename, smoke,
shared header) green in isolation, ruff + pyright clean.
Note: per this phase's file-level staging, frontend/assets/styles.css
also carries the small same-day in-flight owner rework already in the
working tree (the .sign-in-mobile companion rule for the phase-48
mobile sign-in copy); the phase-49 change is the upload form's block.
Owner request (2026-08-28): the two admin-only nav items read like the
same thing, so they are relabeled — the document-catalog link
(#nav-sources, /sources.html) becomes "RAG" and the source-manager link
(#nav-git-sources, /git-sources.html) becomes "Sources".
Phase 48 (48_nav_rename_sources), label-only per the locked decision:
- all six pages (index, sources, git-sources, tuning, document, login):
the two <a> texts swap; ids, hrefs, hidden defaults, is-active /
aria-current placement, and nav order (Chat, RAG, Sources, Tuning)
are byte-unchanged otherwise.
- header.js: comment/docstring label mentions only — the reveal-by-id
logic is untouched (ship-hidden/reveal contract intact).
- test_git_sources_admin.py / test_mobile_hamburger_nav.py: the two
suites that asserted the old label text are updated; comment-only
label fixes in test_shared_header.py / test_nav_consistency.py.
- tests/e2e/test_nav_rename_sources.py: the story E2E (green in
isolation) — renamed labels + unchanged hrefs/order/markers on all
six pages, click navigation with the active marker, the anonymous
ship-hidden contract, and regression guards for the untouched
controls (#sync-label "Sync sources", viewer #doc-back "Sources").
- All eight surrounding header/nav suites stay green in isolation;
unit+integration green, app/ coverage 99% (frontend-only change),
ruff + pyright clean.
Note: per this phase file-level staging, the six page files and
header.js also carry the same-day in-flight owner rework that was
already in the working tree when phase 48 ran (mobile sign-in dropdown
copy, sync button ship-hidden on the Sources page); the label rename
itself is the two-text swap on each page.
Phase 47 (owner permission 2026-08-27, TODO.md L10–11, roadmap R1): the
full Podman quadlet family (.container, .network, .volume, .image,
.pod, .kube, .swap, .os, .endpoint) and .j2 Jinja templates join the
allowed + default A9 import formats, chunked as plain text (owner
decision — no TOML/Jinja-aware splitter). No env configuration needed:
a default import now indexes them.
- app/config.py: _ALLOWED_IMPORT_EXTENSIONS + the default
import_extensions CSV gain the ten names (the original seven first);
the never-widen BOR_IMPORT_EXTENSIONS validator is untouched and
still rejects truly unknown extensions.
- app/rag/chunker.py: ten _FORMAT_CHUNKERS entries -> chunk_text
(HARD_MAX_CHARS 1200 honored, unknown-suffix fallback unchanged);
docstring/comments cite the A9 revision 2026-08-27.
- tests/fixtures/docs/homelab/: quadlet/compose.container (realistic
quadlet TOML, >1500 chars, [Unit]/[Service]/[Container] sections,
RESE-QUADLET-SENTINEL-77aa), quadlet/lan.network,
quadlet/cache.volume, templates/deploy.j2 (for/set/if Jinja
constructs + RESE-JINJA-SENTINEL-33dd). Every suite that seeds the
fixture tree updates its 9 -> 13 document-count constants.
- tests/unit/test_config.py: allowed set carries all seventeen formats,
default CSV + dotted import_extension_set include the ten, the
validator accepts the new names and still rejects unknowns.
- tests/unit/test_chunker.py: dispatch parity with chunk_text for every
new suffix (parametrized), the .container fixture chunks >=2 under
the cap with the sentinel surviving, the .j2 fixture keeps {{ }}
verbatim, the unknown-suffix fallback is unchanged.
- tests/unit/test_importer.py: a default-extensions walk over a temp
tree indexes exactly the ten new files (unknown/hidden/excluded
filtered), the original seven still walk, stem-title fallback holds.
- tests/integration/test_import_quadlet_jinja.py (new): import_sources
over a temp tree with .container/.volume/.j2 -> documents + chunks
rows with stem titles; delta re-import updates only the changed .j2
doc; prune drops the deleted .volume doc with cascade.
- tests/e2e/test_quadlet_jinja_import.py (new, story suite, mock-only,
isolation): GET /api/docs (admin session) lists the four new-format
docs with non-zero chunk counts and stem titles; the Sources table
renders a row + .doc-link per file; the phase-26 modal shows the
.container TOML ([Container] section + sentinel) with stem title and
the container format badge; a RESE-JINJA-SENTINEL-33dd question
FTS-matches the .j2 chunk -> honest-positive (A8: LOW requires zero
FTS hits) — the bubble is not .is-deflected and a source chip names
templates/deploy.j2.
- README.md + .env.example: the extended default format set (A9
revised 2026-08-27, plain-text chunking, narrow-only rule intact).
- .agent/PLAN.md: the A9 revision (owner-locked R1) — A9 row status,
the revision note under the anchors table, and the §5 chunking-policy
+ §11 workflow lines. The only PLAN edit this phase.
Gates: uv run pytest 795 passed; app/ coverage TOTAL 99% (>90%);
ruff check + pyright clean; story E2E 4/4 in isolation (DB up);
regression E2E suites test_import_documents (3) / test_sync_button
(3) / test_git_sources_admin (6) green in isolation.
Also records the 47_quadlet_jinja_import task-file moves (01–03)
todo/ -> complete/.
TODO.md L9 (owner permission 2026-08-27, roadmap A5): "The navbar on
mobile is way too squished. Make it a hamburger dropdown menu with a
nice animation." At <=640px the nav links leave the bar — a 44px
#nav-toggle opens #app-nav as an animated (180ms slide+fade)
edge-to-edge dropdown with comfortable rows and the auth visibility
contract intact inside the menu; at >640px the bar is byte-identical
to pre-phase-46 (hamburger absent, inline pills as before).
- frontend/*.html (all six pages): the shared bar gains the
#nav-toggle button (type=button, aria-expanded=false,
aria-controls="app-nav", aria-label="Menu", aria-hidden 3-line
SVG icon) immediately before the nav, and the nav gains
id="app-nav" — one <nav>, no duplicated links, so the whoami reveal
works inside the menu unchanged (phase-34 same-bar contract intact).
- frontend/assets/styles.css: .nav-toggle is display:none outside media
queries (desktop untouched); the <=640px block adds the 44px toggle
(+hover in the .steering-toggle:hover family, sized 20px icon), turns
.app-nav into the dropdown (absolute top:100% edge-to-edge under the
sticky header, surface + hairline + --shadow-lg, z-index 21 =
header+1, closed state invisible + non-interactive with the 180ms
opacity/transform/visibility-delayed pair, .is-open the only
opener), and comfortable 1rem/0.75rem menu rows — superseding the
phase-34/35 pill-squeeze rules for .nav-link/.app-nav (the 900px
tablet block, action pills, and 58px bar height untouched). The
reduced-motion block stills BOTH the closed and .is-open states: the
.is-open rule (0,2,0) out-specifies a bare .app-nav (0,1,0), so the
override must name both — verified live in Chromium (task 03).
- frontend/assets/header.js: ONE module-owned binding (import-time,
null-safe like the sign-out binding): click toggles .is-open +
aria-expanded in sync, a delegated nav-link click closes, Esc closes
and refocuses the toggle, and matchMedia("(max-width: 640px)")
change drops the state on resize back to desktop. The binding
touches only the container — ship-hidden whoami links stay hidden.
- tests/unit/test_hamburger_nav.py (new): the markup/CSS/JS contract
pins (six identical toggles in the shared row, desktop byte-
identical, dropdown + .is-open + 180ms + reduced-motion rules, the
superseded squeeze rules gone, the one-binding behavior).
- tests/e2e/test_shared_header.py: assert_shared_bar gains mobile=True
(at <=640px the bar shows the hamburger + the closed nav; the
per-role menu contents are pinned by the story suite).
- tests/e2e/test_mobile_hamburger_nav.py (new, story suite, 375x812):
toggle is a visible >=44px target, menu closed (opacity 0 /
visibility hidden), no horizontal overflow; anonymous menu shows
exactly "Chat" (admin-only links stay hidden inside); admin menu
shows all four links (whoami reveal inside the menu); a link click
navigates + the arrival page ships closed; Esc closes and refocuses
the toggle (outside click does NOT close — accepted: the locked
close set is Esc + link + resize, no backdrop); the 180ms
opacity/transform pair is live and reducedMotion:reduce stills both
states with open/close still working; 1280x800 regression — toggle
display:none, all four inline links inside the header band.
Gates: unit+integration 773 passed; app/ coverage TOTAL 99%
(unchanged — frontend-only phase); story E2E 7 passed in isolation
(mock LLM, DB up); regression suites test_nav_consistency (6) /
test_header_consistency (3) / test_shared_header (6) /
test_responsive_polish (7) / test_tuning_nav_link (4) all pass in
isolation; ruff check + pyright clean. A11 honored: no CDN, no new
assets.
Also records the 46_mobile_hamburger_nav todo/ -> complete/ move.
Phase 45 (owner permission 2026-08-27, TODO.md L8: "allow the LLM
to make as many tool calls as it wants"): the phase-37 per-turn tool
budgets (BOR_AGENT_LIST_CALLS / BOR_AGENT_READ_CALLS, default 1 each)
and their exhaustion refusals are removed — a grounded turn now offers
list_documents / read_document for the whole turn (re-lists included),
bounded only by the round cap:
- app/config.py: agent_max_rounds (BOR_AGENT_MAX_ROUNDS, default 10,
negative rejected) replaces agent_list_calls / agent_read_calls;
.env.example + README document the single knob; app/rag/prompts.py
docstrings follow.
- app/rag/agent.py: the loop runs tools until the model answers or
rounds >= max_rounds, at which point it forces one final no-tools
answer (the cap is the only forced exit); 0 = no tools — exactly one
tools=None request, byte-identical to the pre-phase-37 path (the
kill switch). Rejected calls (unknown tool / missing args /
already-in-context / unknown path) still consume a round, so
pathological rejected-call streams are bounded by the cap. The
per-call log line is now tool/args/round=N/M; the per-turn
tool_calls=N field and the tool SSE event are unchanged.
- tests/e2e/mock_llm.py: MULTI_READ_TRIGGER ("read two documents") —
the deterministic list -> read #1 -> read #2 -> forced-answer flow
(byte-stable "I read <sp1> and <sp2>." line), classified by the
count of tool-role read results; the phase-37 single-read flow stays
byte-identical (unit-pinned in tests/unit/test_mock_tool_flow.py).
- tests/e2e/test_agent_unlimited_tools.py (new, story suite,
mock-only): three tool frames/lines in order (one list, two reads —
the second read is what the old read budget refused) + the
both-named non-deflected answer; done.sources + chips = retrieval
doc + both reads, deduped; no budget refusal rendered; the
single-read marker flow regression (exactly one read, single tool
pair).
- .agent/PLAN.md: the phase-45 SSE revision note (owner-locked, R2) —
the only PLAN edit this phase; the phase-37 note's budget clause is
marked removed.
Unit/integration rewrites (test_agent.py round-cap matrix incl. the
kill switch and rejected-call spam, test_config.py, test_chat_api.py
agent_max_rounds=0 fixtures) landed with the server core so every gate
stays green.
uv run pytest: 756 passed, app/ coverage 99%; ruff + pyright clean;
story E2E 4/4 in isolation (ran twice); regression E2E suites
(agent_document_tools unmodified, chat_rag, smoke) green in isolation.
Also records the 45_agent_unlimited_tools todo/ -> complete/ task-file
moves (00/01/02 pending in the working tree, task 03 moves on success).
GFM pipe tables in the shared renderer (TODO.md L6): a table-protection
pass in frontend/assets/markdown.js (fences -> tables -> escape order)
pulls each header+separator+body block out as a placeholder, renders
cells escape-first with the same inline transforms, and reinserts a
semantic <table class="md-table"> inside a horizontal-overflow
.md-table-wrap — so a pipe table in a chat answer, the document
viewer/modal, and the thinking block all render the same semantic
table. Fences win over tables; lone pipes stay text.
- styles.css: .md-table palette rules (PLAN §7.2 tokens, no motion);
min-width: max-content so a WIDE table keeps its natural width and
the wrapper is the real scroller (width:100% alone wrapped the wide
table's cells — proven by the new E2E).
- mock_llm.py: TABLE_TRIGGER ("show me a table") -> byte-stable
TABLE_ANSWER (3-column table, <img onerror> XSS probe line, wide
5-column table), checked before DEFLECT_MODE like SUMMARY_MODE.
- tests/fixtures/docs/homelab/tables.md: 3x3 pipe table + pipe-heavy
fenced block (viewer/fence subject); the shared fixture set grows
8 -> 9 docs, so every suite pinning the count (added/formats/
stat-docs/EXPECTED_ROWS) is updated accordingly.
- tests/e2e/test_markdown_tables.py (new, story suite): chat table
shape + non-deflection, wide-table wrapper scroll (no page
overflow), XSS probe inert, viewer modal table, fence-not-a-table,
lone pipe stays text.
- tests/e2e/test_agent_document_tools.py: fix a pre-existing flake —
the "Calling tool…" label window is ~0.4 s at the mock's 0.1 s
tool-frame pacing, and a polling expect could stride over it
(failed 3 of 5 runs on the committed baseline). The pre-submit
MutationObserver record is the deterministic source of truth; the
racy to_have_text gate is gone.
uv run pytest: 738 passed, app/ coverage 99% (TOTAL unchanged);
ruff + pyright clean; story E2E 6/6 in isolation; regression E2E
suites (chat_rag, document_viewer, document_summaries, smoke) green.
TODO.md L7: "Add scrolling back to the thinking block, but have it
autoscroll while thinking content is generating." Owner direction
2026-08-27 (roadmap A2) reverses the phase-21 no-scroll choice
(2026-08-24): details.thinking .thinking-text is user-scrollable again
(overflow-y: auto — the 320px clip stays, owner-confirmed), and the
phase-17 per-chunk bottom-pin is GATED: the window follows the live
tail only while the user is pinned near its bottom (THINKING_NEAR_
BOTTOM_PX = 32); scrolling up pauses the follow, returning to the
bottom re-arms it on the next chunk (the gate re-runs on every frame).
- frontend/assets/styles.css: .thinking-text overflow-y: hidden ->
auto; the phase-21 owner-choice comment is replaced with the
2026-08-27 direction; max-height: 320px and every other declaration
in the rule byte-identical.
- frontend/assets/app.js: export const THINKING_NEAR_BOTTOM_PX = 32 +
isThinkingNearBottom(textEl) (scrollHeight - scrollTop -
clientHeight <= band); the thinking-handler pin becomes
`if (block.open && isThinkingNearBottom(textEl))` — a scrolled-up
reader is never re-pinned and a closed (restored) block is never
pinned; everything else in the handler (and phase 42's no page
scroll) untouched.
- tests/unit/test_thinking_scroll.py (new, replaces the deleted
tests/unit/test_thinking_no_scroll.py): pins the CSS contract (auto
+ 320px + owner-direction comment, no hidden/scroll left), the
exported 32px band, the gate math, the gated pin (no unconditional
`if (block.open)` remains), and the surviving collapsed-restore pin.
- tests/e2e/test_thinking_scroll.py (new, mock-only, replaces the
deleted tests/e2e/test_thinking_no_scroll.py — its pins asserted the
reversed phase-21 behavior, so both phase-21 files are deleted in
this commit): user scroll restored on the frozen 4s-hesitation tail
(wheel is 1:1; click+Home keyboard — the plain div is not
keyboard-focusable by design, tabindex is test scaffolding; the
literal drag holds the user's position — headless Chromium's
overlay scrollbars are not grabbable by synthetic mouse events,
documented in the suite), follow-while-pinned at the 2nd-to-last
and last chunk (±1px) with the last chunk's text inside the visible
rectangle, no re-pin over ≥5 mid-stream chunks after a
half-window scroll-up, re-pin on the next chunk after returning to
the bottom, the CSS contract, plus the phase-11 (long answer: page
scrolls, bubble overflow untouched) and phase-17 (restored
collapsed block with full text) regressions.
- tests/unit/test_chat_persistence.py: the CSS pin flips with the
contract (auto in, hidden out — owner direction 2026-08-27).
- tests/unit/test_frontend_scroll.py: the "page-level band constant is
gone" pin now excludes the phase-43 window-level
THINKING_NEAR_BOTTOM_PX (a different band — the window's, not the
page's).
E2E (isolation): test_thinking_scroll 7/7 (twice); regressions
test_thinking_display 5/5, test_chat_persistence 4/4,
test_no_reply_autoscroll 5/5, test_smoke 3/3; unit+integration 725
passed, app/ coverage 99% (unchanged — frontend-only phase);
ruff + pyright clean.
TODO.md L5: "Get rid of the chat reply autoscroll, it's breaking things
like making it impossible for the user to scroll while a reply
generates." Owner direction 2026-08-27 (roadmap A1) revises the
phase-18 follow-the-bottom choice: the page NEVER auto-scrolls while a
turn streams. Kept (owner decision): the submit reveal (the user's own
message) and the one-shot phase-14 restore landing.
- frontend/assets/app.js: delete NEAR_BOTTOM_PX + isNearBottom;
scrollReveal becomes the one unconditional scrollIntoView (still
smooth, still "auto" under prefers-reduced-motion via SCROLL);
addMessage(who, html, scroll = false) carries an explicit scroll
intent — only the submit (", true") and the two restore landings
scroll. The thinking/tool/delta handlers and the typing indicator
drop their page-scroll calls; the thinking block's INTERNAL
bottom-pin (textEl.scrollTop, phase 17 — reworked separately in
phase 43) and the turn-end focus({ preventScroll: true }) survive.
- tests/unit/test_frontend_scroll.py: rewritten pin for the new
contract — phase-18 gate absent, helper unconditional, explicit
intent at submit/restore, no page-scroll call in the streaming
handlers, typing bubble scroll-free, SCROLL reduced-motion intact.
- tests/unit/test_chat_persistence.py: restore-landing pin updated to
the new signature (the old forced "auto" is gone; the landing
rides the default SCROLL — noted at the call site).
- tests/e2e/test_no_reply_autoscroll.py (new, replaces the deleted
test_follow_bottom_scroll.py): no autoscroll across >=10 samples
(1px tolerance) during a long answer and during the thinking stream;
submit-from-the-top still reveals the user message; the restore
landing lands one-shot on the latest message and stays; long answer
+ sources and the collapsed thinking block persist and restore.
E2E (isolation): test_no_reply_autoscroll.py 5/5; regressions
test_chat_rag 3/3, test_thinking_display 5/5,
test_chat_persistence 4/4, test_long_answers 2/2, test_smoke 3/3;
unit+integration 723 passed, app/ coverage 99%; ruff + pyright clean.
TODO.md L4: with a dead model endpoint the sync discovered it only
mid-import, after slow clones — and a tooltip on the button is not a
readable error.
- app/rag/llm.py: ModelUnavailableError + check_models(llm) — a tiny
pre-sync probe (one short embedding + one 1-token-scale completion)
that fails naming the unavailable model (embed first, then the
summary model); the sync sanitizer still masks credentials.
- app/api/sync.py: the probe is step 1 of _run_sync — before source
resolution and before any clone_or_pull; a model failure is just
another 'failed' state (no new endpoint, A10/A12 untouched).
- frontend/assets/header.js: applySyncFailure now also opens the
module-owned error modal (every page carrying #sync-btn, zero
page-markup changes): lazily built backdrop + role=alertdialog
panel, error text via textContent, close via button / Esc /
backdrop, focus in-and-out to #sync-btn (with a body→#sync-btn
fallback — the run's disabled button drops focus to <body>).
- frontend/assets/styles.css: the modal on the phase-08 error palette
(z-index above the header, .is-open open/close, reduced-motion
stilling, 44px close target).
- Tests: probe unit tests (both up / embed down / summary down /
custom model names), sync integration (fail-fast before any clone,
probe-before-effective_sources ordering, credential masking,
healthy regression), the phase-41 source pins, and the story E2E
(two module apps on distinct ports — dead endpoint on a closed
loopback port vs session mock: ≤10 s fail-fast + modal contract,
all three dismissal paths with focus out to #sync-btn, button
title/.is-error + Sources banner untouched, healthy phase-32
lifecycle regression to 'Synced HH:MM').
E2E (isolation): test_sync_model_down.py 4/4, test_sync_button.py
3/3, test_git_sources_admin.py 6/6, test_local_directory_sources.py
3/3; unit+integration 721 passed, app/ coverage 99%; ruff + pyright
clean.
#steering-toggle (the header 'Tuning' button) shipped visible in all
six pages and was only removed after /api/whoami resolved, so
anonymous visitors saw it flash for the whole round-trip (TODO.md L3).
It now ships hidden on every page and initSharedHeader unhides it only
for admin — the same ship-hidden / reveal-for-admin contract as the
admin-only nav links; the anonymous end-state (removed from the DOM,
phase-16 'absent, not hidden') is unchanged.
Adds the story E2E suite (MutationObserver proves zero visible frames
for anonymous on every page, admin reveal + panel + count badge,
nav-contract regression) and the source-level unit pins. Also fixes
test_steering.py's BASE_SCRIPT_COUNT (2 → 3: brand.js + markdown.js +
app.js, since phase 39).
Converts the 9 TODO items into an executable phase roadmap (Protocol B,
appended after phase 39):
- 40 tuning toggle anonymous flash (TODO L3)
- 41 sync fail-fast + modal when a model is down (TODO L4)
- 42 no reply autoscroll (TODO L5)
- 43 thinking scroll back — user scroll + gated autoscroll (TODO L7)
- 44 markdown tables (TODO L6)
- 45 agent unlimited tool calls behind BOR_AGENT_MAX_ROUNDS (TODO L8)
- 46 mobile hamburger nav (TODO L9)
- 47 quadlet + jinja import formats, A9 revision (TODO L10–L11)
Each phase carries a user story, a dedicated Playwright E2E suite plan,
and owner-locked decisions (R1 A9 format extension, R2 phase-37 budget
revision, A1–A5 scope decisions) confirmed 2026-08-27.
Also records the completed phases 30–39 todo/ -> complete/ moves that
were pending in the working tree. TODO.md is cleared (items now live in
.agent/phases/todo/).
One env var (BOR_APP_NAME, default "Brain of Reese") now drives the app's
display name everywhere (TODO.md L12 — owner ask: "a way to customize the
name for 'Brain of'. Should be an env var."). The existing app_name setting
is the source of truth (phase locked decision — no new variable, no rename);
with the variable unset the app is byte-identical to before.
Endpoint (A10 public/stateless, no secrets):
GET /api/config → exactly {app_name, version} (app/api/config.py, the
health.py pattern; registered before the static mount). Integration tests:
anonymous 200, default values, a Settings override follows, key set is
exactly two keys — no other setting may leak in later.
Frontend brand layer (A11 — runtime fetch, static templates stay static):
assets/brand.js — a CLASSIC script, first on all six pages, so its top
level runs at parse time: window.BOR_BRAND = "Brain of Reese"
synchronously (the default renders immediately, no blank flash), then a
no-store fetch of /api/config applies the name — document.title (global
replace), every .brand-text (a name starting "Brain of " keeps the bold
split Brain of <strong>rest</strong>, any other name renders plain; the
operator-controlled name is HTML-escaped before innerHTML), a TreeWalker
over text nodes (script/style rejected — page source never rewritten),
and the aria-label/placeholder/meta-content attributes. Fetch failure
keeps the default + console.warn (the loadHealth house style).
app.js (status labels, typing label, elapsed-hint aria, tool labels) and
document.js (viewer titles) read window.BOR_BRAND at CALL time via
brand() — a label set after the fetch lands carries the configured name.
Containerfile: esbuild minify line for brand.js (classic, like markdown.js);
the phase-33 ?v= cache-busting picks the new asset ref up automatically.
E2E (A16 — one story, one file, isolated): test_configurable_brand.py boots
a SECOND app instance (same DB/mock-LLM/admin-auth env block, port APP_PORT+1,
BOR_APP_NAME="Brain of Testy") — the shared conftest server keeps the
default name so every other suite's title/label assertions stay untouched —
and asserts /api/config on both instances, the index title/brand/greeting/
#messages aria-label, the sources + login page titles, and one pre-token
chat turn (think out loud marker) whose #send-status reads "Brain of Testy
is thinking"; the no-op regression pins the shared server's default bytes.
Docs: .env.example App section + README configuration reference — what it
affects (titles, header brand, status labels, aria text), the default, the
bold-split rendering rule.
Gates: 695 unit+integration passed, app/ coverage 99% (>90%), story E2E
green in isolation (two consecutive runs), brand-string suites (smoke,
shared header, header consistency, chat persistence) green, ruff + pyright
clean.
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.
Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.
API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).
Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.
Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.
Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.
Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
Grounded chat turns now run the agent loop (app/rag/agent.py) instead
of a bare chat_stream: while the per-turn budgets last
(BOR_AGENT_LIST_CALLS / BOR_AGENT_READ_CALLS, default 1 each) the model
gets list_documents (the indexed catalog, /api/docs order) and
read_document (full text, never truncated — A7-revised contract); once
both budgets are spent the tools key is dropped from the request and
the model must answer. Rejected calls (unknown tool, unknown/missing
path, document already in context, spent budget) consume no budget.
Budgets 0/0 make exactly one tools=None request — byte-identical to
the pre-phase path (budgets-as-kill-switch). Deflected turns keep the
direct chat_stream (A8 unchanged; the LOW prompt never carries the
<tools> section).
SSE contract gains {"type":"tool","name":...,"argument":
"source/path"|null} frames ahead of the answer deltas (PLAN §4
extension, owner permission 2026-08-26); done.sources, query_log.sources
and the per-turn log line (gains tool_calls=N) report the retrieval
docs + read docs, deduped. The UI shows a "calling tool"
button/label state and one visible .tool-call line per call above the
answer; the lines persist with the chat record and re-render on
reload. chat_stream passes tools through and accumulates streaming
tool_calls deltas into ToolCallPiece (tools=None stays byte-identical).
E2E: deterministic mock tool flow ("use your tools" + <tools> marker:
list -> read first catalog line -> quoted answer) plus the story suite
(marker flow, reload re-render, plain/deflected no-tool regressions).
Docs: .env.example + README (the two tools, the budgets, the SSE tool
frame, the "calling tool" UI state).
probe: turbo tool_calls=supported 2026-08-26 (uv run python -m
scripts.llm_probe --tools — non-streaming + streaming
finish_reason=tool_calls, indexed delta.tool_calls partials)
Phase 33 (story: .agent/user_stories/cache-busting.md).
- app/core/caching.py: asset_version() — git short SHA (a commit is a
deploy), stable content-hash fallback for non-git checkouts, "dev"
for a missing static dir; computed once per process. CachingMiddleware
— the five HTML pages revalidate (no-cache) with ?v=<token> asset refs
rewritten in flight; /assets/* is public, max-age=31536000, immutable;
everything else (all /api/*, the SSE chat stream in particular) passes
through byte-identical.
- tests/e2e/test_cache_busting.py: fresh-Chromium wire assertions —
document no-cache, versioned CSS/JS request URLs sharing one token,
immutable asset headers, /api/health baseline headers, SSE chat to
done (mock LLM).
- README 'Caching / deploys' section + story file.
Also fixed two prod-image defects surfaced by this phase's podman smoke
(the full app would not boot):
- Containerfile: ship the scripts/ package — app/api/sync.py (phase 32)
imports scripts.git_sync / scripts.import_docs at module level, so the
container crashed on boot (ModuleNotFoundError: No module named
'scripts').
- compose.yaml: pass BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET through to
the app service (:- defaults keep 'podman compose up -d db' working;
the app's own fail-loud gate still names missing admin auth).
Smoke: podman compose --profile prod up -d on a fresh image + a fresh
Chromium profile — /, /sources.html and /login.html all served
Cache-Control: no-cache; all 8 asset requests versioned with one shared
token (content-hash fallback inside the image — no .git there);
/assets/* immutable for a year.
ducoterra