Commit Graph
55 Commits
Author SHA1 Message Date
ducoterra 7c6763319b fix(chat): stop autoscrolling while a reply streams (owner direction)
TODO.md L5: "Get rid of the chat reply autoscroll, it's breaking things
like making it impossible for the user to scroll while a reply
generates." Owner direction 2026-08-27 (roadmap A1) revises the
phase-18 follow-the-bottom choice: the page NEVER auto-scrolls while a
turn streams. Kept (owner decision): the submit reveal (the user's own
message) and the one-shot phase-14 restore landing.

- frontend/assets/app.js: delete NEAR_BOTTOM_PX + isNearBottom;
  scrollReveal becomes the one unconditional scrollIntoView (still
  smooth, still "auto" under prefers-reduced-motion via SCROLL);
  addMessage(who, html, scroll = false) carries an explicit scroll
  intent — only the submit (", true") and the two restore landings
  scroll. The thinking/tool/delta handlers and the typing indicator
  drop their page-scroll calls; the thinking block's INTERNAL
  bottom-pin (textEl.scrollTop, phase 17 — reworked separately in
  phase 43) and the turn-end focus({ preventScroll: true }) survive.
- tests/unit/test_frontend_scroll.py: rewritten pin for the new
  contract — phase-18 gate absent, helper unconditional, explicit
  intent at submit/restore, no page-scroll call in the streaming
  handlers, typing bubble scroll-free, SCROLL reduced-motion intact.
- tests/unit/test_chat_persistence.py: restore-landing pin updated to
  the new signature (the old forced "auto" is gone; the landing
  rides the default SCROLL — noted at the call site).
- tests/e2e/test_no_reply_autoscroll.py (new, replaces the deleted
  test_follow_bottom_scroll.py): no autoscroll across >=10 samples
  (1px tolerance) during a long answer and during the thinking stream;
  submit-from-the-top still reveals the user message; the restore
  landing lands one-shot on the latest message and stays; long answer
  + sources and the collapsed thinking block persist and restore.

E2E (isolation): test_no_reply_autoscroll.py 5/5; regressions
test_chat_rag 3/3, test_thinking_display 5/5,
test_chat_persistence 4/4, test_long_answers 2/2, test_smoke 3/3;
unit+integration 723 passed, app/ coverage 99%; ruff + pyright clean.
2026-08-28 00:29:25 -04:00
ducoterra 6cf1df9bf2 feat(sync): fail fast with a modal when a model is unavailable
TODO.md L4: with a dead model endpoint the sync discovered it only
mid-import, after slow clones — and a tooltip on the button is not a
readable error.

- app/rag/llm.py: ModelUnavailableError + check_models(llm) — a tiny
  pre-sync probe (one short embedding + one 1-token-scale completion)
  that fails naming the unavailable model (embed first, then the
  summary model); the sync sanitizer still masks credentials.
- app/api/sync.py: the probe is step 1 of _run_sync — before source
  resolution and before any clone_or_pull; a model failure is just
  another 'failed' state (no new endpoint, A10/A12 untouched).
- frontend/assets/header.js: applySyncFailure now also opens the
  module-owned error modal (every page carrying #sync-btn, zero
  page-markup changes): lazily built backdrop + role=alertdialog
  panel, error text via textContent, close via button / Esc /
  backdrop, focus in-and-out to #sync-btn (with a body→#sync-btn
  fallback — the run's disabled button drops focus to <body>).
- frontend/assets/styles.css: the modal on the phase-08 error palette
  (z-index above the header, .is-open open/close, reduced-motion
  stilling, 44px close target).
- Tests: probe unit tests (both up / embed down / summary down /
  custom model names), sync integration (fail-fast before any clone,
  probe-before-effective_sources ordering, credential masking,
  healthy regression), the phase-41 source pins, and the story E2E
  (two module apps on distinct ports — dead endpoint on a closed
  loopback port vs session mock: ≤10 s fail-fast + modal contract,
  all three dismissal paths with focus out to #sync-btn, button
  title/.is-error + Sources banner untouched, healthy phase-32
  lifecycle regression to 'Synced HH:MM').

E2E (isolation): test_sync_model_down.py 4/4, test_sync_button.py
3/3, test_git_sources_admin.py 6/6, test_local_directory_sources.py
3/3; unit+integration 721 passed, app/ coverage 99%; ruff + pyright
clean.
2026-08-27 23:44:35 -04:00
ducoterra 6f9e033117 fix(header): ship the tuning toggle hidden — no anonymous flash
#steering-toggle (the header 'Tuning' button) shipped visible in all
six pages and was only removed after /api/whoami resolved, so
anonymous visitors saw it flash for the whole round-trip (TODO.md L3).
It now ships hidden on every page and initSharedHeader unhides it only
for admin — the same ship-hidden / reveal-for-admin contract as the
admin-only nav links; the anonymous end-state (removed from the DOM,
phase-16 'absent, not hidden') is unchanged.

Adds the story E2E suite (MutationObserver proves zero visible frames
for anonymous on every page, admin reveal + panel + count badge,
nav-contract regression) and the source-level unit pins. Also fixes
test_steering.py's BASE_SCRIPT_COUNT (2 → 3: brand.js + markdown.js +
app.js, since phase 39).
2026-08-27 22:38:58 -04:00
ducoterra 02c76ad328 chore(agent): phase roadmap from TODO.md — 8 phases (40–47), 24 tasks
Converts the 9 TODO items into an executable phase roadmap (Protocol B,
appended after phase 39):

- 40 tuning toggle anonymous flash (TODO L3)
- 41 sync fail-fast + modal when a model is down (TODO L4)
- 42 no reply autoscroll (TODO L5)
- 43 thinking scroll back — user scroll + gated autoscroll (TODO L7)
- 44 markdown tables (TODO L6)
- 45 agent unlimited tool calls behind BOR_AGENT_MAX_ROUNDS (TODO L8)
- 46 mobile hamburger nav (TODO L9)
- 47 quadlet + jinja import formats, A9 revision (TODO L10–L11)

Each phase carries a user story, a dedicated Playwright E2E suite plan,
and owner-locked decisions (R1 A9 format extension, R2 phase-37 budget
revision, A1–A5 scope decisions) confirmed 2026-08-27.

Also records the completed phases 30–39 todo/ -> complete/ moves that
were pending in the working tree. TODO.md is cleared (items now live in
.agent/phases/todo/).
2026-08-27 18:25:53 -04:00
ducoterra 492d8275e7 add git to containerfile
Build and Push Containers / build-and-push-app (push) Successful in 2m17s
Build and Push Containers / build-and-push-db (push) Successful in 15s
2026-08-27 10:32:12 -04:00
ducoterra beaca4fe97 add db build and push
Build and Push Containers / build-and-push-app (push) Successful in 16s
Build and Push Containers / build-and-push-db (push) Successful in 10s
2026-08-27 10:00:21 -04:00
ducoterra fe55be0c35 feat(brand): configurable app name — BOR_APP_NAME drives /api/config + the frontend brand layer
Build and Push Containers / build-and-push (push) Successful in 1m50s
One env var (BOR_APP_NAME, default "Brain of Reese") now drives the app's
display name everywhere (TODO.md L12 — owner ask: "a way to customize the
name for 'Brain of'. Should be an env var."). The existing app_name setting
is the source of truth (phase locked decision — no new variable, no rename);
with the variable unset the app is byte-identical to before.

Endpoint (A10 public/stateless, no secrets):
  GET /api/config → exactly {app_name, version} (app/api/config.py, the
  health.py pattern; registered before the static mount). Integration tests:
  anonymous 200, default values, a Settings override follows, key set is
  exactly two keys — no other setting may leak in later.

Frontend brand layer (A11 — runtime fetch, static templates stay static):
  assets/brand.js — a CLASSIC script, first on all six pages, so its top
  level runs at parse time: window.BOR_BRAND = "Brain of Reese"
  synchronously (the default renders immediately, no blank flash), then a
  no-store fetch of /api/config applies the name — document.title (global
  replace), every .brand-text (a name starting "Brain of " keeps the bold
  split Brain of <strong>rest</strong>, any other name renders plain; the
  operator-controlled name is HTML-escaped before innerHTML), a TreeWalker
  over text nodes (script/style rejected — page source never rewritten),
  and the aria-label/placeholder/meta-content attributes. Fetch failure
  keeps the default + console.warn (the loadHealth house style).
  app.js (status labels, typing label, elapsed-hint aria, tool labels) and
  document.js (viewer titles) read window.BOR_BRAND at CALL time via
  brand() — a label set after the fetch lands carries the configured name.
  Containerfile: esbuild minify line for brand.js (classic, like markdown.js);
  the phase-33 ?v= cache-busting picks the new asset ref up automatically.

E2E (A16 — one story, one file, isolated): test_configurable_brand.py boots
a SECOND app instance (same DB/mock-LLM/admin-auth env block, port APP_PORT+1,
BOR_APP_NAME="Brain of Testy") — the shared conftest server keeps the
default name so every other suite's title/label assertions stay untouched —
and asserts /api/config on both instances, the index title/brand/greeting/
#messages aria-label, the sources + login page titles, and one pre-token
chat turn (think out loud marker) whose #send-status reads "Brain of Testy
is thinking"; the no-op regression pins the shared server's default bytes.

Docs: .env.example App section + README configuration reference — what it
affects (titles, header brand, status labels, aria text), the default, the
bold-split rendering rule.

Gates: 695 unit+integration passed, app/ coverage 99% (>90%), story E2E
green in isolation (two consecutive runs), brand-string suites (smoke,
shared header, header consistency, chat persistence) green, ruff + pyright
clean.
2026-08-27 02:24:16 -04:00
ducoterra 94d7228510 feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.

Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.

API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).

Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.

Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.

Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.

Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
2026-08-27 01:04:16 -04:00
ducoterra 15c1272828 feat(rag): agent document tools — list/read tools with env-tuned budgets, SSE tool events + "calling tool" UI
Grounded chat turns now run the agent loop (app/rag/agent.py) instead
of a bare chat_stream: while the per-turn budgets last
(BOR_AGENT_LIST_CALLS / BOR_AGENT_READ_CALLS, default 1 each) the model
gets list_documents (the indexed catalog, /api/docs order) and
read_document (full text, never truncated — A7-revised contract); once
both budgets are spent the tools key is dropped from the request and
the model must answer. Rejected calls (unknown tool, unknown/missing
path, document already in context, spent budget) consume no budget.
Budgets 0/0 make exactly one tools=None request — byte-identical to
the pre-phase path (budgets-as-kill-switch). Deflected turns keep the
direct chat_stream (A8 unchanged; the LOW prompt never carries the
<tools> section).

SSE contract gains {"type":"tool","name":...,"argument":
"source/path"|null} frames ahead of the answer deltas (PLAN §4
extension, owner permission 2026-08-26); done.sources, query_log.sources
and the per-turn log line (gains tool_calls=N) report the retrieval
docs + read docs, deduped. The UI shows a "calling tool"
button/label state and one visible .tool-call line per call above the
answer; the lines persist with the chat record and re-render on
reload. chat_stream passes tools through and accumulates streaming
tool_calls deltas into ToolCallPiece (tools=None stays byte-identical).

E2E: deterministic mock tool flow ("use your tools" + <tools> marker:
list -> read first catalog line -> quoted answer) plus the story suite
(marker flow, reload re-render, plain/deflected no-tool regressions).
Docs: .env.example + README (the two tools, the budgets, the SSE tool
frame, the "calling tool" UI state).

probe: turbo tool_calls=supported 2026-08-26 (uv run python -m
scripts.llm_probe --tools — non-streaming + streaming
finish_reason=tool_calls, indexed delta.tool_calls partials)
2026-08-26 22:39:14 -04:00
ducoterra 9efffcb428 feat(viewer): show document summary together with the original (TODO.md L5) 2026-08-26 19:11:15 -04:00
ducoterra 1925bb66a8 feat(sources): admin page to add and remove git sources (TODO.md L4) 2026-08-26 18:42:28 -04:00
ducoterra b2d8696741 feat(ui): one consistent navbar on every page (TODO.md L3) 2026-08-26 15:39:42 -04:00
ducoterra 0a46f07fa8 chore(agent): phase roadmap from TODO.md, 3 phases (34-36) 2026-08-26 09:41:32 -04:00
ducoterra 8fabb7efda perf(ui): cache busting — HTML no-cache + versioned asset URLs (?v=) with immutable 1y asset caching
Phase 33 (story: .agent/user_stories/cache-busting.md).

- app/core/caching.py: asset_version() — git short SHA (a commit is a
  deploy), stable content-hash fallback for non-git checkouts, "dev"
  for a missing static dir; computed once per process. CachingMiddleware
  — the five HTML pages revalidate (no-cache) with ?v=<token> asset refs
  rewritten in flight; /assets/* is public, max-age=31536000, immutable;
  everything else (all /api/*, the SSE chat stream in particular) passes
  through byte-identical.
- tests/e2e/test_cache_busting.py: fresh-Chromium wire assertions —
  document no-cache, versioned CSS/JS request URLs sharing one token,
  immutable asset headers, /api/health baseline headers, SSE chat to
  done (mock LLM).
- README 'Caching / deploys' section + story file.

Also fixed two prod-image defects surfaced by this phase's podman smoke
(the full app would not boot):
- Containerfile: ship the scripts/ package — app/api/sync.py (phase 32)
  imports scripts.git_sync / scripts.import_docs at module level, so the
  container crashed on boot (ModuleNotFoundError: No module named
  'scripts').
- compose.yaml: pass BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET through to
  the app service (:- defaults keep 'podman compose up -d db' working;
  the app's own fail-loud gate still names missing admin auth).

Smoke: podman compose --profile prod up -d on a fresh image + a fresh
Chromium profile — /, /sources.html and /login.html all served
Cache-Control: no-cache; all 8 asset requests versioned with one shared
token (content-hash fallback inside the image — no .git there);
/assets/* immutable for a year.
2026-08-25 22:42:10 -04:00
ducoterra 52136fe307 feat(admin): one-click sources sync — admin-only button triggers git clone/pull + re-import + KB overview refresh with polled live status 2026-08-25 21:39:38 -04:00
ducoterra 0654b304e1 feat(rag): lite-generated KB overview in the system prompt — stored single row, regenerated on import, <knowledge_base> section in HIGH+LOW prompts 2026-08-25 20:22:51 -04:00
ducoterra 572a4190a6 feat(rag): lite-model document summaries — non-markdown docs summarized at import, summary chunk retrieves and resolves to the full source doc 2026-08-25 17:48:37 -04:00
ducoterra 9809482a4b chore(agent): phase roadmap from TODO.md — 4 phases (lite document summaries, KB overview in prompt, admin sync button, cache busting) 2026-08-25 15:43:44 -04:00
ducoterra 3841bd5a30 feat(ui): expose Global Tuning from Chat + Sources headers (admin-only Tuning nav link) 2026-08-25 14:51:16 -04:00
ducoterra 3d044f33a1 feat(rag): git-based import sources — BOR_GIT_SOURCES repos cloned (first run, --depth 1) or pulled (--ff-only) into BOR_SOURCES_DIR/<repo>/ then indexed; --source still wins; a failed sync aborts before importing anything 2026-08-25 14:23:02 -04:00
ducoterra 589e26dbe9 feat(rag): global tuning manager — /tuning.html + PUT /api/steering/{id}: create, edit, list, delete steering notes without a chat 2026-08-25 13:46:32 -04:00
ducoterra fcde1fd37b feat(ui): documents open in an almost-fullscreen modal instead of a new page — same-page overlay on chat + Sources, /document.html kept as the no-JS/direct-link fallback 2026-08-25 13:45:57 -04:00
ducoterra 476aa0e066 chore(agent): phase roadmap from TODO.md — 3 phases (document modal, global tuning, git-based sources) 2026-08-25 10:15:42 -04:00
ducoterra 025f57beb5 fix(ui): background no longer moves — static grid, three glow spots fading in and out on their own slow cycles (owner 2026-08-25) 2026-08-25 09:43:19 -04:00
ducoterra 1e6ae360e0 feat(rag): feed whole matched documents to the LLM — no context truncation (A7 revised) 2026-08-24 23:37:44 -04:00
ducoterra d7a4064616 fix(build): Containerfile builds again — relative module imports, all four pages and shared assets in the image 2026-08-24 22:54:30 -04:00
ducoterra 0adc9b5801 fix(ui): animated background actually animates — grid drift and glow breathe per the phase-08 design 2026-08-24 22:16:41 -04:00
ducoterra 22a6121411 fix(ui): thinking window no longer scrolls — live 320px view pinned to the stream tail 2026-08-24 20:28:12 -04:00
ducoterra 986e704eb1 chore(agent): add phase 24 from TODO.md — whole-document context (no truncation, A7 revised); clear TODO.md 2026-08-24 18:53:24 -04:00
ducoterra 04a7f4c05d fix(ui): thinking window no longer scrolls — live 320px view pinned to the stream tail 2026-08-24 18:24:57 -04:00
ducoterra 76c6a01199 fix(chat): keep the in-flight answer when navigating away mid-turn — partial answer restored on return 2026-08-24 15:53:07 -04:00
ducoterra 824914ca3d chore(agent): add phases 20-23 from TODO.md — four owner-confirmed bug fixes; clear TODO.md 2026-08-24 15:23:49 -04:00
ducoterra 2afc77ee56 feat(ui): shared header — Sign in/Sign out and New Chat on every page; hide the Sources nav link from anonymous users 2026-08-24 12:32:45 -04:00
ducoterra fd7f02ce68 remove tests from pipeline
Build and Push Containers / build-and-push (push) Failing after 52s
2026-08-24 11:02:13 -04:00
ducoterra c37477d3f9 add gitea build and push
Build and Push Containers / test (push) Failing after 14s
Build and Push Containers / build-and-push (push) Skipped
2026-08-24 11:01:06 -04:00
ducoterra 076358db94 uncommit .agent 2026-08-24 10:59:45 -04:00
ducoterra bc0158f858 feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport 2026-08-24 10:53:04 -04:00
ducoterra b16deb2b1d feat(chat): stream model thinking over SSE and show it in a collapsible block 2026-08-24 09:52:27 -04:00
ducoterra cbc263a4b2 feat(auth): single-admin password login (signed cookie) — gate tuning + Sources catalog, keep chat and document viewer public 2026-08-23 19:58:39 -04:00
ducoterra fc0d9a2d5c feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt 2026-08-22 16:44:42 -04:00
ducoterra 19df7df99d feat(ui): persist the chat conversation in localStorage — survives refresh and navigation, with a New chat reset 2026-08-22 15:52:53 -04:00
ducoterra 2485b50af0 fix(ui): document viewer back button returns to the page you came from (chat or sources) 2026-08-22 15:26:43 -04:00
ducoterra 8ca564cd83 fix(ui): uniform header bar height on chat, sources, and the document viewer 2026-08-22 15:14:31 -04:00
ducoterra 0da5275eeb fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off 2026-08-22 11:30:19 -04:00
ducoterra 6ec6181c7b feat(ui): clickable document viewer — open any cited document in the browser from chat chips and the sources table 2026-08-22 02:08:49 -04:00
ducoterra 7e8d14702e feat(rag): hybrid FTS+vector retrieval and multi-format ingestion — name-your-tool questions find the right document 2026-08-22 01:27:02 -04:00
ducoterra 2f738a7f19 feat(ui): dark tech theme — emoji-free chrome, subtle animated CSS background, WCAG AA dark palette 2026-08-21 23:34:36 -04:00
ducoterra 1b29b1cf9d feat(ui): responsive + WCAG AA polish pass across chat and sources — v1 feature complete 2026-08-21 19:44:46 -04:00
ducoterra e5810b0bcf feat(ui): explicit chat state machine — typing indicator, streaming progress, timeout and error recovery 2026-08-21 18:45:27 -04:00
ducoterra 2364e1ee7d feat(ui): onboarding suggestion chips with one-tap submit, keyboard access, and mobile scroll row 2026-08-21 18:18:47 -04:00