Commit Graph
37 Commits
Author SHA1 Message Date
ducoterra 15c1272828 feat(rag): agent document tools — list/read tools with env-tuned budgets, SSE tool events + "calling tool" UI
Grounded chat turns now run the agent loop (app/rag/agent.py) instead
of a bare chat_stream: while the per-turn budgets last
(BOR_AGENT_LIST_CALLS / BOR_AGENT_READ_CALLS, default 1 each) the model
gets list_documents (the indexed catalog, /api/docs order) and
read_document (full text, never truncated — A7-revised contract); once
both budgets are spent the tools key is dropped from the request and
the model must answer. Rejected calls (unknown tool, unknown/missing
path, document already in context, spent budget) consume no budget.
Budgets 0/0 make exactly one tools=None request — byte-identical to
the pre-phase path (budgets-as-kill-switch). Deflected turns keep the
direct chat_stream (A8 unchanged; the LOW prompt never carries the
<tools> section).

SSE contract gains {"type":"tool","name":...,"argument":
"source/path"|null} frames ahead of the answer deltas (PLAN §4
extension, owner permission 2026-08-26); done.sources, query_log.sources
and the per-turn log line (gains tool_calls=N) report the retrieval
docs + read docs, deduped. The UI shows a "calling tool"
button/label state and one visible .tool-call line per call above the
answer; the lines persist with the chat record and re-render on
reload. chat_stream passes tools through and accumulates streaming
tool_calls deltas into ToolCallPiece (tools=None stays byte-identical).

E2E: deterministic mock tool flow ("use your tools" + <tools> marker:
list -> read first catalog line -> quoted answer) plus the story suite
(marker flow, reload re-render, plain/deflected no-tool regressions).
Docs: .env.example + README (the two tools, the budgets, the SSE tool
frame, the "calling tool" UI state).

probe: turbo tool_calls=supported 2026-08-26 (uv run python -m
scripts.llm_probe --tools — non-streaming + streaming
finish_reason=tool_calls, indexed delta.tool_calls partials)
2026-08-26 22:39:14 -04:00
ducoterra 9efffcb428 feat(viewer): show document summary together with the original (TODO.md L5) 2026-08-26 19:11:15 -04:00
ducoterra 1925bb66a8 feat(sources): admin page to add and remove git sources (TODO.md L4) 2026-08-26 18:42:28 -04:00
ducoterra b2d8696741 feat(ui): one consistent navbar on every page (TODO.md L3) 2026-08-26 15:39:42 -04:00
ducoterra 8fabb7efda perf(ui): cache busting — HTML no-cache + versioned asset URLs (?v=) with immutable 1y asset caching
Phase 33 (story: .agent/user_stories/cache-busting.md).

- app/core/caching.py: asset_version() — git short SHA (a commit is a
  deploy), stable content-hash fallback for non-git checkouts, "dev"
  for a missing static dir; computed once per process. CachingMiddleware
  — the five HTML pages revalidate (no-cache) with ?v=<token> asset refs
  rewritten in flight; /assets/* is public, max-age=31536000, immutable;
  everything else (all /api/*, the SSE chat stream in particular) passes
  through byte-identical.
- tests/e2e/test_cache_busting.py: fresh-Chromium wire assertions —
  document no-cache, versioned CSS/JS request URLs sharing one token,
  immutable asset headers, /api/health baseline headers, SSE chat to
  done (mock LLM).
- README 'Caching / deploys' section + story file.

Also fixed two prod-image defects surfaced by this phase's podman smoke
(the full app would not boot):
- Containerfile: ship the scripts/ package — app/api/sync.py (phase 32)
  imports scripts.git_sync / scripts.import_docs at module level, so the
  container crashed on boot (ModuleNotFoundError: No module named
  'scripts').
- compose.yaml: pass BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET through to
  the app service (:- defaults keep 'podman compose up -d db' working;
  the app's own fail-loud gate still names missing admin auth).

Smoke: podman compose --profile prod up -d on a fresh image + a fresh
Chromium profile — /, /sources.html and /login.html all served
Cache-Control: no-cache; all 8 asset requests versioned with one shared
token (content-hash fallback inside the image — no .git there);
/assets/* immutable for a year.
2026-08-25 22:42:10 -04:00
ducoterra 52136fe307 feat(admin): one-click sources sync — admin-only button triggers git clone/pull + re-import + KB overview refresh with polled live status 2026-08-25 21:39:38 -04:00
ducoterra 0654b304e1 feat(rag): lite-generated KB overview in the system prompt — stored single row, regenerated on import, <knowledge_base> section in HIGH+LOW prompts 2026-08-25 20:22:51 -04:00
ducoterra 572a4190a6 feat(rag): lite-model document summaries — non-markdown docs summarized at import, summary chunk retrieves and resolves to the full source doc 2026-08-25 17:48:37 -04:00
ducoterra 3841bd5a30 feat(ui): expose Global Tuning from Chat + Sources headers (admin-only Tuning nav link) 2026-08-25 14:51:16 -04:00
ducoterra 3d044f33a1 feat(rag): git-based import sources — BOR_GIT_SOURCES repos cloned (first run, --depth 1) or pulled (--ff-only) into BOR_SOURCES_DIR/<repo>/ then indexed; --source still wins; a failed sync aborts before importing anything 2026-08-25 14:23:02 -04:00
ducoterra 589e26dbe9 feat(rag): global tuning manager — /tuning.html + PUT /api/steering/{id}: create, edit, list, delete steering notes without a chat 2026-08-25 13:46:32 -04:00
ducoterra fcde1fd37b feat(ui): documents open in an almost-fullscreen modal instead of a new page — same-page overlay on chat + Sources, /document.html kept as the no-JS/direct-link fallback 2026-08-25 13:45:57 -04:00
ducoterra 025f57beb5 fix(ui): background no longer moves — static grid, three glow spots fading in and out on their own slow cycles (owner 2026-08-25) 2026-08-25 09:43:19 -04:00
ducoterra 1e6ae360e0 feat(rag): feed whole matched documents to the LLM — no context truncation (A7 revised) 2026-08-24 23:37:44 -04:00
ducoterra d7a4064616 fix(build): Containerfile builds again — relative module imports, all four pages and shared assets in the image 2026-08-24 22:54:30 -04:00
ducoterra 0adc9b5801 fix(ui): animated background actually animates — grid drift and glow breathe per the phase-08 design 2026-08-24 22:16:41 -04:00
ducoterra 04a7f4c05d fix(ui): thinking window no longer scrolls — live 320px view pinned to the stream tail 2026-08-24 18:24:57 -04:00
ducoterra 76c6a01199 fix(chat): keep the in-flight answer when navigating away mid-turn — partial answer restored on return 2026-08-24 15:53:07 -04:00
ducoterra 2afc77ee56 feat(ui): shared header — Sign in/Sign out and New Chat on every page; hide the Sources nav link from anonymous users 2026-08-24 12:32:45 -04:00
ducoterra bc0158f858 feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport 2026-08-24 10:53:04 -04:00
ducoterra b16deb2b1d feat(chat): stream model thinking over SSE and show it in a collapsible block 2026-08-24 09:52:27 -04:00
ducoterra cbc263a4b2 feat(auth): single-admin password login (signed cookie) — gate tuning + Sources catalog, keep chat and document viewer public 2026-08-23 19:58:39 -04:00
ducoterra fc0d9a2d5c feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt 2026-08-22 16:44:42 -04:00
ducoterra 19df7df99d feat(ui): persist the chat conversation in localStorage — survives refresh and navigation, with a New chat reset 2026-08-22 15:52:53 -04:00
ducoterra 2485b50af0 fix(ui): document viewer back button returns to the page you came from (chat or sources) 2026-08-22 15:26:43 -04:00
ducoterra 8ca564cd83 fix(ui): uniform header bar height on chat, sources, and the document viewer 2026-08-22 15:14:31 -04:00
ducoterra 0da5275eeb fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off 2026-08-22 11:30:19 -04:00
ducoterra 6ec6181c7b feat(ui): clickable document viewer — open any cited document in the browser from chat chips and the sources table 2026-08-22 02:08:49 -04:00
ducoterra 7e8d14702e feat(rag): hybrid FTS+vector retrieval and multi-format ingestion — name-your-tool questions find the right document 2026-08-22 01:27:02 -04:00
ducoterra 2f738a7f19 feat(ui): dark tech theme — emoji-free chrome, subtle animated CSS background, WCAG AA dark palette 2026-08-21 23:34:36 -04:00
ducoterra 1b29b1cf9d feat(ui): responsive + WCAG AA polish pass across chat and sources — v1 feature complete 2026-08-21 19:44:46 -04:00
ducoterra e5810b0bcf feat(ui): explicit chat state machine — typing indicator, streaming progress, timeout and error recovery 2026-08-21 18:45:27 -04:00
ducoterra 2364e1ee7d feat(ui): onboarding suggestion chips with one-tap submit, keyboard access, and mobile scroll row 2026-08-21 18:18:47 -04:00
ducoterra cbf8e39e63 feat(rag): honest deflection gate with amber UI state and alternative-question chips 2026-08-21 17:50:33 -04:00
ducoterra 396e4d47fb feat(rag): stream grounded RAG answers over SSE with source citations
Phase 03 (Story: Chat RAG Answer — happy path):
- app/rag/retriever.py: top-k cosine search + parent-doc selection with
  per-doc dedupe and BOR_MAX_CONTEXT_CHARS cap ([…truncated…] marker)
- app/rag/prompts.py: locked persona + HIGH/DEFLECT prompt builders
- app/rag/llm.py: LLMError + chat_stream (turbo, temp 0.4, max 700, stream)
- app/api/chat.py: POST /api/chat SSE — delta* then done{deflected,
  sources, suggestions}; query_log row + PLAN §9 per-turn log line;
  structured error event on mid-stream failure, JSON 503 when DB down
- frontend: SSE reader, live bubble streaming, source chips -> /sources.html,
  red role=alert banner, Send button state that always recovers
- fix(scaffold): [hidden] { display: none !important } — .kb-banner's
  display:flex was overriding the hidden attribute (banner always visible)
- tests: unit (retriever/prompts/sse/llm) + integration (real Postgres RAG
  turn, query_log, error + 503 paths, mid-turn failures) + Playwright story
  suite (grounded answer, log row, raw SSE shape); smoke placeholder test
  replaced with the real never-stale-button contract
2026-08-21 17:17:02 -04:00
ducoterra 99c48cbe06 feat(rag): index markdown KB — chunker, embed client, delta importer, Sources page
Phase 02 (story: import documents):

- fence-aware markdown chunker (heading sections, 200-char overlap,
  heading anchor on every chunk, 1200-char hard cap, fence blocks
  kept atomic and split under the cap)
- LLMClient over aipi (LiteLLM) reusing the openai client's httpx
  transport to send a clean {model, input} payload — the openai SDK
  injects encoding_format, which aipi's openai_like group rejects;
  token-budget batching + halving retry for the endpoint's
  ~1024-token per-request input cap
- two-phase per-file upsert importer: sha256 delta (unchanged skip),
  atomic commit, A9 exclusion walk, per-source prune, per-file error
  tolerance (rollback + log + continue, non-zero CLI exit), adaptive
  re-chunk at half target for URL-dense files the endpoint rejects
- scripts/import_docs CLI (repeatable --source, --prune, --limit,
  defaults ~/Homelab + ~/Deployments)
- GET /api/docs with per-doc chunk counts; Sources page wired to the
  real endpoint (stat cards, full-width a11y table, designed empty
  state, DOM-built rows — no innerHTML)
- tests: 63 passed (chunker/llm/importer units, docs API + importer
  integration), story E2E 3/3 (real endpoints, in-thread import);
  app/ coverage 98%
- real KB imported: 672 docs / 8969 chunks in ~3m, idempotent
  re-run (672 unchanged, 0 batches)
- harness: .agent/validate.sh now gates through uv (pytest +
  coverage >90% + ruff + pyright) instead of system python3
2026-08-21 16:24:45 -04:00
ducoterra 022da8e2bc feat: scaffold Brain of Reese — FastAPI RAG chat over Postgres 17 + pgvector
Foundation (phase 01, verified):
- FastAPI app: /api/health, /api/suggestions, /api/chat (placeholder),
  static frontend served locally (no CDN)
- Postgres 17 + pgvector via db/Containerfile + compose.yaml
  (podman compose up -d db), Alembic initial migration (documents,
  chunks with vector(768), query_log)
- LLM client targeting https://aipi.reeseapps.com/v1 (turbo/embed);
  scripts/llm_probe.py verified models + 768-dim embeddings live
- Conditional debugpy: imported only when DEBUGPY=1 (attach on demand,
  :5678); logging config for clean single-line logs
- Frontend shell: mobile-first chat + Sources pages, tokens, a11y baselines
- Tests: 24 unit+integration (99% coverage on app/), ruff + pyright clean,
  Playwright smoke E2E (3 tests) against a deterministic mock LLM
- Planning: .agent/PLAN.md (architecture + LOCKED decisions), AGENTS.md,
  6 user stories, 7 phase files (one story / one phase / one Playwright
  suite each)
2026-08-21 13:42:21 -04:00