brain-of-reese/README.md

🧠 Brain of Reese

A chippy, honest RAG chatbot over your homelab documentation — git repositories, local directories, and uploaded archives registered on the admin Sources page. Ask it anything about your notes — markdown, YAML, JSON, Python, plain text, and more — and it retrieves the relevant chunks with hybrid search (pgvector cosine ∪ Postgres full-text search, fused with Reciprocal Rank Fusion), feeds the whole relevant document to a self-hosted LLM (turbo via https://aipi.reeseapps.com/v1), and streams a grounded answer back.

If it doesn't have notes for your question, it admits it — and offers 2-3 concrete alternative questions about things it does have notes on.

Image support is opt-in (BOR_IMAGES): standalone images become retrievable documents, and you can attach an image to a question — see Image support.

Updated your notes? Re-run the import — it's idempotent and only re-embeds what changed:

uv run python -m scripts.import_docs --prune

See Updating the documents for details.

---

Quick Start

# 1. Dependencies
uv sync

# 2. Configure (copy and edit)
cp .env.example .env
# Set BOR_LLM_API_KEY — your aipi key (falls back to $AIPI_KEY)
# Set BOR_ADMIN_PASSWORD and BOR_SESSION_SECRET — both are REQUIRED:
# the app refuses to start while either is empty (see Admin & Sign-in)

# 3. Start the database
podman compose up -d db
podman compose ps          # wait until "healthy"

# 4. Apply migrations & import
uv run alembic upgrade head
uv run python -m scripts.import_docs

# 5. Run the app
uv run uvicorn app.main:app --reload
# → http://localhost:8000  (chat)

Stack: FastAPI · Pydantic v2 · SQLAlchemy 2 · Alembic · pgvector · vanilla HTML/CSS/JS (no CDN) · Playwright E2E

---

Pages at a glance

The app is a single-page application — all views load from / with a client-side router. Direct URLs deep-link to the matching view.

Page	URL	Who can see it
Chat	/	Everyone (token gate for non-shared)
RAG / Knowledge base	/sources.html	Admin only
Git sources	/git-sources.html	Admin only
Tuning	/tuning.html	Admin only
Saved chats	/history.html	Admin only
Access tokens	/tokens.html	Admin only
Theme	/theme.html	Admin only
Settings	/settings.html	Admin only
Document viewer	/document.html?source=X&path=Y	Token users & admins
Login	/login.html	Everyone
Shared chat	/shared/<token>	Everyone (anonymous)
Edit doc	/doc-edit.html	Admin only

Chat (/)

Ask questions; answers stream in with source chips that cite the exact documents used (a cited image document gets no chip — its compact figure is the citation, phase 130). Clicking a chip or figure opens the document in an almost-fullscreen modal on the same page (no new tab); the browser Back button closes the modal — instead of navigating the view behind it — and Forward re-opens the same document (phase 144). The modal's Full page link opens the dedicated /document.html viewer in a new tab carrying its origin: a document opened from Chat returns through the viewer's back link to Chat, one opened from Sources returns to Sources (phase 153, feeding the phase-13 back contract on every path). The onboarding suggestion chips follow the opening questions of the 3 most recent saved chats (each chat's first question — a follow-up never chips, phase 103) — on a fresh deployment they seed from BOR_SUGGESTIONS. A long opening question caps at 90 displayed characters with a … (phase 157) — the full text stays in the hover tooltip, and clicking the chip submits the full question.

No empty answer box mid-turn (phase 166): the bubble that will hold the answer is created on the turn's first thinking/tool frame — it is the DOM anchor the Reasoning container inserts itself above — but it stays hidden until the first answer token lands in it, so an in-flight turn shows the typing indicator and the Reasoning bubble only, never a padded empty rectangle (nothing replaces the box: the existing loading feedback is the turn's visible state). The same rule covers a restored or shared answer whose persisted text is empty, which paints no box either.

Markdown links render as links (phase 180, TODO.md L4): an inline [text](url) in an answer, the thinking block, a saved/shared chat, the document viewer or the document modal — every surface that goes through the one shared renderer — becomes a real <a class="md-link">. Only allowlisted URLs become live elements: http(s):// (which also carries target="_blank" rel="noopener noreferrer"), mailto:, /-rooted and .//../ relative paths, and #fragment links — the last three stay in the same tab and carry no target. Anything else (javascript:, data:, vbscript:, file:, blob:) renders as the plain [text](url) text it is today, never an element. The URL is used verbatim from the already-escaped text, so a & in a query string stays & and nothing is ever unescaped (escape-first is the security boundary); a URL containing a space or a literal ) is a documented miss. Styling is one .md-link rule for every surface: underlined (a link never relies on colour alone) in the theme's AA-safe brand text colour — 9.9:1 on the answer bubble, 8.3:1 on a deflected one — and the user bubble's own ink on the brand fill (5.2:1), where the underline is what marks the link; Tab focus is the app's global 3px --brand ring.

Markdown images render as images (phase 180, the in-answer half of TODO.md L3): an inline ![alt](src) — and a linked image [![alt](src)](url), which renders as that image wrapped in the link — becomes a lazy <img class="md-img" loading="lazy"> on every surface that shares the renderer, so a diagram referenced inside an answer or a document paints instead of printing its reference text. alt is always emitted (empty allowed for a decorative image). The image allowlist is its own function and accepts /-rooted served paths (/api/documents/<id>/image, /assets/…), relative paths whose first segment is at most two characters (./x, ../x, d/x — a longer first segment like images/x.png is a documented miss), http(s):// and data:image/… — the data: case needs no CSP change because the app's own img-src 'self' data: already permits it. A javascript:, file:, blob: or non-image data: src stays visible as the inert ![alt](src) text it is today, never an element, and a broken served path degrades to the browser's alt text (the renderer emits an HTML string, so it carries no inline onerror handler — the JS fallbacks live in document.js/app.js). .md-img carries the app's bordered-image card treatment and is capped to its container — max-width: 100%, height: auto and object-fit: contain — so a wide screenshot never produces a horizontal scrollbar at 375px or at 1440px. An SVG document that declares no width or height is sized by phase 181's .doc-image-img--vector class in the document viewer instead (see "Image documents"); an inline answer image keeps the container cap above.

The heading ladder reaches six levels (phase 180, TODO.md L5): a #### x line in an answer, the thinking block, a saved/shared chat, the document viewer or the modal renders <h5>x</h5>, and ##### x / ###### x render <h6>x</h6> (HTML stops at h6). The levels above are UNCHANGED and deliberately flattened — #/## both render <h3>, ### renders <h4> — so an answer's deep sections gain the two rungs below ### without re-ranking what already rendered. Like every other inline rule the ladder needs a space after its hashes and sits at column 0 of its line, so a mid-sentence #### and an indented #### x stay plain text, and a #### inside a fenced block stays code. The two new rungs are sized into the existing type scale — h3 1.15rem → h4 1rem → h5 0.95rem → h6 0.88rem, margins tightening with depth — so a deep heading neither collapses into body text (the UA default is 0.83em/0.67em) nor out-shouts the answer around it, in the bubble, the thinking block and the viewer alike.

The renderer has one more pass, and it is opt-in (phase 182): renderMarkdown(md) is now renderMarkdown(md, opts), and the ONLY truthy option is { verbatim: true }, passed by exactly the FOUR answer renderers — the live per-delta write, the done-time re-render, the chat page's restore and the shared page's record renderer. Every other call site omits it, so the feature is inert elsewhere. With it on, a ```bor <source/path> <range> locator fence is replaced — fence line, body and closer included — by the pending verbatim panel, so the model's re-typed copy of a file is never painted (see Verbatim text display). Pass order with the option on is verbatim → fences → tables → escape+inline → blockquotes → paragraphs → restore; with it off — the document viewer, the thinking runs, and every question bubble on both pages — the output is byte-identical to pre-phase, so a document that legitimately contains a ```bor fence still renders its own text.

The document's meta row (the chat modal and the standalone viewer at /document.html) carries a Copy path button: one click copies the document's source/path (the exact form the read tool, the chips, and the RAG table use) to the clipboard; on a non-secure (http) homelab origin the browser rejects the clipboard, so the click instead offers the full path in an inline fallback field that selects itself on focus — click or Tab, then copy (phase 153).

Edit the newest question (phase 159, re-housed in phase 170): the latest user bubble carries an Edit button that opens the editor inside the bubble — the editor inherits the bubble's own box, background, and padding, so it keeps the bubble's dimensions (the bubble stays visible; the rendered question is replaced by the form). A labelled textarea is prefilled with the question (the 4,000-character cap is enforced with maxlength; Ctrl/⌘+Enter submits, Esc / Cancel restore the rendered question byte-identical, and a whitespace-only question is rejected inline). Send rewrites the question in the chat and the saved chat in place, drops every turn that followed it, and re-asks the trimmed question — a fresh answer streams in the old answer's place (the question is never duplicated; an attachment on the question survives the edit and the re-ask stays text-only). A turn that failed with "Question too long — trim it and re-ask" (or a mistyped question) is fixed by trimming it in the editor instead of retyping it. Earlier bubbles never carry the button (last-question-only scope, owner-locked).

When a long answer spends its research budget (phase 160): the answer's meta row carries a Continue researching button (row order Tune → Retry → Continue → Save-as-doc; not admin-gated — chat is public, so any visitor can keep a long research session going). Each tool result tells the model how many tool calls remain; calls attempted after the budget is spent are rejected (not executed, not shown) instead of being dumped into the chat, and the turn ends with a final no-tools reply — the model is coached to answer from the results it already has and to tell you it can keep going, and a deterministic "budget spent" fallback line stands in if that reply carries no visible content. Clicking the button sends one plain continue turn — the budget is per-turn, so the new turn has a whole new round budget, and nothing server-side knows or cares that this is a continuation (A10, stateless). The button is inert while a turn is streaming and single-shot (a stale button is removed when the new turn starts); the flag is persisted on the saved record, so the button restores on a refresh — but a shared chat never shows it.

RAG / Knowledge base (/sources.html)

The indexed document list. The Path column opens each document in the same almost-fullscreen modal (and /document.html renders it through the one shared core). Content renders by kind (phase 165): markdown becomes a rendered document, an image's description or a plain-text (.txt) body renders as wrapping prose in the 72rem reading column, and code and code-ish formats keep the monospace block with a horizontal scroller (long lines never wrap in code — locked; the themed modal scrollbar of phase 116 keeps applying to whatever still scrolls). Document rows carry a Created date and folder rows an Updated date (phase 106) — sourced at sync time (git commit dates for git sources, restored metadata for archive uploads, file mtime otherwise; unresolvable or future dates fall back to today) — and the admin can correct a document's date from the document viewer. The date also feeds the small recency boost in retrieval ranking (BOR_RECENCY_BOOST). Admin-only — anonymous visitors see a sign-in gate instead.

The tree payload is bounded (phase 167): GET /api/docs/tree builds only depth levels (default 1 — the sources plus one level, the page's first paint; depth=2..8 builds deeper, anything else is a 422) instead of the old ~2 MB full recursive dump. A node whose children were not built carries "truncated": true and ships with its recursive document count, updated date, description and pending marker (children: []); untruncated nodes are byte-for-byte what they were before. The three stat cards read the server-computed top-level totals (documents, chunks, last_indexed) so they stay honest whatever depth was requested. Responses also ship gzipped (see Caching & deploys). Deeper levels load on demand through GET /api/docs/tree/node?source=X&path=Y&depth=N (admin-only, phase 167): one node — the source root for path="" — answered in exactly the shape the full tree produces for it (unknown source or folder → 404 unknown node), built from the same single catalogue query. The RAG view fetches each drilled level once and caches it (re-opening a folder costs no network), showing one Loading… row while a level arrives and a Couldn't load … — Retry line on failure with the current level staying rendered; a folder that vanished since it was listed (the node 404) resets the view to the top level. The breadcrumb, the bounded scroll panel, and the description editors behave exactly as before. All three catalogue reads (/api/docs, /api/docs/tree, /api/docs/tree/node) also carry a weak ETag + Cache-Control: private, must-revalidate, so reloading a page you just looked at costs a bodiless 304 (see Caching & deploys).

Git sources (/git-sources.html)

The admin-managed source registry: add or remove git repositories, upload source archives (.tar, .tar.gz, .tgz, .zip), and register local directories — all in one table with a kind discriminator. No .env editing, no restart. The Sync sources button — which lives on the RAG page (/sources.html), not here — clones/pulls/walks every source and re-imports in one click.

Tuning (/tuning.html)

Set global instructions that are injected into the system prompt of every chat turn. Add notes like "be more concise" or "assume I'm on NixOS". List, edit, or remove them here — no conversation required.

Saved chats (/history.html)

Every conversation is saved automatically. Click a title to return to that chat. Admin-only view. The list is paginated server-side (25 rows per page) — with more than 25, a Prev / “Page N of M” / Next pager appears under the table to walk it, and Refresh re-fetches the current page. GET /api/chats carries the same contract: ?limit= (default 25, clamped 1–100) / ?offset= and a full total on every response.

Access tokens (/tokens.html)

Generate tokens and hand them out — a token opens chat, the answers, and the documents they cite. Shared chats stay open to everyone.

Settings (/settings.html)

Edit any setting that today lives in .env — every BOR_* setting, grouped and type-correct — from the browser. A value you Save is stored in the database and wins over the environment variable from that instant on, with no restart (boot-wired settings — the pool, the session middleware, the log level — carry a restart required badge and apply on the next start); Reset hands control back to the environment, and Clear all overrides empties the override table. .env is never written: the environment stays the base, the database row wins. Secrets are write-only (you set a value, never read one back); a value that fails validation is rejected before anything is stored. Admin-only — anonymous visitors see a sign-in gate.

The overlay can be switched off for the whole process with BOR_SETTINGS_DB=0 (env-only; the panel stays visible and says so).

Shared chat (/shared/<token>)

A read-only snapshot of any conversation, shareable via a link. Open to everyone — no login or token required. The page still ships no composer, no Save/Share/Tune/Retry and no chips-as-buttons; the ONE interactive exception is phase 182's answer code: a shared verbatim panel and the answer's ordinary code blocks carry a Copy button and the panel a plain /document.html?source=…&path=… link, because a page that shows a slice of a file has to let the reader copy it and check it. Both come from the same verbatim.js the chat page calls (so a shared turn renders identically to the owner's), and a guest who follows the viewer link lands on the sign-in page — the documents API is admin-gated.

Conversation payload limits

A conversation has one shape, and it carries the same static bounds whether it rides back to the model as history on the next turn or is written to a saved chat — the two paths mirror each other on purpose, so saving a conversation can never be stricter than continuing it (phase 164):

Bound	Value	What it admits
One message (text, thinking)	600 000 chars	a full-length answer at the largest answer cap — BOR_MAX_OUTPUT_TOKENS at 131 072 tokens ≈ 524 300 chars — as history and as a saved record, with margin (the old 32 000 ceiling broke even at the 32 768-token default)
Turns sent as history	500	the whole browser conversation (the client sends everything it holds, minus the question being asked)
Messages per saved chat	500	the same conversation, saved
One verbatim block (answer_text[].text)	600 000 chars	the same answer-text cap (phase 182) — comfortably above BOR_SHOW_TEXT_MAX_CHARS, so raising that budget never hits the wire limit
Verbatim blocks per saved message	50	the tools / segments number (phase 182) — far past the BOR_SHOW_TEXT_MAX_BLOCKS default of 5
Total chars per saved chat	8 000 000	~15 full-length answers; the one bound on the anonymous save surface, which stays bounded on purpose — and it counts the verbatim blocks' bytes too (phase 182), so the display blocks cannot become a second, unbounded byte channel
One question (message)	4 000 chars	unchanged; the composer counts it live

These are sanity ceilings, not the prompt's budget: the real gate is BOR_HISTORY_MAX_TURNS / BOR_HISTORY_MAX_CHARS, and a turn that does not fit the budget is dropped whole from later prompts (never cut mid-answer). So a very long answer stops being context for the next question — the follow-up still works, just with a trimmed window — raise BOR_HISTORY_MAX_CHARS if you want it kept.

A failed turn states what actually happened (phase 164): when the app answers but rejects the send before the model runs (an HTTP 4xx/5xx — a validation rejection, an outage page, a proxy error), the bubble reads "I couldn't take that — my API rejected the request. Use Retry to ask again." — not the connection line — and the banner adds that nothing was lost but that send. A genuine transport failure and a model-side error keep "My answer didn't make it — the connection dropped." Either way the exact reason rides verbatim on the bubble's Failed note (API validation details serialize as their msg text — never [object Object], and rejected request content is never shown), persists with the record, and lands Retry + Edit.

Edit doc (/doc-edit.html)

Review and adjust an AI-generated answer before committing it to the docs repository. A draft that should never be pushed can be Discarded from the same page (permanent; admin-only). Admin-only flow page.

---

Admin & Sign-in

Brain of Reese has exactly one account: the admin (you). Signing in unlocks the full RAG catalog (/sources.html), the answer-tuning controls, and token management. Shared chats are the only content that stays open to anonymous visitors.

Setup (one-time)

python -c 'import secrets;print(secrets.token_hex(32))'   # → paste into .env

BOR_ADMIN_PASSWORD=your-password        # plaintext — homelab scope, by design
BOR_SESSION_SECRET=<the hex from above> # signs the session cookie

Fail-loud: while either variable is empty the app refuses to start, naming the missing one(s).

How authentication works

Endpoint	Purpose
POST /api/login	Password → signed bor_session cookie
POST /api/logout	Clears session cookie
GET /api/whoami	`{"authenticated": bool, "role": "admin"
POST /api/token-auth	Token (bor_…) → same cookie, role user

The cookie uses same_site="lax", https_only off — no HTTPS enforcement (homelab HTTP; the cookie is single-admin convenience, not a cloud boundary). Max age is BOR_SESSION_MAX_AGE (default 43200 = 12 h).

Access tokens

The admin can hand out access without sharing the admin password.

1. Generate: sign in → Tokens in the navbar. Type a label and hit Generate — a bor_ + 32-hex token appears in the shown once block. Copy it now: only its SHA-256 hash is stored.
2. Use: the holder enters the token at the in-app gate. The browser caches it in localStorage["bor.token"] for silent re-auth on reloads.
3. Revoke: click Revoke on the token's row. Revocation is immediate.

A token user can chat and open documents — nothing else. The RAG catalog, git sources, tuning, and history stay admin-only.

---

Using the App

Updating the documents

The knowledge base is refreshed by re-running the import. It is idempotent and delta-based: each file's RAW BYTES are hashed (sha256, streamed) and checked against the last index's stamp before any extraction or vision work — an unchanged file costs one hash + one row lookup (no re-extract, no vision, no re-embed), while a changed file re-indexes fully. That row lookup reads metadata only (phase 185: Document.content is deferred), so a full-resync walk's cost is metadata-bound instead of corpus-bound; the rare unchanged file whose summary is still NULL re-reads its stored content — one query, one time, by design — because that backfill genuinely needs the text. The one exception is a PDF row that still claims a page strip: it is re-gated from its current bytes (no vision, no LLM — see "The strip repairs itself" below). A refresh after a normal editing session takes seconds — pictorial PDFs included (their expensive first-index vision burns only once per file version):

# After editing/adding/removing notes:
uv run python -m scripts.import_docs                 # re-index what changed
uv run python -m scripts.import_docs --prune         # also drop deleted files

# Point it at extra directories (repeatable):
uv run python -m scripts.import_docs --source ~/SomeOtherDocs

With git-based sources each run first pulls the latest commits of your repos, so this same command is the whole update loop: commit → re-run.

Then check the RAG page — the knowledge-base catalog at http://localhost:8000/sources.html (the navbar labels it RAG; the separate Sources page, /git-sources.html, is the source registry): the documents / chunks counters and last indexed timestamp should reflect the new files.

• --prune removes a pruned file completely — its documents row and its chunks rows (the embeddings) go together, the latter through the database's own ON DELETE CASCADE, so the cost of a mass prune is bounded by documents, not chunks (see A mass prune costs documents, not chunks under Sync from the UI). The CLI run follows the same two phase-184 knobs as the UI sync (BOR_SYNC_SESSION_RESET_FILES, BOR_SYNC_PRUNE_BATCH): it loads only the prune's six columns and commits every Nth delete (0 on either knob restores the pre-phase-184 run).
• The import prints one line per file and ends with a greppable summary, so it is safe to run from cron or after every commit.
• Indexed formats: md, markdown, txt, yaml, yml, json, py, the Podman quadlet family (container, network, volume, image, pod, kube, swap, os, endpoint), j2 Jinja templates — and, opt-in, pdf (add the pdf token to BOR_IMPORT_EXTENSIONS; see PDF documents).
• Hidden files and common cache dirs (.venv, node_modules, .git, __pycache__, etc.) are skipped — unless the per-source Index hidden folders checkbox on the Git sources page is on for that source (phase 105): then dot-prefixed folders/files with an importable extension are indexed like visible ones, and switching it off prunes them on the next sync.
• Non-markdown files get format-aware chunking (YAML keys, JSON keys, Python classes/defs).
• With BOR_IMAGES=1, standalone images (png, jpg, jpeg, webp, gif, bmp, svg) are indexed as documents too — svg is the one vector member, indexed from its own markup text with no vision call and previewed as a graphic (phase 181) — see Image support.
• A run that changed the KB also regenerates the KB overview and the folder summaries (the drill-down catalog's stored per-folder descriptions) with the lite model — best-effort: a summary failure never fails the import. The stored overview ends with a deterministic Sources: <name> (<k> doc|docs), … line — every source with its exact document count, catalog order, computed without a model call — so no source (not even a 1-doc one) is invisible to <knowledge_base> (phase 162); the LLMError/zero-doc paths leave the previous row (tail included) untouched. Because the sync only rewrites the folders a change touches, a one-shot uv run python -m scripts.regen_folder_summaries refreshes EVERY stored folder summary in one run (full mode: every candidate folder is regenerated, owner-edited rows are kept, one lite call per folder, one cron-safe summary line; exit 0, non-zero with a named stderr line on a boot/DB failure).

Git-based sources

Rather than pointing the import at local folders, point it at git repositories. The admin Git sources page (/git-sources.html) is the primary management surface — add or remove repositories there and the list is stored in Postgres.

# .env — the fallback list (fresh setups, or until the admin page
# stores a source; the page becomes the source of truth)
BOR_GIT_SOURCES=https://git.reeseapps.com/reese/homelab.git,git@github.com:reese/deployments.git
BOR_SOURCES_DIR=~/bor-sources   # default; each repo lands in <dir>/<repo-name>/

• Public repos need no setup: auth is whatever the machine supplies — HTTPS via the OS credential helper, or SSH via your key. The API accepts https://, http://, ssh://, git@, and file:// URLs (phase 135: a file:// local-transport repo — e.g. on a mounted volume — is a valid source too).
• Private repos: the add/edit form on the Git sources page takes an optional masked token field (phase 121). The token is stored in a dedicated DB column and injected into the clone URL only at sync/clone time — it is never returned by any API response or shown in the UI (legacy rows that already embed user:token@ in the stored URL keep working — the token is sanitized on output).
• A git source can pin the branch it mirrors (phase 135): the add/edit form's optional branch field (or the API's branch) stores the branch in the git_sources.branch column — a fresh clone checks that branch out, an existing checkout tracks it (git fetch origin <branch> + git merge --ff-only FETCH_HEAD), and a checkout sitting on a different branch fails the sync loudly with a remove-and- re-add message (never a silent switch). Rows without a branch — every pre-existing row and the env-fallback rows — track the checkout's default branch (byte-identical clone behavior).
• Every run clones (first time, full history) or brings an existing checkout to the head of the tracked branch (phase 136: git fetch origin <branch> + git merge --ff-only FETCH_HEAD) and then indexes the checkout. If the remote was force-pushed — a fast-forward is impossible — the checkout is reset hard to that head with one loud log line instead of failing the sync: the checkout is a cache of the remote head, and the KB is rebuilt from it on the same run, so a force push is a logged, self-healing event. A genuinely broken fetch (dead network, a branch deleted upstream) still fails the sync loudly. A legacy shallow checkout is unshallowed once on its first post-phase-107 sync (git fetch --unshallow — the one-time self-heal) so commit dates stay real.
• A failed sync aborts the run — no partial junk.

Archive upload sources

Upload a .tar, .tar.gz, .tgz, or .zip archive to make it a source. The form on the Git sources page — and the POST /api/git-sources/upload route behind it — accepts archives and unpacks + registers them (no scan — the upload indexes nothing; the Sync sources button on the RAG page (/sources.html) performs the scan, which leaves room to edit the new source's ignore list first).

• Re-uploading the same filename replaces the source in place (no second folder, no duplicate row).
• After the 202 the form settles into the ready-for-sync line "Uploaded () — press Sync sources to import it." — the completion stamp (phase 154: the same date-qualified freshness rule as the sync badge — HH:MM on the same local day, M/D, h:MM AM/PM otherwise, M/D/YYYY across years) rides the line on EVERY render path, so a line that outlives its upload is self-dating: after a reload the source name is gone (it was page-local) but the stamp survives, coming from the server payload's finished_at.
• Zip-bomb guard: absolute member paths, .. traversal, and symlinks escaping the unpack folder are rejected.
• Archives unpack under BOR_UPLOAD_DIR (default ~/bor-sources/uploads).
• BOR_UPLOAD_MAX_MB (default 512) caps both compressed and extracted size.

Local directory sources

A plain directory can be a first-class source too. It shares the git sources' one table (the git_sources registry with a kind discriminator: git | local), one admin page, and one Sync button.

• Register via API: POST /api/git-sources with {"kind": "local", "path": …}.
• Sync walks it directly — no clone, no checkout copy.
• The directory is re-verified to exist at sync time; a missing directory fails the run loudly.

Sync from the UI

The Sync sources button on the RAG page (/sources.html, admin only) runs the whole source refresh in one click:

1. clone/pull + walk every configured source — git repos and local directories, including uploaded archives.
2. re-import with prune — files deleted upstream leave the index.
3. regenerate the KB overview (the <knowledge_base> outline every chat turn injects) — only when the import changed the knowledge base.
4. refresh the folder summaries (the drill-down catalog's stored per-folder descriptions) — a changed knowledge base rewrites ONLY the folders the changed files touch (each file's folder chain up to its source root — phase 127: one changed file re-stamps a handful of rows, not the whole table); an unchanged re-sync instead gap-fills any missing rows (only_missing).

• States: while polling GET /api/sync/status every 2 s, the button becomes the spinner (icon only — its label parts are hidden while a run is in flight, and its accessible name is the full live label), and the full live detail — "Syncing… (n/m)" during the import phase (the live file, its (n/m) file position, and — while a file's embed batch is in flight — the pinned · chunk n/m embed position, batch granularity: the first chunk of the in-flight batch over the file's total content chunks; and — while a PDF's images are being described — the · image n/m vision position: the 1-based image about to be described over the file's total images, ONE continuous counter across the PDF's three vision spans, so a long image pass visibly climbs instead of sitting frozen on the last chunk position) — shows in a text box under the button row, wrapping (never clipped); the same untruncated label rides the button title and the box, which is the SOLE aria-live region for the sync area (role="status", polite — phase 150 folded the pre-phase result-line announcer into the box). At settle the box STAYS visible to carry the result line — the counts line ("N added · …") after a success, "Sync cancelled" after a cancel, each trailed by the run's peak memory (" · peak 1936 MB", phase 187; nothing at all when the reading is null) — and the button leaves the running state (the "Synced " pill after a success — the date-qualified freshness stamp (phase 154): HH:MM on the same local day, M/D, h:MM AM/PM otherwise (M/D/YYYY across years — a badge that tells the owner when), the bare "Synced" when the stamp is missing — the retry-ready "Sync sources" pill after a failure); the box hides only when its text is empty (a failed sync — the banner + modal carry the error — and the true idle). No client-side timeout — a clone + embed can legitimately take minutes.
• One sync at a time: a second trigger while a run is in flight gets a 409.
• A long clone never freezes the server (phase 179). The run's git steps (clone_or_pull / file_commit_dates, the blocking subprocess calls) execute in a worker thread (asyncio.to_thread), so the event loop keeps serving the whole HTTP surface for their duration — every page, GET /api/health, and above all the 2 s GET /api/sync/status poll that watches the run. Before this a big first clone stalled the loop thread and the entire app looked dead until git returned.
• A mass prune costs documents, not chunks (phase 183). A pruned file leaves the index completely — its documents row AND its chunks rows (every embedding) — but the chunk rows go in the database, through chunks_document_id_fkey's ON DELETE CASCADE (alembic 0001), because Document.chunks is declared passive_deletes=True. So a prune of 3,000 documents issues batched DELETE FROM documents statements and zero SELECTs of the 768-dim vectors it is about to remove: the memory a prune costs is bounded by the document count, not by chunks × ~24.8 KiB of hydrated embedding. Before this, session.delete(doc) loaded every child collection first, and the 2026-10-15 sync's 3,008-document prune materialised ~72k embeddings (~720 MB) on a 3,904 MB VM with no swap. The same holds for removing a source on the Git sources page (the other mass-delete surface). Pinned by tests/integration/test_prune_cascade_statements.py — the statement count, the zero-orphan-chunk-rows contract, and a live-FK drift guard that fails the suite if the cascade clause ever disappears; the owner flow itself (delete upstream → re-sync → exactly those documents and chunks gone) is pinned by tests/e2e/test_prune_cascade_cleanup.py.
• A sync's memory is bounded per RUN, not per knowledge base (phase 184). One import run holds one session, and expire_on_commit=False (app/db.py) means a commit releases nothing — so before this phase the run kept every row it touched (each unchanged document with its full ~15 KB content) until it finished, and the prune read every document of every pruned source whole, deletes held in ONE transaction. Now the walk releases its session every BOR_SYNC_SESSION_RESET_FILES files (plus once per source), and the prune (_prune) reads only the SIX columns its guards and its copy cleanup need (id, source, path, is_image, image_path, page_count — never content) ordered by id, committing every BOR_SYNC_PRUNE_BATCH deletes. Where the operator sets the ceiling: those two BOR_-prefixed settings (defaults 200 / 500, live on the next sync without a restart, 0 on either one restores the exact pre-phase-184 run as the no-redeploy rollback). Counts, log lines, changed_paths, the phase-122/126 prune guards and the database are byte-identical either way — pinned by tests/integration/test_import_session_retention.py (the identity-map bound, the no-content prune SELECT with no per-row lazy load, the 3-commits-vs-1 cadence, both prune guards on light rows) and end to end through the RAG page (/sources.html) by tests/e2e/test_sync_session_reset.py — two module apps on their own ports (E2E_APP_PORT_SYNCRESET / E2E_APP_PORT_SYNCRESET_ZERO), one per cadence: the ten-file walk lands every document exactly once with the totals the stat cards show, the re-sync is all-unchanged with an unchanged chunk total, a five-document prune at batch 2 clears both folders with zero orphan chunk rows, and a 0-cadence run of the same fixture is byte-identical to the 3-cadence one (per-path chunk counts, content, summary, both hashes, the counts line).
• A full walk reads row metadata, not the corpus (phase 185). The four row lookups the unchanged fast path shares (_index_file and the PDF / PDF-embedded-image / standalone-image branches) go through ONE statement, _doc_row_stmt, which defers the single column the fast path never reads — Document.content. The 2026-10-15 sync inspected ~27k files and re-indexed 217, yet hydrated all 17,407 rows whole (avg 15,217 chars = 252.6 MB of markdown, held for the whole run by expire_on_commit=False and phase 184's stamp); now an inspected row carries file_hash, summary, dates, page count — and no markdown. Exactly ONE column is deferred, and there is deliberately no raiseload/noload: the phase-118 summary-NULL backfill genuinely reads the stored content, so that one row pays one extra SELECT documents.content … WHERE id = … (per-row, one-time, already rare) instead of crashing. Counts, log words, the D1 manual-date lock, the phase-163 strip re-gate and the backfill result are byte-identical — pinned by tests/integration/test_import_fast_path_columns.py (the captured fast-path SELECT names no documents.content, one SELECT per unchanged file with a summary, exactly one lazy follow-up for the backfill), tests/unit/test_importer_defer_content.py (content is the only deferred column, all four loads share the helper, no raiseload/noload anywhere in the importer) and, through the UI, tests/e2e/test_fast_path_deferred_content.py (a hand-nulled summary backfills to its byte-identical text from the RAG-page (/sources.html) re-sync with exactly one is_summary chunk, an unchanged doc's viewer still shows the full stored text, and a rewritten file re-indexes to the new text with the next sync all-unchanged again).
• The commit-date walk streams (phase 186). A git source's per-file last-commit dates come from one git log --name-only --format=@@%cI over the checkout — a walk that scales with the repository's entire history, not its working tree (the incident box measured 346,829 lines / 15 MB for one source, and the removed server/nextcloud source showed how little that has to do with the files on disk). It no longer arrives as one string: stream_git_lines (the second, documented spawn site beside run_git) yields it line by line off the pipe, so no git log is ever held in RAM, and stderr is drained on a worker thread so a chatty git cannot deadlock the pipe. The parser takes any iterable of lines (_parse_commit_dates(str | Iterable[str]) — a string still behaves exactly as before), so the grammar is untouched. The dates are byte-identical — first-sighting-wins, the same fail-soft arms ({} + the same warning lines + the mtime fallback), the D1 manual date untouched — and what IS still retained is the per-source date dict, held by the run for its whole duration (phase 186 A3: it is ~10 % of the cost this removed, and bounding it would change dates, not just memory). Pinned by tests/unit/test_git_sync.py (the stream, the run_git-shaped GitSyncError, the 340 KB-stderr no-deadlock case, the terminate-on-abandon case, and test_only_two_git_spawn_sites, the AST pin that enumerates EVERY subprocess.* spawn in app/ and scripts/ — the two git sites plus the one non-git exception, the boot-time Valkey probe in app/core/caching.py — so a new spawn anywhere fails the suite), tests/integration/test_git_file_dates.py (the streamed dict EQUALS the buffered one across a commit-1-only file, a commit-2-modified file, a subdirectory file and a never-committed file that falls back to its mtime) and, through the browser, tests/e2e/test_git_log_stream_dates.py (a real two-commit file:// source synced from the RAG page (/sources.html) shows each file's own last-commit date in the table and the viewer, an uncommitted file its mtime, the owner's manual date survives the re-sync, and an unchanged re-sync moves no date at all). One pre-existing quirk is recorded while we are here, unchanged and pinned by that suite: with git's default core.quotepath=true a non-ASCII path reaches the walk quoted and octal-escaped ("notes/caf\303\251.md"), the parser's whitespace split and \→/ normalisation turn it into a key that can never match the importer's path, and such a document therefore shows its file mtime — the day the checkout last wrote the file — instead of its commit date. Fixing it means changing the quoting grammar — a date change for every non-ASCII document already in a KB, so it is a future phase's call, not a transport phase's.
• The run's memory is a logged number and a line on the page (phase 187). Every sync lifecycle line ends with rss_mb= — and every terminal with rss_mb= rss_peak_mb= — read through app/core/memory.py (PROCESS RSS, never the host's cgroup figure, which counts page cache; an unknowable reading prints ?, never None), and the SAME pair rides GET /api/sync/status as the additive rss_mb / rss_peak_mb keys (int | null). The RAG page (/sources.html) appends it to the result line it already spoke — 3 added · 1 unchanged · peak 1936 MB, Sync cancelled · peak 1936 MB — no new control, and nothing at all when the pair is null. "How much RAM did that sync use?" is therefore one grep of the app's own log or one glance at the page that started the run, instead of a dig through journalctl unit-teardown lines from three boots (the 2026-10-15 incident). The fields are appended, so every older grep still matches, and the sampler keeps the pair honest (peak >= current — a kernel high-water refreshed lazily is raised to the current reading, because a peak below the run's own footprint is a false number, not a smaller one). Pinned by tests/unit/test_core_memory.py (every helper branch, the never-raise contract), tests/unit/test_sync_button.py (the sampling points, the 5 s throttle, the pair's invariant), tests/integration/test_sync_api.py (the four lifecycle lines, the additive keys, the terminals keeping the final pair, the anonymous 403) and end to end in the browser by tests/e2e/test_sync_peak_memory_visible.py. See Operational notes: the sync's memory (phase 187).
• Cancel: while a run is in flight, a Cancel button appears next to the sync button; clicking it posts POST /api/sync/cancel (admin only) and registers a cooperative cancel (202) — the run stops at the next pipeline boundary (a git clone/pull already in flight runs to completion in its worker thread — the request is not held, and the rest of the app keeps answering meanwhile) and settles into a neutral cancelled terminal state (error: null — intent, not failure: a "Sync cancelled" result line, no error banner or modal, the button retry-ready) that the status endpoint reports; with no run in flight the request gets a 409 (the UI treats it as a harmless race — the poll settles). A sync started after a cancel runs clean (the importer's content-hash delta re-imports what the cancel skipped).

---

Thinking

The self-hosted turbo model reasons before it answers. That reasoning is streamed with the turn as thinking SSE events and shown as "Thinking" blocks — one block per contiguous thinking run, in arrival order (phase 158) — all wrapped in one collapsible "Reasoning" bubble above the answer bubble (phase 161): the turn renders as a chronological timeline (Thinking → Tool call (1) → Thinking → … → answer), condensed into a single bounded live window (320 px, one scroll surface) that follows the tail while you watch. The Reasoning bubble is the ONE disclosure (owner refinement, 2026-10-06): the per-run blocks inside it are plain and never fold on their own.

The bubble is created on the turn's first thinking (or tool) frame and stays open from creation until the turn settles — including a mid-round reply (the model answered, called a tool, then thinks again: the reply streams into the answer while the bubble stays open, and the new thinking run gets its own block inside it instead of resuming on top of it). It folds when the turn ends, is stopped, or fails. While open, its summary carries a live status ("Thinking…" / "Calling …") and reads "Reasoning — N steps" once the turn settles; a turn with no thinking and no tools renders no bubble at all (the answer layout is byte-identical to pre-phase).

The per-run blocks are plain and always visible inside the open bubble: each block appears and fills in live while its run streams, and the runs simply stop accumulating frames when the first answer token lands, when the turn ends, or when the turn is stopped/failed. They are distinct from each other by label + rail: the brand-ink "Thinking" label on a brand rail, the accent-ink "Tool call(s) (N)" label on a neutral rail.

The tool calls ride the same timeline: each contiguous run of tool calls gets its own "Tool call(s) (N)" block (one per run, in arrival order; always visible while the bubble is open), and the per-line elapsed-seconds suffix targets the newest line of the active run.

The timeline is persisted with the saved turn (an optional segments array — the per-run order; the concatenated thinking and the tools record stay for history and older clients), so a reload and the shared page re-render the same timeline collapsed inside the Reasoning bubble, with the step-count summary.

To hide it, set BOR_STREAM_THINKING=0.

---

Agent document tools (ls + read + grep)

Retrieval only puts the top documents in context. When an answer depends on a file a note references ("the exact JSON shape is in example-record-file.json"), the model can extend its own context with server-side tools — on grounded (high-relevance) turns only:

Tool	Description
ls	Lists the knowledge base as a drill-down tree, one level at a time: no path = every source (document count + stored description), a source name = that source's top level, a source/folder path = one level deeper; file lines are source: X | path: Y | title: Z
read(path, offset?, limit?)	Appends one indexed document's text to context by its combined source/path; optional offset (1-based first line) + limit (line count) read just that part of a large document; very large reads are truncated at BOR_READ_MAX_CHARS with a notice that teaches the workflow — grep reports line numbers, then read that part with offset + limit
grep(pattern, path?)	Searches indexed documents for an exact string (case- and accent-insensitive fixed substring — accents fold: besta matches BESTÅ, phase 162); returns up to 20 source/path:line: text matches; an optional path limits to one document
show_image(path)	(only when BOR_ANSWER_IMAGES=1) shows one standalone image document as part of the answer — image documents only, at most one per answer
show_text(path, offset?, limit?)	(only when BOR_ANSWER_TEXT=1, the default) displays the named range of a text document to the user verbatim — the server shows the bytes and keeps them byte-identical; the model describes them, never retypes them; image documents are refused (use show_image); per-answer display budgets enforced server-side

(See Answer images for the show_image tool and Verbatim text display for show_text.)

The prompt teaches the search discipline (phase 162). A summary or an outline is a hint, never an inventory: the system prompt tells the model to run ls with no path — the full source list — before ever claiming the knowledge base lacks something, and to grep the artifact's part number or a filename token. One grep pattern is one string, so a name's split tokens (frame, white) and its part number (AA-1272068) each need their own grep; and a note saying a file is gitignored describes its repo status, not whether it is indexed as a source.

Git-source provenance (phase 145). When a document comes from a git source, its read result also carries an origin: line (the repository's URL, credentials stripped) and, when the source pins one, a branch: line. The prompt teaches the model to include that original source in its answer — a link to the repository with the document's path named (and the pinned branch when one is shown) — so you can open the complete original in one step rather than asking again for the source. Local (non-git) sources carry no origin: line, so nothing is added and no link is invented. The same origin rides the document-content payload (origin_url + origin_branch when pinned), and the document viewer — the chat's doc modal AND the full /document.html page, one render site — shows it as an Original external link in the document's meta row (the full repository URL as the link text, opening in a new tab; the pinned branch rides the link's title). Non-git documents show nothing new.

Each call is executed against Postgres only (no extra LLM round trip) and streamed as an SSE tool frame. In the chat, each call shows a transient calling-tool status alongside "thinking" (the send button keeps its busy state — "Stop" — for the whole turn).

The search line shows where it searches (phase 146). When a grep re-searches one document (the optional path), the chat's tool line and the calling-tool status show the document the search runs in — 🔎 Searching for <pattern> in <source/path> — so a whole-KB search and a re-search of a single document read as distinct searches instead of looking like the model is repeating itself. The scope rides the SSE tool frame additively (path, present on scoped greps only) and is persisted on the saved turn's tool record, so it re-renders after a save → restore; a bare grep renders exactly as before (frame, line, status, and saved shape all byte-identical to pre-phase — old saved turns restore unchanged).

The model may call tools as many times as needed, bounded by a round cap:

Env	Default	Meaning
BOR_AGENT_MAX_ROUNDS	10	hard cap on agent tool rounds per grounded turn (0 = no tools, the kill switch); every tool result carries the model's remaining count (phase 160)

At the cap, further tool calls are rejected (never executed, never shown) and the turn ends with a final no-tools reply: the model is coached to answer from the results it already has and to tell the user they can say "continue" to keep researching; a reply with no visible content answers with a deterministic "budget spent" fallback line instead of an error frame (phase 160, the recorded revision of the phase-71 cap-path terminal). The done frame carries an additive budget_exhausted key (ABSENT on every non-cap turn) that drives the Continue researching button on the answer's meta row — see the Chat (/) section.

---

Tuning your answers

Admin-only — sign in first. If an answer isn't quite right — too chatty, wrong assumption, missing context — tune Brain right there:

1. Press "Tune" in the meta row under any completed answer (deflected ones included).
2. Type a short instruction (1–2000 chars), e.g. "be more concise" or "assume I'm on NixOS", and Save.

The note is stored in Postgres (steering_notes) and read into the system prompt of every subsequent chat turn as a <tuning> section (numbered, oldest first, capped at BOR_STEERING_MAX_CHARS chars — default 8000, overflow marked […truncated…]).

List or remove notes from the "Tuning" button in the chat header (count badge, newest-first, per-note delete). The API is stateless JSON if you prefer curl:

curl -s localhost:8000/api/steering                                  # list
curl -s -X POST localhost:8000/api/steering \
     -H 'Content-Type: application/json' -d '{"note": "be more concise"}'
curl -s -X DELETE localhost:8000/api/steering/<note-id>              # remove

---

How retrieval works (hybrid)

Every question is embedded and also lexically tokenized (OR-joined, English stemming) and searched twice against Postgres:

1. Vector — pgvector cosine top-N (default BOR_HYBRID_VECTOR_CANDIDATES=100) over the ix_chunks_embedding_hnsw HNSW index (phase 188), searched at BOR_RETRIEVAL_EF_SEARCH (default 400, set per transaction with SET LOCAL — never a global setting). 400 is not a guess: the live battery in benchmarks/phase188_live_gate/ measured the graph recovering the exact top-100 for only 0.9814 of rows on average (0.94 worst) at 100 — the candidate count — against 1.0000 at 400, with every A8 verdict, top-5 set and top_score unchanged at both depths (the honesty thresholds were never touched). A value below the candidate count is clamped up to it with one WARNING line in the server log (retrieval: ef_search=… raised to …): a smaller ef_search returns fewer candidates than the tier promises, which is a silent recall regression the honesty gate can turn into a deflect. Raise it (max 1000) if recall matters more than the tier's latency.
2. Lexical — a stored tsvector (GIN-indexed) matched with to_tsquery, top-N by ts_rank (default BOR_HYBRID_LEXICAL_CANDIDATES=30), led by name hits — documents whose path matches a question token (gitea → gitea/…, qwen38 → qwen3.8-27b-…; phase 119). Name normalization is accent-folded (phase 162): a question's BESTÅ folds to besta and matches the ASCII filename, and an accented filename (BÅsta.md) matches an ASCII question — both directions, stdlib fold only.

The two ranked lists are fused with Reciprocal Rank Fusion (score = Σ 1/(k + rank), BOR_RRF_K=60) — a chunk in both lists scores nearly double, which lets a name-your-tool question ("gitlab") find its own document even when the question embeds close to generic templates.

The honesty gate (A8) then answers (HIGH) when the best cosine is ≥ BOR_RELEVANCE_THRESHOLD (default 0.62) or when at least one chunk matched lexically (fts_hits > 0) and the best cosine clears BOR_LEXICAL_SUPPORT_FLOOR (default 0.35) — the vector signal must corroborate a lexical hit (A8 revised 2026-09-14). It deflects (LOW) otherwise, including a weak single-token hit with vector-unsupported docs. The top BOR_TOP_N_DOCS full documents are what the LLM sees.

---

Document summaries

Raw yaml/json/py/txt embeds badly — flags and keys are not language, so retrieval can miss exactly the documents that are all configuration. At import time, every new or changed document gets a lite-model summary — markdown included: the summary is a natural-language retrieval target and the seed text for the onboarding suggestion blocks (phase 118 retired the old "markdown is already natural language" exclusion):

• The summary is stored on documents.summary and indexed as one extra embedded chunk (chunks.is_summary, position −1), so hybrid search has a natural-language target to hit instead of the raw text.
• The last line is a code-deterministic pointer — Source: <source>/<path> — appended by the app, never model-generated.
• The model only sees the first BOR_SUMMARY_MAX_CHARS (default 12000) characters; overflow is cut and marked with […truncated…].

Summary generation is best-effort: if lite fails, the document is still indexed (without a summary), the failure is logged, and counted in the import summary line (summaries=N summary_errors=N).

---

Image support

One toggle gates both image capabilities: BOR_IMAGES (default off). Enable it only when your chat model (BOR_LLM_CHAT_MODEL) supports vision — the chat model is the vision model: it describes image documents and answers questions with attached images. The embedding model never sees pixels in either flow.

Image documents

With BOR_IMAGES=1, a standalone image file — inside an uploaded source archive, or as a file in a git/local source walk — becomes a first-class document:

• Described by the chat model. On first index, a single multimodal chat-model call describes the image. The description becomes the document's content and summary — and is the only text that gets embedded, so hybrid search has a natural-language target to hit instead of pixels.
• The bytes persist. Each ingested image is copied to BOR_IMAGE_DIR/<uuid>.<ext> and served from that copy (uploads are replaced and checkouts re-cloned, so the served image outlives its source file).
• Retrieved & displayed. Image docs appear in the RAG catalog with a small lazy-loaded thumbnail (progressive enhancement — a fetch failure falls back to the doc icon), the document viewer renders the image with the description below it as wrapping prose (phase 165 — the full description reads by scrolling down, never sideways), and a cited image doc renders in the chat's sources images row (phase 130: its own .msg-meta.msg-images cluster below the chips row — no chip for an image doc, the figure is the citation; alt text = the description; a failed image load replaces the figure in place with the plain chip, so the citation survives a lost byte copy). The agent's read tool returns the description with a marker telling the model it is reading an image document.
• Prune guard. While BOR_IMAGES is off, syncs never prune existing image documents — an image doc is invisible to the walk, not a deleted file. With the toggle on, a deleted image file prunes its doc and its copy.
• Fail-soft. A failed or empty description skips that image (no row, no copy), logs a warning, and counts in the import summary line (images_failed=N) — the sync continues.

Accepted formats are png, jpg, jpeg, webp, gif, bmp, svg (BOR_IMAGE_EXTENSIONS, case-insensitive) — a separate set from BOR_IMPORT_EXTENSIONS; images are never added via that variable, the toggle is the single knob. With BOR_IMAGES=0 an .svg is therefore a plain text document — raw XML in the viewer, no image copy, no thumbnail — and the walks never admit it as an image (no special-casing of the off state).

SVG is the one vector member (phase 181). An .svg takes the same image path — its bytes are copied to BOR_IMAGE_DIR and served as image/svg+xml, so the chat figure, the document modal and the /document.html viewer show the picture instead of the XML — but it is never sent to the vision model (vision endpoints reject image/svg+xml). Its searchable text comes out of the file itself: the <title>, <desc> and <text> words, or — for a drawing that carries no words at all — one honest structural line built from the file (its name, its shape census, its viewBox). A file the extractor refuses (over 2 MiB, carrying a <!DOCTYPE/<!ENTITY declaration, or malformed XML) fails soft like any other image: no row, no copy, images_failed. A successful one logs one import: svg indexed source=… path=… structural=… chars=… line and adds to the log-only svg_indexed=N on the import summary line (added/updated count it like any other image doc). The next sync also converts an .svg that an earlier run indexed as plain text (its bytes hash is unchanged, so a hash-only fast path would freeze it as XML forever) — existing libraries need no re-import by hand. In the viewer the graphic spans the image card: a viewBox-only SVG carries no intrinsic size, so doc.format === "svg" adds .doc-image-img--vector (width: 100% + object-fit: contain) instead of letting the drawing sit in the browser's 300×150 default box; the chat figure and the RAG-table thumbnail keep their fixed caps (96px / 48px) and the modal is the full-size preview. Question images (the paperclip) stay raster-only: an .svg upload is a 422.

Those bytes are served sandboxed: because the /api/documents/<id>/image URL is directly navigable (not just an <img> source, which can never run SVG script), an SVG response carries Content-Security-Policy: default-src 'none'; sandbox so a stored drawing can never execute script or act with the app's origin — raster image responses keep their headers exactly as they were.

Question images

With the toggle on, the composer gains a paperclip button. Attach one image to a question and the chat model answers with text and image:

• Server-stored, path-persisted. On send the file is first uploaded to POST /api/chat-images (capped at BOR_CHAT_IMAGE_MAX_MB, default 10 MiB — over is a 413) and stored under BOR_CHAT_IMAGE_DIR. Saved and shared chats carry the returned path — never base64.
• Raster only (phase 181). The accepted upload extensions are the raster members of BOR_IMAGE_EXTENSIONS — an .svg is a 422. The vector family belongs to indexing, not to question attachments: a vision endpoint rejects image/svg+xml, and the question-image serve route deliberately never hands out a .svg, so accepting one would store an unanswerable, unservable file.
• Visible while uploading. While the upload is in flight the preview strip shows a spinner + "Uploading image…" (its remove ✕ disabled) and the Send button is disabled with the "Uploading…" label — a second press is dead, visually and functionally (no second upload, no second turn). On success the state clears and the turn proceeds; on failure it clears, the banner shows, and the typed text
  • the attachment stay for the retry (phase 132).
• Current turn only. The model receives a multimodal message for this question; prior turns' images are never replayed into history (the text of a prior turn stands alone).
• Rendered everywhere. The image renders in the user bubble, after a page reload, and on the shared chat page; a failed load degrades to a small "image unavailable" note (never a broken-image icon).
• Blocked on failure. If the image upload fails, the send is blocked with a banner — the question is never sent without the image the user attached.
• Toggle off ⇒ the attach control is absent, and an API request carrying an image gets an error frame with the hint: Enable BOR_IMAGES in the server's .env (and restart) to ask with an image.

A question image is never indexed as a document — it is turn-local storage, not a source.

Answer images (phase 124)

A second, independent toggle — BOR_ANSWER_IMAGES (default off): when on, the grounded chat model may show a standalone image document as part of its answer:

• The model decides. The agent is offered a fourth tool, show_image(path), and calls it only when the image itself contains the answer (a diagram, screenshot, or photo of the thing asked about) — never for mere topical relevance. At most one image per answer (enforced server-side); the text answer is always given in full — the image supplements, never replaces.
• Rendered as an answer figure. A shown image appears as a first-class figure below the answer text and above the source clusters (the chips row, then the images row — phase 130) — on the live turn, after a page reload, and on the shared chat page; a failed load collapses the figure (never a broken-image icon).
• Image citations on zero-read turns. A retrieved image document in the suggested tier is auto-cited: its figure (in the images row — no chip, phase 130) appears even when the model never reads it — the model's seed block already carries the image's full description, so a real model has no reason to read it (this applies with BOR_IMAGES=1, any BOR_ANSWER_IMAGES).
• Off = byte-identical. No fourth tool offered, no figure, no answer_image key in any SSE frame or saved record.

---

Verbatim text display (phase 182)

The mirror of the answer-image trust model, for words: when the user asks to see part of a document — a stanza, a passage, an exact range of lines — the model names the location and the server shows the bytes, byte-for-byte. The model never retypes quoted text, so a displayed quote cannot drift.

• The model decides. The grounded agent is offered a display tool, show_text(path, offset?, limit?), and calls it when the user wants to SEE, COPY or RUN a file — a quadlet, a config, a script, a unit file — or read its exact wording. Optional offset (1-based first line) and limit (line count); an overshooting limit is clamped, an offset past the last line is refused with the true line count.

• What the prompt teaches. With the switch on, the grounded <tools> section gains one clause (after the show_image clause when that rides too) saying the two things the tool description cannot: the user copies from what is displayed, not from the model's prose — so after a show_text call the model describes the lines instead of retyping them — and the inline locator fence for placing a displayed block mid-answer: a line of three backticks + bor + the combined source/path + an optional 12-40 range, with an empty body the app replaces with the stored text. Line numbers come from grep/read, never from a guess — when unsure, the range is omitted and the whole document is displayed. BOR_ANSWER_TEXT=0 removes the clause too, so the prompt never teaches a tool the model was not given.

• Server-side budgets. At most BOR_SHOW_TEXT_MAX_BLOCKS blocks per answer (a second identical show is refused — it is already on screen); each block is capped at BOR_SHOW_TEXT_MAX_CHARS, cut at a line boundary with a truncation marker. Image documents are refused with a pointer to show_image — the two display tools never convert into each other.

• The bytes ride the done frame. A turn that displayed text ends with one extra, additive SSE key: done.answer_text, the list of blocks in display order — answer order first (one per ```bor fence, the fence controls placement), then the show_text blocks the answer never located. Each entry is {source, path, title, start_line, end_line, total_lines, text, truncated, locator} where text is the server's slice of documents.content (never model output) and start_line/end_line are the server's resolved 1-based inclusive extent — the numbers a caption shows can never be the model's claim. locator is the canonical matching key (source/path, or source/path#12-40 for a range). There is no new bytes route and no eighth SSE event type: the bytes travel inline, and a saved or shared chat renders its panels from its own record. The same array persists on the saved brain record (bor.chat.v1 answer_text), so a reload and an anonymous shared page show the same blocks.

• Deflected turns resolve nothing. A ```bor fence written into a deflected answer leaks no KB text — the disclaimer turn resolves no block and its frame carries no answer_text key.

• Off = pre-phase. BOR_ANSWER_TEXT=0 removes the tool from every offer and its <tools> clause (the grounded system prompt is byte-identical to pre-phase); answers return to composed text (the model quoting from what it read). A turn with no blocks serializes without the key — absent, never null, never [] — so every block-free frame and every pre-phase saved record keeps its exact pre-phase bytes. The SERVER surface is byte-pre-phase; the CLIENT rule is flag-independent, so a stray ```bor fence in a flag-off answer still cannot paint the model's copy — with no answer_text to hydrate it, it settles to the same honest sentence (the phase E2E's App B spec).

• The client never paints the model's copy. The one shared markdown renderer (frontend/assets/markdown.js) takes a SECOND, optional argument — renderMarkdown(md, { verbatim: true }) — and with it ON a ```bor locator fence is replaced, info line, body and closing fence included, by a pending panel: <div class="verbatim verbatim-pending" data-bor-locator="…"> holding the source/path caption and the live-region line Retrieving the exact text…. The locator is normalized client-side by the exact mirror of the server's canonical_locator (source/path, or source/path#12-40 — digits only, so the two sides match on a string compare), the model's bytes appear NOWHERE in the output, and a fence the model has not finished typing shows that same pending state instead of raw backticks. An unparseable locator degrades to an ordinary code block with the model's own body — no placeholder, no badge. With the argument OFF — every other surface in the app: the document viewer, the thinking blocks, the question bubbles — the renderer's output is byte-identical to pre-phase, so a document that legitimately contains a ```bor fence still renders its own text. Pass order with the option on: verbatim → fences → tables → escape+inline → blockquotes → paragraphs → restore.

• Hydration paints the panel, and the panel is the file. The client half is one classic script, frontend/assets/verbatim.js, loaded after markdown.js on the chat page and the shared page. At done (and on every restore) it matches each placeholder's normalized data-bor-locator against the frame's answer_text[].locator with a plain string compare — markdown.js::borLocator stays the ONLY locator grammar in the client, so the two sides cannot drift — and paints:

  • .verbatim-caption — source/path · lines 12–40 of 210, or source/path · all 210 lines when the block is the whole document: the numbers are the server's extent, so a locator whose range the model got wrong still shows what was actually read (a panel matched on source/path alone is A2 made visible).
  • .verbatim-badge — the word VERBATIM: these bytes came from the knowledge base, not from the model.
  • .verbatim-lines — the server's line numbers, user-select: none, so a selection or a Copy never drags digits into the clipboard. A capped block's marker line gets a BLANK cell, because the marker is not a line of the file.
  • .verbatim-text — answer_text[].text through textContent, byte for byte (never re-wrapped, never trimmed, never joined).
  • .verbatim-copy — Copy → Copied (the state is TEXT), accessible name Copy N lines from <source/path>; the clipboard receives the server string exactly.
  • .verbatim-open — the document viewer: on the chat page the same same-page modal as a source chip (back → the chat), on the shared page a plain /document.html?source=…&path=… link.
  • .verbatim-more — on a capped block, one muted line saying the rest is in the document, plus that link.

  A locator the server did not resolve renders the honest sentence I could not verify this against the knowledge base — open <source/path> with NO badge, NO body and none of the model's bytes; and every terminal path (the error frame, the 300 s guard, Stop, the failed-turn funnel) settles any leftover placeholder, so nothing ever sits at Retrieving the exact text… once a turn is over. A show_text block the answer never located is appended in the answer_image band — below the answer text, above the source clusters.

• Every settled answer code block can be copied. enhanceAnswerCodeBlocks wraps each <pre> inside a settled answer bubble that is not part of a panel in .code-block and adds a .code-block-copy pill: no badge and no caption, because a VERBATIM claim belongs only to a server-resolved block — an answer's code is either verbatim-and-badged or plainly the model's, never a silently-substituted hybrid. Buttons appear when the turn settles, never mid-stream. The share link's copy path delegates to the same copyTextWithFallback, so the http-origin inline select-to-copy degradation exists once (the owner-locked share strings and behavior are unchanged).

• Parity, and what stays untouched. The live turn, a reload and an anonymous shared page call the same three globals in the same order, so one answer renders one way. Only the two answer renderers (plus the shared page's) opt in: a question, a thinking run and the document viewer stay one-argument calls, and document.html does not even load the panel client.

---

PDF documents (phase 126)

PDFs are opt-in: add the pdf token to BOR_IMPORT_EXTENSIONS (a stock .env never indexes them). Once admitted, a PDF is a first-class document through every entry point (git/local walks and uploaded archives):

• Text, page-aware. Text is extracted page by page (pymupdf); one document per PDF, its text-bearing pages joined with ---PAGE-N--- separators. Chunking is page-aware — every chunk keeps a Page N: anchor, so a retrieval hit stays readable in context.
• Summarized + dated like any document. The lite model summarizes it, sync sources its date (git commit / file metadata), and it appears in the catalog, the KB tree, and the agent's ls / read / grep tools.
• Embedded images join the image pipeline. With BOR_IMAGES=1, images embedded in the PDF are extracted and run through the same image-document flow as standalone images (chat-model description, persistent copy under BOR_IMAGE_DIR, retrievable + displayable); with the toggle off the pass is skipped and the PDF text indexes normally. Embedded images in a format outside BOR_IMAGE_EXTENSIONS are skipped with a warning. The extraction is de-duplicated and viewer-faithful (phase 151): repeated placements of the same image yield ONE document (first placement wins), an image with a soft mask (/SMask) is stored as the composited image a PDF viewer shows (alpha PNG — not the blank base stream), and near-white, nearly-uniform images (the table-shading "blank white" underlays) are skipped entirely — no row, no copy, no vision call. The run counts the drops log-only on the import summary line (pdf_images_deduped=N pdf_images_blank_skipped=N).
• Document context (phase 137). Every embedded image is described with the document it came from: the vision prompt — and the stored image_context + enriched content — carry an Extracted from <source>/<pdf> (page N): origin line (always present, even for a text-less page) plus the page's own text (capped at 6 000 chars; dropped when the phase-133 gate flags the text layer as garbage — origin line only). Markdown ![alt](path) references to the image's synthetic path ride along after it (phase 125, document block first), so the description names the document and the page the image illustrates. Image docs a pre-phase-137 sync left without the block (image_context NULL or markdown-only) are backfilled on the next sync — re-described once with the context (the parent's page text is re-read from the source while the PDF is still there; parent gone → origin line only), idempotent (the following sync burns zero vision calls for them), fail-soft (a failure leaves the row untouched and retries next sync), counted log-only on the import summary line (image_context_backfilled=N). The stored context is also VISIBLE in the chat suggestion block: an image doc's <document> body leads with a one-line locator — From <source>/<pdf> (page N) for a PDF-embedded image (with ; Referenced in … appended when markdown reference blocks are stored too — document block first), Referenced in <source>/<path>: "alt" for a markdown-referenced image (phase 125) — the page-text body is never rendered.
• Garbage text layer → page-screenshot fallback (phase 133). Before indexing, the extracted text passes a deterministic garbage gate (empty text layer, or ≥ 30% tag-looking lines, or ≥ 5% non-printable chars — the class of PDF that reads as raw XML/binary soup). On failure every page is rendered to a screenshot and read by the vision model; the stored content is the ---PAGE-N----anchored page descriptions (a failed page gets a page N — description unavailable placeholder — fail-soft, the doc still lands). Clean text layers never burn a fallback vision call. The run counts the affected PDFs on the import summary line (pdfs_screenshotted=N). A garbage-text PDF takes this fallback instead of the pictorial pass below — the precedence is locked, never double vision.
• Pictorial pages get vision-described (phase 140, on by default). For a PDF whose text layer is usable but thin — the IKEA assembly-instruction class (20 of 24 pages: 4–73 chars of markers around vector-drawn pictograms) — every page whose STRIPPED text is under BOR_PDF_PICTORIAL_MIN_CHARS characters (default 100; a deterministic per-page threshold, no LLM in the gate) is rendered to a PNG and described by the chat model, and the description appends to that page's content as [Page N illustration: …] — so the embeddings carry what the thin text layer couldn't, and a question about the illustrations hits. NO page cap: every candidate page is described (A21 — the LLM calls are free; a slower sync is acceptable). A PDF with zero candidate pages burns no render or vision call — its indexed content is byte-identical to pre-phase-140. The run counts the affected PDFs and pages on the import summary line (pdfs_pictorial=N pdf_pictorial_pages=N). BOR_PDF_PICTORIAL=0 turns the whole pass off (byte-identical pre-phase-140 content).
• Served as text — or as pages (phase 148). The document viewer renders the extracted text — never the binary PDF. The exception is the two page-qualifying classes: a PDF whose text layer fired the phase-133 screenshot fallback, or a PDF whose phase-140 pictorial pass described ≥ 1 sparse page. For those, every page is rendered to a persistent PNG at index time (under BOR_PDF_PAGE_DIR) and the viewer — the doc modal AND /document.html, one shared core — shows the PAGES: a vertical strip of lazy page images with "Page N" captions, in place of the text content and the Summary panel (the per-page descriptions stay in the DB for Q&A — the embeddings and summary are untouched, just not displayed). Since phase 163 this strip-only shape is the PRIMARY class only — the pictorial class keeps its text behind a Text/Pages toggle, per the next bullet. A lost copy shows a "Page N unavailable" note for that page only. Text- heavy PDFs (usable layer, zero pictorial candidates — and the whole sparse class while BOR_PDF_PICTORIAL=0) keep the text view, byte-identical to pre-phase.
• Which kind of strip it is (phase 163). The two qualifying classes are not the same problem: the screenshot class has no usable text to show, while a book with one sparse page (an illustration map page in a 36-page text) does. The document row therefore records the strip's CLASS beside the page count (documents.page_strip_mode — primary = the screenshots ARE the content, so the strip is the view; additive = a usable text layer, so the text + Summary view stays and the pages sit behind a Text/Pages toggle), and the document-content payload carries it as page_strip_mode. The key is ABSENT — never null — when there is no strip, which is also how a row written before this phase serializes; the viewer treats an absent class as primary, so every existing document renders exactly as before until the next sync stamps it (the sync's self-heal pass, phase 163).
• The strip repairs itself (phase 163). An unchanged PDF is not assumed to still have its page images on disk. On every sync, a row that claims a strip is re-gated deterministically from the current file bytes and the current settings — the same gate the changed path uses (BOR_PDF_PICTORIAL and BOR_PDF_PICTORIAL_MIN_CHARS included), with zero vision or LLM calls (rendering is CPU-only): the class is re-stamped, any <doc-id>-p{N}.png missing from BOR_PDF_PAGE_DIR is re-rendered (write-before-delete, atomically per file), and a strip that no longer qualifies — the kill switch turned off, the sparse-page bar lowered past it — is cleared along with its copies. So a wiped pages directory, or a copy set lost to an interrupted sync, repairs itself on the next sync instead of leaving "Page N unavailable" forever. It logs one PDF strip re-gate … line only when it actually acts, and a failure there never fails the sync. PDFs without a strip pay nothing extra (no extra read at all).

---

Caching & deploys

A deploy is a commit — and the browser must see it without a hard refresh. The first three bullets below are that A19 rule, applied at the transport layer by app/core/caching.py; the last two are phase 167 — gzip from app/core/compression.py and per-route validators from app/core/http_cache.py. Outermost to innermost, the stack is security-headers → gzip (only when BOR_COMPRESS_MIN_BYTES > 0) → the byte-budget logger → caching → routes, so security headers land on every response and bytes= in the budget line is the route's own, uncompressed payload (see Operational notes):

Security headers (A20). Every response carries the app-wide Content-Security-Policy (default-src 'self' plus the phase-123 img-src 'self' data: carve-out), X-Frame-Options: DENY and X-Content-Type-Options: nosniff — and the middleware preserves a CSP an inner layer already set. One route uses that deliberately: SVG document bytes (phase 181) ship default-src 'none'; sandbox instead, because a directly-navigated stored .svg must never become a script-capable same-origin document. Raster/PDF image responses gain nothing — their header set is unchanged.

• HTML pages ship Cache-Control: no-cache, no etag, no last-modified — each visit always gets a fresh 200 body.
• Assets are versioned and cached for a year. Pages reference their CSS/JS with a token (/assets/styles.css?v=<token>), and every /assets/* response ships Cache-Control: public, max-age=31536000, immutable.
• The token is the deploy. In a git checkout it is the short SHA of HEAD — so every commit/deploy flips the token and the versioned asset URLs change with it. A checkout without .git falls back to a stable content hash of the frontend/ tree.
• Responses are gzipped (phase 167). Every HTTP response except the POST /api/chat SSE stream goes through selective gzip (app/core/compression.py): when the client accepts gzip and the body is at least BOR_COMPRESS_MIN_BYTES (default 860; 0 = the feature is OFF), KB JSON ships ~8-12x smaller — the fix for the ~2 MB /api/docs/tree load. The SSE stream is skipped by EXACT path (A15: frames keep arriving one by one, uncompressed), small responses stay identity, and an already-encoded body is never double-encoded, so a fronting nginx may gzip too.
• Catalogue reads revalidate (phase 167). GET /api/docs, GET /api/docs/tree and GET /api/docs/tree/node ship a content-exact weak ETag (SHA-256 of the served body, app/core/http_cache.py) with Cache-Control: private, must-revalidate — a repeat load costs a 304 with no body, and a sync or an admin edit (folder summary, document date) changes the body, hence the tag, so it is seen on the next load. The tag is deliberately NOT sources_meta.version (bumping that marks saved chats stale, and those edits never bump it). This is a scoped extension of the rule above: validators live on these three read-only routes only — HTML pages keep publishing none, and the SSE stream and every mutating route are untouched.

No CDN, no new services, no build-step change: the middleware rewrites the asset references of the known pages in flight.

Deploy note: the very first deploy onto this scheme needs one normal page visit, so the browser revalidates the HTML once and starts requesting the versioned assets; every commit after that is picked up automatically.

Operational notes: the response byte budget (phase 167)

Every /api/* response writes ONE parseable line, so "which endpoint is heavy, right now?" is answerable from the app's own log:

2026-10-08 07:40:57 INFO     app.response_budget :: api method=GET path=/api/docs/tree status=200 bytes=8474 enc=gzip duration_ms=12

grep ' :: api ' server.log                    # every measured response
grep ' :: api ' server.log | grep 'bytes=[0-9]\{6,\}'   # the six-figure ones

• bytes= is what the ROUTE produced (uncompressed), enc= is what went on the wire (gzip or identity) — "how fat is it, and how fat did the client see it". A revalidated 304 logs bytes=0 on purpose: that is the win you want to see.
• Level: INFO at or over BOR_LOG_LARGE_RESPONSE_BYTES (default 65536), DEBUG below it, so the default log stays readable. BOR_LOG_LARGE_RESPONSE_BYTES=0 logs EVERY /api/* response at INFO — the mode for a one-off investigation. A negative value refuses startup.
• stream=1 marks the SSE turn (POST /api/chat, A15): bytes is the streamed total and duration_ms the stream's life. One line per stream, written when it ends — including a stream the client cancels (with the bytes that got out). The middleware is pure ASGI and never buffers.

Measuring the whole API (the audit behind the fix):

# the corpus that matches production's order of magnitude
#   (destructive to the local DB — see the restore-test-db skill)
uv run uvicorn app.main:app --port 8000
uv run python -m scripts.api_payload_audit --base-url http://localhost:8000

It signs in with BOR_ADMIN_PASSWORD (or --password; the single-admin login has no username), measures every candidate read route in BOTH identity and gzip forms plus one conditional revalidation, prints the corpus it measured, and marks anything over --budget (default 262144) with LOOK. It is read-only (safe against the deployed app) and always exits 0 when it got its numbers — it is a report, not a gate (LOCKED A5: nothing in this phase rejects a large payload). Exit 2 means the app was unreachable or the login was rejected. The measured before/after tables and the verdict per endpoint live in the phase record — 167_api_payload_diet/AUDIT.md under .agents/phases/.

Deploy note (compression): the app gzips its own responses, so nginx gzip on is optional — leaving it off ships the same bytes, and leaving it on is fine too: an already-encoded body is never double-encoded by either side. Passing the client's header through untouched (proxy_set_header Accept-Encoding $http_accept_encoding;, or simply not overriding it) is correct; unsetting it would hand every client uncompressed JSON.

Operational notes: the sync's memory (phase 187)

The 2026-10-15 incident made "how much RAM did that sync use?" a dig through journalctl unit-teardown lines from three boots — and the boot that killed the box had no teardown record at all, because it never shut down cleanly. Every sync lifecycle line now carries the number, so the question is one grep:

2026-10-15 04:12:07 INFO     app.api.sync :: sync: done detail={'files': 27411, …} rss_mb=412 rss_peak_mb=1936

grep ' :: ' server.log | grep ' rss_peak_mb='          # every sync terminal
grep ' :: ' server.log | grep 'sync: started '         # where each run began
grep ' :: ' server.log | grep 'sync: failed state='    # the failed ones, with the error

• rss_mb= is the process's CURRENT resident set size in MB (Linux: VmRSS from /proc/self/status), read at the moment that line is written — the run's start, the import's summary line, and every terminal.

• rss_peak_mb= is the process HIGH-WATER mark (getrusage(RUSAGE_SELF).ru_maxrss): the most the process ever held, which is the number the incident was actually about. It appears on the three terminals (sync: done …, sync: cancelled by user …, sync: failed state=failed …) beside the final counts. The two numbers come from different kernel counters and both floor to whole MB, and the kernel refreshes the high-water lazily, so a raw rss_peak_mb() can sit under a raw rss_mb() (measured: 164 against 162). The run's own sampling point (app.api.sync._sample_rss) therefore lifts a lagging peak to the current reading before anything publishes it — so on every sync line, on /api/sync/status and on the page you always get peak >= current. An unknowable peak (? / null) is never fabricated from the current reading.

• ? means "unknowable here", never 0 and never None — app/core/memory.py never raises, so a platform without /proc (or without resource) degrades the reading and the sync keeps running.

• Process vs cgroup — know which number you are reading. These are PROCESS numbers. The host's /sys/fs/cgroup/memory.peak — and the systemd "… memory peak" line the incident started from — is a different, larger figure, because a cgroup counts page cache too: that box's app cgroup showed ~2.16 GB of file against ~211 MB of anon while idle. Neither figure is wrong; the log says which one it is (LOCKED A1), and app.core.memory.cgroup_peak_mb() exists for an operator who wants the other one for context.

• Append-only (LOCKED A2): the fields trail the existing ones on the pre-phase lines (sync: started repos=… origin=… git=… local=…, sync: done detail=…, sync: cancelled by user state=cancelled files_done=… files_total=…, import: summary files=… … formats=…), and the failed funnel keeps its logger.exception traceback while gaining ONE sanitized one-line summary (error= carries exactly the masked string the status field does). So every grep written before this phase still matches.

• On the page, not only in the log. The same pair rides GET /api/sync/status (admin-only, like every sync surface) as the additive rss_mb / rss_peak_mb keys, and the RAG page (/sources.html) appends them to the result line it already speaks — 3 added · 1 unchanged · peak 1936 MB, or Sync cancelled · peak 1936 MB. A terminal KEEPS the final reading next to the final counts, so a page reloaded an hour later still shows what that run reached, and sync: done … logs the very same numbers (one reading, two surfaces — the page and the log can never disagree).

• Sampled, never per-file (LOCKED A4): the run reads its own memory at the run start, at each phase transition (import → overview → summaries), in every terminal, and — throttled to at most one read per 5 s — from the import's per-file progress hook, so the number moves while you watch a long walk without costing 27k /proc reads.

• null renders NOTHING. No reading (idle, a run that predates this phase, the upload job's status, a platform that cannot know) shows no clause at all — never peak ? MB, never peak 0 MB. The failed terminal's box stays empty by the existing contract (the banner and the modal carry the error, byte-identical); a failed run's numbers are in its log line and in the status payload.

There is deliberately no memory audit script and no metrics endpoint (LOCKED A3/A2 — no aux services, A12): the grep above is the interface.

---

Configuration reference

Everything is an environment variable (the BOR_ prefix, plus DEBUGPY*) read from .env. Every variable in .env.example is documented below — copy that file to .env and adjust.

DB-over-env overrides (phase 168): the precedence for every setting is app_settings row → BOR_* environment → code default — an override the admin saves in the Settings panel is stored in Postgres (app_settings, one row per key) and wins over the env var for that key — with no restart. The panel never writes .env (the env value stays the base; per-key Reset deletes the row and the env value wins again). BOR_SETTINGS_DB=0 (set in the container env, not .env — a committed 0 is a silent "my change did nothing") disables all DB overrides — the pre-phase behavior. Two keys are NOT overridable: BOR_DATABASE_URL (it builds the very engine that would read the override) and BOR_EMBEDDING_DIM (fixed at chunks table creation). A stored value that fails the setting's validators is skipped at read with a logged reason — a DB row never bricks boot. "No restart" is not universal: the settings the panel marks restart required (DB pool sizes, session cookie wiring, log level, compression, the static dir) apply on the next podman compose restart app; everything else applies immediately. Admins read/write the overrides through the admin-only /api/admin/settings routes (GET the catalogue, PUT /{key} set one value, DELETE /{key} reset one, DELETE all clear everything); secrets are write-only there (the response reports has_value, never the plaintext). A docs_repo PUT with embedded credentials (://user:pass@ — the phase-173 leak class) is refused with a 422 pointing at the docs_user / docs_token fields; a legacy .env value of that shape is split into those two settings once at startup (the DB rows shadow the env, .env is never rewritten).

App & branding

Env	Default	Meaning
BOR_APP_NAME	Brain of Reese	Display name everywhere (page titles, header brand, status labels)
BOR_APP_VERSION	0.1.0	Version the FastAPI app reports (GET /api/health, the OpenAPI doc)
BOR_INPUT_PLACEHOLDER	Ask me anything…	Chat composer placeholder
BOR_FOOTER_TEXT	Powered by self-hosted models	Footer line on every page
BOR_ENVIRONMENT	development	development or production (the prod compose profile sets production)
BOR_LOG_LEVEL	INFO	App log level
BOR_STATIC_DIR	frontend	Where static files are served from (the container image sets /app/static)
BOR_COMPRESS_MIN_BYTES	860	Selective-gzip floor (phase 167): responses ≥ this size compress when the client accepts gzip; 0 = OFF (no middleware), negative refuses startup. POST /api/chat (SSE) is never compressed
BOR_LOG_LARGE_RESPONSE_BYTES	65536	INFO/DEBUG cut for the per-response api method=… bytes=… enc=… byte-budget line (phase 167, grep ' :: api '); 0 = log every /api/* response at INFO, negative refuses startup. See Operational notes

Database & concurrency

Env	Default	Meaning
BOR_DATABASE_URL	local compose URL	SQLAlchemy URL (psycopg)
BOR_DB_POOL_SIZE	5	DB connection pool size
BOR_DB_POOL_MAX_OVERFLOW	10	Overflow connections above BOR_DB_POOL_SIZE
BOR_DB_POOL_RECYCLE	3600	Recycle pool connections after N seconds
BOR_CHAT_MAX_CONCURRENT	10	Max concurrent /api/chat turns

LLM (aipi)

Env	Default	Meaning
BOR_LLM_BASE_URL	https://aipi.reeseapps.com/v1	OpenAI-compatible endpoint
BOR_LLM_API_KEY	— (falls back to $AIPI_KEY)	aipi API key
BOR_LLM_CHAT_MODEL	turbo	Chat model — must support vision when BOR_IMAGES=1
BOR_LLM_EMBED_MODEL	embed	Embedding model
BOR_LLM_SUMMARY_MODEL	lite	One-shot completions: document summaries at import, KB overview, folder summaries
BOR_LLM_CHAT_THINKING_LEVEL	low	The chat turn's thinking level (pi-style, free-form — no allowlist); rides the streaming chat-completions requests as reasoning_effort; empty = the parameter is not sent
BOR_LLM_SUMMARY_THINKING_LEVEL	(empty)	The one-shot summary thinking level (document/folder summaries, KB overview, the model probe); rides the one-shot chat() calls as reasoning_effort; empty (default) = the parameter is not sent
BOR_EMBEDDING_DIM	768	Vector dimension (fixed at table creation)
BOR_LLM_RETRIES	3	Retries: chat-turn stream before the first token + empty one-shot summary replies; 0 = off
BOR_LLM_RETRY_DELAY	5	Seconds between LLM retries
BOR_LLM_TIMEOUT	120	HTTP timeout (s) for LLM API calls
BOR_STREAM_THINKING	1	Stream the model's thinking as thinking SSE events (0 = suppress)

RAG & prompt budgets

Env	Default	Meaning
BOR_TOP_N_DOCS	2	Ceiling for the cited tier, not a quota (phase 113) — a single strong doc yields one citation
BOR_RELEVANCE_THRESHOLD	0.62	Answer when best cosine ≥ this or a lexical hit corroborated by cosine ≥ BOR_LEXICAL_SUPPORT_FLOOR; otherwise honest deflection
BOR_LEXICAL_SUPPORT_FLOOR	0.35	Best-cosine floor an FTS hit must clear to flip the gate HIGH (A8 revised 2026-09-14)
BOR_SOURCE_USEFULNESS_FLOOR	0.35	Cosine bar for a citation slot (phase 113): a doc earns the cited tier only when its best-chunk cosine clears it; 0 = no bar
BOR_RELATED_MAX_DOCS	2	Cap on the related-doc tier (the de-emphasized "nearby docs" row — never a citation chip); 0 = no related docs
BOR_SUGGESTED_DOCS	5	Cap on the "start here" suggestion tier (phase 118): top-N doc summaries seeded into the grounded prompt; no cosine floor; ≥ 1
BOR_SUGGESTION_PREVIEW_CHARS	400	NULL-summary fallback preview in a suggestion block (phase 118): first N content chars + truncation marker; > 0
BOR_HISTORY_MAX_TURNS	40	Prior client-provided history turns kept per chat turn (phase 74, newest first); 0 = no history (the pre-phase-74 two-message requests)
BOR_HISTORY_MAX_CHARS	24000	Total char budget for the kept history — len(text) + len(thinking) per turn (phase 74); a turn that would overflow is dropped whole, never cut mid-answer; 0 = no history
BOR_MAX_OUTPUT_TOKENS	32768	Max answer length in tokens (answers must not be cut off)
BOR_STEERING_MAX_CHARS	8000	Char budget for the <tuning> (steering notes) prompt section
BOR_SUMMARY_MAX_CHARS	12000	Cap on document content sent to the lite summary model
BOR_KB_OVERVIEW_MAX_CHARS	4000	Char budget for the <knowledge_base> prompt section — the stored outline plus its deterministic Sources: … tail (phase 162); the cap keeps the longest-fitting prefix, so a value below the default clips the tail first (the default always holds both)
BOR_OVERVIEW_INPUT_MAX_CHARS	40000	Cap on the document list sent to lite for KB overview generation
BOR_FOLDER_SUMMARY_INPUT_MAX_CHARS	8000	Cap on a folder's document list sent to lite for folder summaries (phase 94)

Chunking & embedding

Env	Default	Meaning
BOR_CHUNK_TARGET_CHARS	2000	Target chunk size (chars)
BOR_CHUNK_OVERLAP_CHARS	200	Overlap between adjacent chunks (chars)
BOR_EMBED_BATCH_SIZE	16	Chunks per embedding API call
BOR_EMBED_QUESTION_MAX_CHARS	1200	Chat question-embed prefix cap (phase 114): the embed step embeds at most this many chars of the question — the full question still reaches the LLM prompt

Hybrid retrieval

Env	Default	Meaning
BOR_HYBRID_VECTOR_CANDIDATES	100	Cosine list width for the RRF fusion
BOR_RETRIEVAL_EF_SEARCH	400	HNSW ef_search for the cosine tier (phase 188) — SET LOCAL per transaction; clamped UP to BOR_HYBRID_VECTOR_CANDIDATES with a WARNING when lower (that clamp is the recall floor, so 0/negative fails startup); raise it for more recall, Postgres caps it at 1000. 400 is the measured depth at which the graph recovers the exact top-100 (see How retrieval works)
BOR_HYBRID_LEXICAL_CANDIDATES	30	FTS list width for the RRF fusion
BOR_RRF_K	60	RRF damping constant (1/(k + rank))
BOR_RECENCY_BOOST	0.0007	Max additive fused-score bonus a zero-age doc gets (phase 106); 0 = off
BOR_RECENCY_HALF_LIFE_DAYS	365	Recency decay timescale, days (phase 106) — the boost decays by e^-1 (~0.37x) per timescale of age
BOR_NAME_HIT_BONUS	0.005	Selection-time bonus a name-hit doc's best fused score gets (phase 119); 0 = off

Agent tools

Env	Default	Meaning
BOR_AGENT_MAX_ROUNDS	10	Hard cap on agent tool rounds per grounded turn (0 = no tools); every tool result carries the model's remaining count (phase 160)
BOR_READ_MAX_CHARS	128000	Cap on a read tool result (chars); over-cap reads truncate + a notice that teaches the grep → read(offset, limit) workflow (phase 95, rewritten phase 131)
BOR_ANSWER_TEXT	1	Verbatim file display (phase 182): the master switch, ON by default — an answer's code block is typed by the server from the indexed bytes, never retyped by the model. 0 = the escape hatch: the show_text tool is not offered, its <tools> teaching clause is absent, ```bor locator fences are not resolved, and the done frame carries no answer_text key (the server surface is exactly pre-phase; a stray fence on the client still settles to the honest unresolved sentence rather than the model's copy)
BOR_SHOW_TEXT_MAX_CHARS	12000	Per-block character budget for ONE displayed verbatim slice (phase 182): the slice is cut at a line boundary, the shared […truncated…] marker is appended and the caption's line range shrinks to what is actually shown. 0 = inert (nothing displays); a negative value refuses startup naming the field
BOR_SHOW_TEXT_MAX_BLOCKS	5	Per-answer cap on displayed verbatim blocks (phase 182), counted in answer order. 0 = inert (nothing displays); a negative value refuses startup naming the field

Import & sources

Env	Default	Meaning
BOR_IMPORT_EXTENSIONS	A9 family	Importable text formats — the A9 default is md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2; any well-formed extension is allowed. Add the pdf token to make PDFs first-class, retrievable documents (phase 126, opt-in). Images are not added here — see BOR_IMAGES
BOR_PDF_PICTORIAL	1	Master switch for the pictorial PDF pass (phase 140, on by default): a usable-text PDF's sparse pages (stripped text under BOR_PDF_PICTORIAL_MIN_CHARS) are rendered + vision-described into the indexed content ([Page N illustration: …]); 0 ⇒ content byte-identical to pre-phase-140 (no render, no vision call)
BOR_PDF_PICTORIAL_MIN_CHARS	100	Sparseness bar for the pictorial pass (phase 140): a page whose STRIPPED text is STRICTLY shorter than this many chars is a candidate (a page with exactly this many is not); 0 = every page (A21); a negative value fails startup loudly
BOR_PDF_IMAGE_SKIP_BLANK	1	Skip blank/decorative images embedded in PDFs (phase 151, on by default): near-white, nearly-uniform images (the table-shading "blank white" underlays) are dropped from the embedded-image pass — no row, no copy, no vision call; 0 ⇒ every extracted image is indexed again (pre-phase behavior; dedupe + soft-mask compositing unaffected)
BOR_PDF_PAGE_DIR	~/bor-sources/pages	Persistent home for the rendered PDF page PNGs (phase 148): a page-qualifying PDF (the phase-133 screenshot class or the phase-140 pictorial class) gets every page rendered at index time as <doc-id>-p{N}.png here, so the document viewer shows the actual pages (the descriptions stay embedded for Q&A, just not displayed); text-heavy PDFs keep the text view. Clearing the directory is self-healing — the next sync re-gates any strip that still claims pages and re-renders what is missing, with zero vision calls (phase 163)
BOR_GIT_SOURCES	— (empty)	CSV of git repo URLs — fallback while the admin Git sources page's list is empty
BOR_SOURCES_DIR	~/bor-sources	Where git repos are cloned/pulled
BOR_UPLOAD_DIR	~/bor-sources/uploads	Where uploaded source archives are unpacked
BOR_UPLOAD_MAX_MB	512	Cap (MiB) for uploaded source archives (zip-bomb guard)
BOR_SYNC_SESSION_RESET_FILES	200	Files between releases of the import run's SQLAlchemy session (phase 184): every Nth file the walk hands back everything the session still holds (expunge_all), so an unchanged-file sync stops keeping one full document row per file walked. 0 = never release — the exact pre-phase-184 behaviour; a negative value refuses startup naming the field. The release also runs once per source, before the two post-passes
BOR_SYNC_PRUNE_BATCH	500	Pruned documents per COMMIT inside the prune (phase 184) instead of one commit at the end of the whole import, so a mass prune releases its deletes as it goes instead of holding one enormous transaction. 0 = one commit at the end — the exact pre-phase-184 behaviour; a negative value refuses startup naming the field. With a non-zero batch a crash mid-prune leaves a prefix pruned, and the next sync converges (seen is recomputed from disk every run)
BOR_SUGGESTIONS	built-in list	JSON seed for onboarding chips (shown before the first saved question)

Image support (phases 122/123/124/181)

Env	Default	Meaning
BOR_IMAGES	0	Master switch for image support — off by default; enable only when BOR_LLM_CHAT_MODEL supports vision. Gates both image documents and question images
BOR_IMAGE_EXTENSIONS	png,jpg,jpeg,webp,gif,bmp,svg	Extensions indexed as image documents (case-insensitive; separate from BOR_IMPORT_EXTENSIONS). svg is the one vector member: indexed from its own <title>/<desc>/<text> with no vision call, served as image/svg+xml for a real preview
BOR_IMAGE_DIR	~/bor-sources/images	Persistent home for the served image bytes (uploads are replaced, checkouts re-cloned)
BOR_CHAT_IMAGE_DIR	~/bor-sources/chat-images	Where question images are stored (saved/shared chats carry the path, never base64)
BOR_CHAT_IMAGE_MAX_MB	10	Question-image upload cap (MiB; must be > 0)
BOR_ANSWER_IMAGES	0	Let the chat model show a standalone image document as part of its answer (phase 124): offers the agent the show_image tool and renders the shown image as an answer figure — independent of BOR_IMAGES (that toggle indexes image docs; this one shows them). The text-display sibling (BOR_ANSWER_TEXT, phase 182) lives in Agent tools

Docs push

Env	Default	Meaning
BOR_DOCS_REPO	— (empty)	The git repo chat answers can be committed to (URL or local path); while empty "Save as doc" is hidden and the push endpoint 409s — a clean URL only: credentials in the repo URL are no longer supported (phase 173), use BOR_DOCS_USER / BOR_DOCS_TOKEN. A legacy ://user:pass@ value (env or a saved override) is auto-split on startup into those settings, one-time and idempotent — the .env file is never rewritten
BOR_DOCS_USER	— (empty)	The docs-push remote's username; travels to git as the GIT_CURL_USERNAME environment variable (never in the command line); empty = the push is anonymous (phase 173)
BOR_DOCS_TOKEN	— (empty)	The docs-push remote's token — write-only: the settings API reports is_set and never the value, and the admin panel renders it masked; travels to git as GIT_CURL_PASSWORD (with GIT_TERMINAL_PROMPT=0); empty = the credential env is not set at all (phase 173)
BOR_DOCS_BRANCH	bor-docs	Branch pushes land on (git push --ff-only — you open the PR yourself)
BOR_DOCS_BASE_BRANCH	main	Branch each push is cut fresh from
BOR_DOCS_WORK_DIR	~/bor-docs	Where BOR_DOCS_REPO is checked out on the server

Admin & debugging

Env	Default	Meaning
BOR_ADMIN_PASSWORD	(required)	The single admin's password — app refuses to start when empty
BOR_SESSION_SECRET	(required)	Signing key for the bor_session cookie
BOR_SESSION_MAX_AGE	43200	Session-cookie lifetime in seconds (12 h, sliding)
BOR_SESSION_COOKIE	bor_session	Name of the signed session cookie
DEBUGPY	0	1 ⇒ attach-on-demand debugpy on DEBUGPY_PORT
DEBUGPY_PORT	5678	debugpy listen port

Customizing the look

Every identity string is an env var: BOR_APP_NAME (display name), BOR_INPUT_PLACEHOLDER (chat composer placeholder), and BOR_FOOTER_TEXT (footer line) — all served by GET /api/config and applied by assets/brand.js at boot.

The colors are set from the admin Theme tab (/theme.html, admin-only — phase 91): the 17 palette colors (the 9 identity colors plus the 8 semantic state colors — success/error/accent families) and the three strings above are edited with text fields and color pickers, persisted in the ui_settings table, and injected into every served page as an inline <style> BEFORE first paint (no red flash, no pop-in). The browser-tab favicon follows the same effective palette: GET /favicon.svg renders the hexagon art from the saved colors on every request (no process cache — a theme change is visible on the next request, no restart) and degrades to the built-in art with 200 on a DB blip (never 500). In the Theme tab, a live (unsaved) pick of surface / brand / brand_ink repaints the favicon in real time (a CSP-legal data: URI), and Save / Reset / re-show restore the /favicon.svg route — the settled palette always wins. The old CSS-file theming is retired — a leftover BOR_THEME line in a deployment's .env is simply ignored. Leave everything unset and the app renders the built-in dark-tech palette byte-identically.

---

Checking retrieval quality

Ask the real pipeline (live aipi embeddings + the current KB) whether a question lands on the right document, with gate verdict and per-document cosine / FTS / fused scores:

uv run python -m scripts.eval_retrieval "How did I install gitlab?"
uv run python -m scripts.eval_retrieval --from-file questions.txt --top 8

Requires an aipi key — BOR_LLM_API_KEY in .env (the stock configuration) or $AIPI_KEY in the environment — and an imported knowledge base.

---

Debugging

debugpy is off by default and never imported unless you opt in:

DEBUGPY=1 uv run uvicorn app.main:app
# → log line: debugpy: remote debugging ENABLED, listening on 0.0.0.0:5678

Attach from VS Code — the repo ships .vscode/launch.json with the matching configuration ("Python Debugger: Remote Attach), mapped for a local dev run:

{
  "name": "Python Debugger: Remote Attach",
  "type": "debugpy",
  "request": "attach",
  "connect": { "host": "localhost", "port": 5678 },
  "pathMappings": [
    { "localRoot": "${workspaceFolder}", "remoteRoot": "." }
  ]
}

Attaching to a run inside the container instead needs the container's own root: "remoteRoot": "/app" (that is where the image puts the code).

The port is non-blocking and attach-on-demand: the app keeps running normally until you attach. Override the port with DEBUGPY_PORT.

---

QA / Testing

Three layers — the project rule is one phase, one Playwright suite (a phase may implement a user story or an owner-directed change; see AGENTS.md):

# Unit + integration (FastAPI TestClient)
uv run pytest

# Coverage gate (phases require >90% on app/)
uv run pytest --cov=app --cov-report=term-missing

# Lint + static types
uv run ruff check .
uv run pyright

# Playwright E2E — install the browser once:
uv run playwright install chromium

# Each phase's E2E runs IN ISOLATION — one file per run:
uv run pytest tests/e2e/test_import_documents.py -v --no-cov
uv run pytest tests/e2e/test_chat_rag.py -v --no-cov
# ...one file per phase in tests/e2e/ (the stories live in
# .agents/user_stories/, see .agents/phases/todo/).
#
# Isolation is REQUIRED, not a style preference: every suite shares ONE
# Postgres, one mock aipi port (8901 — E2E_MOCK_PORT) and the KB it seeds,
# and each file truncates + reseeds the KB for its own fixture. Running
# the whole `tests/e2e/` directory in a single pytest session interleaves
# those fixtures and fails tests that pass file by file.
#
# Ports: the shared session app (tests/e2e/conftest.py) binds 8123
# (E2E_APP_PORT). A suite that needs its OWN app env boots a per-module
# uvicorn on its own port (phase 79) — e.g. E2E_APP_PORT_PRUNECASCADE for
# tests/e2e/test_prune_cascade_cleanup.py — because a second uvicorn on
# the session port dies on bind and the suite would drive the wrong
# server. So "isolation" is about the shared DATABASE, not about ports.

# To sweep every suite, loop over the files (one pytest process each):
for f in tests/e2e/test_*.py; do uv run pytest "$f" --no-cov -q || echo "FAIL $f"; done

git on PATH is an E2E prerequisite (phase 28): the git-source suites drive real clone_or_pull / file_commit_dates calls against generated local file:// fixture repos. The heaviest is phase 179's tests/e2e/test_sync_git_freeze.py: it builds a ~150 MiB pack-free repo (30 commits × 5 MiB of seeded pseudo-random bytes) so a fresh git clone genuinely costs ≈ 9 s — that clone IS the timing fixture for "a long clone must not freeze the server". It is mock-only (E2E_REAL_LLM=1 is skipped there: the clone, not the LLM import, is what the suite times).

Retrieval regression battery. tests/fixtures/retrieval_battery.txt is the tracked retrieval gate (phases 119 + 162): the name-your-tool questions, the versioned-name (qwen 3.8 27b) regression, the out-of-KB deflection sanity, and the besta / BESTÅ accent-fold pair. Run it against a KB with scripts.eval_retrieval (the real pipeline, real embeddings):

uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt

The exact expected ranks are recorded in the fixture's header comments; a question whose source is missing from the KB snapshot is recorded as skipped loudly, never silently dropped.

Deterministic E2E: by default the E2E app talks to a local mock aipi (tests/e2e/mock_llm.py) whose embeddings are real token-overlap vectors — so the cosine relevance threshold behaves like production. To run E2E against the live self-hosted models instead:

E2E_REAL_LLM=1 uv run pytest tests/e2e/test_chat_rag.py -v --no-cov

---

Production Deployment

Build the multi-stage image (frontend minified by esbuild in the builder stage, deps installed by uv, non-root runtime):

podman build -t brain-of-reese/app:latest .

Run standalone (bring your own Postgres + pgvector):

podman run -d --name brain-of-reese \
  -p 8000:8000 \
  -e BOR_DATABASE_URL=postgresql+psycopg://reese:SECRETPASSWORD@dbhost:5432/brain_of_reese \
  -e BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1 \
  -e BOR_LLM_API_KEY=$AIPI_KEY \
  -e BOR_ADMIN_PASSWORD=your-password \
  -e BOR_SESSION_SECRET=<the-hex-from-the-setup-above> \
  brain-of-reese/app:latest

BOR_ADMIN_PASSWORD and BOR_SESSION_SECRET are required — the fail-loud boot gate in Admin & Sign-in refuses to start while either is empty.

The entrypoint runs alembic upgrade head automatically on start.

Deploy note (migration 0029, phase 188): this step builds ix_chunks_embedding_hnsw — an HNSW index over chunks.embedding — and it is the one migration in this repo that takes minutes on a full corpus and needs free RAM to do it. It will not lock the table, it can be interrupted, and it self-heals — but read Deploying migration 0029 (HNSW) before running it on the production box.

Or run the whole stack from compose (app + db):

podman compose --profile prod up -d --build

The app service picks up BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET from the shell environment (or an env file) — the same fail-loud gate applies.

Production hardening: app runs as non-root (uid 10001), slim image, healthcheck on /api/health, debugpy off unless DEBUGPY=1, all assets served locally (no CDN), BOR_ENVIRONMENT=production.

Deploying migration 0029 (HNSW)

Migration 0029 builds ix_chunks_embedding_hnsw — an HNSW graph over the existing vector(768) column (m = 16, ef_construction = 64), so the cosine tier of every chat turn stops sequential-scanning benchmarks/phase188_live_gate/. Everything changes about the plan; nothing changes about the answer (the measured parity is in the benchmark link above and in tests/integration/test_chunks_hnsw_recall.py). It is still the migration to run deliberately rather than as a side effect of a deploy.

Pre-flight — all four, on the machine that runs it.

free -m                                     # free RAM, whole box
psql "$BOR_DATABASE_URL" -c "show shared_buffers;"       # + maintenance_work_mem
psql "$BOR_DATABASE_URL" -c "select extversion from pg_extension where extname='vector';"
podman exec brain-of-reese_db_1 df -h /dev/shm           # the one people miss

• pgvector ≥ 0.5.0 (0.8.0 measured). HNSW does not exist before 0.5 — the migration fails with access method "hnsw" does not exist.
• Disk: budget ~3.9 KB per chunk row — the real ~414k chunks need roughly 1.6 GB for the index on top of the ~3 GB heap. A build that runs out of disk is the one failure the self-heal cannot talk its way out of.
• RAM: the build itself is not the hungry part — a 416,702-row build peaked at 81 MB of RSS in the alembic process at the container's default maintenance_work_mem (64 MB), and 1.6 GB of index is page cache afterwards, which Linux drops under pressure. What the box needs is room for the build to finish while the app keeps serving.
• /dev/shm: see the maintenance_work_mem note below — inside this project's compose container it is 63 MB, and that is what caps the knob, not the box's RAM.

Build it.

uv run alembic upgrade head

CREATE INDEX CONCURRENTLY takes ShareUpdateExclusiveLock: reads and writes continue, the app keeps answering through the whole build, and the build takes ~161 s for 416,702 vectors at maintenance_work_mem = 64 MB (measured, production_order_build.log) — budget minutes, not seconds, and more on a 4 GB-class box.

On maintenance_work_mem:

• Never rely on the PostgreSQL 4 MB default if you can set the knob at all — a 4 MB graph build spills and is dramatically slower.
• But do not raise it above /dev/shm in this container. D5's original suggestion (SET maintenance_work_mem = '512MB') was run at production order and failed: could not resize shared memory segment "/PostgreSQL.3366409952" to 533761504 bytes: No space left on device — the container's /dev/shm is 63 MB, and the dying build leaves an INVALID index behind (build_maintenance_work_mem_512MB_failure.log). The migration's self-heal then did its job (dropped the INVALID graph, rebuilt it valid), so nothing was lost, but the recorded procedure for THIS container is: alembic upgrade head with the container default (64 MB), which costs time and not correctness. Raising maintenance_work_mem only makes sense together with a larger --shm-size / shm_size: on the db container.

CONCURRENTLY semantics, in the owner's terms. No read or write is blocked. The build is long, and if it dies — OOM killer, cancel, network drop, container restart — Postgres does not roll it back: it leaves the index present and marked indisvalid = false. That state is silent (it costs insert time on every sync and serves no query), which is why 0029 post-validates its own build and raises instead of reporting success.

Recovery.

# "did it actually build?" — one row, `t` is the answer
psql "$BOR_DATABASE_URL" -c "select i.indisvalid from pg_index i
  join pg_class c on c.oid = i.indexrelid where c.relname = 'ix_chunks_embedding_hnsw';"

# the recovery for BOTH cases is the same command:
uv run alembic upgrade head

• indisvalid = f after a killed build → upgrade head drops the INVALID remnant and rebuilds (the migration's own self-heal).
• relation "ix_chunks_embedding_hnsw" already exists → the build committed but the version stamp did not land. 0029 handles the INVALID case; a VALID leftover of the same name is dropped by hand and re-run: DROP INDEX CONCURRENTLY ix_chunks_embedding_hnsw then alembic upgrade head.
• Watching a long build from a second terminal: select * from pg_stat_progress_create_index;
• Rolling back is cheap and touches no data: uv run alembic downgrade -1 is DROP INDEX CONCURRENTLY IF EXISTS — an index is a derived structure, and the schema underneath is 0028's exactly.

After it lands. Turn on the index in the app's own terms and confirm the owner-visible parity, in this order:

uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt
uv run python -m scripts.api_payload_audit          # the app must be running

The battery's verdicts should read exactly as they did before the build (the A7/A8 contract — top_score included). The tier's own speed is the part that changed: ~550 ms → ~12–15 ms per cosine search at production order, and the plan flips from Limit → Sort → Seq Scan on chunks to Limit → Nested Loop → Index Scan on ix_chunks_embedding_hnsw — verifiable with EXPLAIN on the tier's statement, or by reading tests/integration/test_chunks_hnsw_recall.py's pin of the same thing.

Order of operations on a 4 GB-class box. Land the phase 183–187 memory work first (passive_deletes cascade, the session-release cadence, the prune's six-column read, the deferred-content fast path, the reported rss_mb/rss_peak_mb pair — see Operational notes: the sync's memory). The box that OOM-killed the 2026-10-15 sync is the box this build lands on; 1.6 GB of new index plus a graph build on top of a sync that still holds its walked documents is how a build gets killed and leaves an INVALID graph for somebody else to notice.

Reverse-proxy body size: a legal chat/save POST is now large — one conversation can carry a full-length answer (see Conversation payload limits) — so an nginx-style proxy in front of the app needs client_max_body_size ≥ 16 MB (the app itself sets no body limit); at the default 1 MB the follow-up of a long answer fails at the proxy instead of reaching the app.

---

Troubleshooting

• 401 from aipi — set BOR_LLM_API_KEY (or $AIPI_KEY).
• litellm.UnsupportedParamsError … encoding_format — the aipi proxy (litellm openai_like) rejects the encoding_format parameter. The app already works around this by POSTing a minimal {model, input} payload. If you see this, you are likely calling the endpoint with a different client — drop the parameter (or set litellm.drop_params = True on the proxy).
• Embedding dimension mismatch — aipi changed models; run uv run python -m scripts.llm_probe, update BOR_EMBEDDING_DIM, then drop + recreate the chunks table.
• No attach button / images ignored on import — image support is off (the default). Set BOR_IMAGES=1 in .env and restart — only when BOR_LLM_CHAT_MODEL supports vision. Turning the toggle off never destroys existing image documents (syncs prune-guard them).
• An .svg shows raw XML instead of the drawing — image support is off, so the file is a plain text document (phase 181). Set BOR_IMAGES=1 and re-sync: the next run converts the row in place (bytes copied, image_url live, no re-import by hand). An .svg the extractor refuses — over 2 MiB, carrying a <!DOCTYPE/<!ENTITY declaration, or malformed XML — stays a text document and is counted in images_failed.
• Honest deflection (the amber deflection bubble) — every question passes the honesty gate: deflection happens when the best cosine similarity is below BOR_RELEVANCE_THRESHOLD (default 0.62) and the lexical signal is absent or not vector-corroborated (fts_hits = 0, or best cosine below BOR_LEXICAL_SUPPORT_FLOOR — default 0.35 — A8 revised 2026-09-14). A weak cosine with a lexical hit (name-your-tool questions) still gets a grounded answer, provided the best cosine clears the floor. When it does deflect, the LLM prompt carries weak-hit titles only (no document content), the reply admits it has no notes on that and offers 2-3 concrete alternative questions about things it does have notes on, the bubble renders amber with "Maybe try" chips derived from the closest indexed titles, and the query_log row records deflected=true. This is a feature, not a bug. With a small local model, a rare turn may still answer from general knowledge with an explicit disclosure when retrieval was borderline — the gate (A8 revised) minimizes this by keeping vector-unsupported docs out of the grounded prompt, and the disclosure is surfaced, never silent.
• Answers deflect too often / too rarely — tune BOR_RELEVANCE_THRESHOLD (lower = answers more, higher = more honest deflection) and BOR_LEXICAL_SUPPORT_FLOOR (the best-cosine floor a lexical hit must clear to flip HIGH — lower = lexical hits ground more easily): 0.0 ⇒ the gate leans entirely on FTS hits; 1.0 ⇒ everything deflects unless a lexical hit is vector-corroborated. The embed model's cosines cluster in a ~0.6–0.85 band on the live KB, so the default is 0.62. Check real scores:

  SELECT question, top_score, fts_hits, deflected
  FROM query_log ORDER BY created_at DESC LIMIT 20;
  

• KB offline banner in the chat — Postgres isn't running: podman compose up -d db.
• Stuck "Thinking…" — the LLM is slow or down; the client's 300 s pre-token guard (TURN_TIMEOUT_MS) turns a hung stream into the error state automatically.

---

Planning & Architecture

The architecture, LOCKED decisions, and the phase roadmap live in .agents/PLAN.md; per-story specs in .agents/user_stories/.