Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): document image support (phases 122/123) + full env reference

Browse Source 
New "Image support" section covering both opt-in capabilities gated by
BOR_IMAGES: image documents (chat-model vision description becomes the
document's content/summary, only the description is embedded, persistent
image_dir copies, Sources thumbnails, viewer + inline chat source
rendering, prune guard, fail-soft images_failed counting) and question
images (config-gated composer paperclip, server-stored bytes under
chat_image_dir, path — never base64 — persisted in saved/shared chats,
current-turn-only multimodal message, toggle-off hinted error frame).
Intro pointer, import-formats note, and a troubleshooting entry added.

The Configuration reference is restructured into grouped tables and now
documents EVERY variable in .env.example (was ~34 of 66): app, database
pool/concurrency, LLM retries/timeout/thinking, RAG tiers + prompt
budgets, chunking/embedding, recency/name-hit ranking, read cap, docs
push, image support, and DEBUGPY_PORT.

LLM-Generated: true
This commit is contained in:
ducoterra
2026-09-25 08:37:28 -04:00
parent bef24e05e2
commit a6293f739b
1 changed files with 183 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+183 -9
View File
@@ -11,6 +11,10 @@ streams a grounded answer back.
If it doesn't have notes for your question, it admits it — and offers
2-3 concrete alternative questions about things it **does** have notes on.
Image support is opt-in (`BOR_IMAGES`): standalone images become
retrievable documents, and you can attach an image to a question —
see [Image support](#image-support).
> **Updated your notes?** Re-run the import — it's idempotent and only
> re-embeds what changed:
> ```bash
@@ -200,6 +204,9 @@ reflect the new files.
`__pycache__`, etc.) are skipped.
- Non-markdown files get format-aware chunking (YAML keys, JSON keys,
Python classes/defs).
- With `BOR_IMAGES=1`, standalone images (`png, jpg, jpeg, webp, gif,
bmp`) are indexed as documents too — see
[Image support](#image-support).
### Git-based sources
@@ -375,6 +382,78 @@ import summary line (`summaries=N summary_errors=N`).
---
## Image support
One toggle gates both image capabilities: `BOR_IMAGES` (default
**off**). Enable it **only when your chat model
(`BOR_LLM_CHAT_MODEL`) supports vision** — the chat model is the vision
model: it describes image documents and answers questions with attached
images. The embedding model never sees pixels in either flow.
### Image documents
With `BOR_IMAGES=1`, a standalone image file — inside an uploaded
source archive, or as a file in a git/local source walk — becomes a
first-class document:
- **Described by the chat model.** On first index, a single multimodal
chat-model call describes the image. The description becomes the
document's `content` **and** `summary` — and is the *only* text that
gets embedded, so hybrid search has a natural-language target to hit
instead of pixels.
- **The bytes persist.** Each ingested image is copied to
`BOR_IMAGE_DIR/<uuid>.<ext>` and served from that copy (uploads are
replaced and checkouts re-cloned, so the served image outlives its
source file).
- **Retrieved & displayed.** Image docs appear in the Sources catalog
with a small lazy-loaded thumbnail (progressive enhancement — a fetch
failure falls back to the doc icon), the document viewer renders the
image with the description below it, and a cited image doc renders
**inline in the chat's sources block** (alt text = the description).
The agent's `read` tool returns the description with a marker telling
the model it is reading an image document.
- **Prune guard.** While `BOR_IMAGES` is **off**, syncs never prune
existing image documents — an image doc is invisible to the walk, not
a deleted file. With the toggle on, a deleted image file prunes its
doc and its copy.
- **Fail-soft.** A failed or empty description skips that image (no
row, no copy), logs a warning, and counts in the import summary line
(`images_failed=N`) — the sync continues.
Accepted formats are `png, jpg, jpeg, webp, gif, bmp`
(`BOR_IMAGE_EXTENSIONS`, case-insensitive) — a separate set from
`BOR_IMPORT_EXTENSIONS`; images are never added via that variable, the
toggle is the single knob.
### Question images
With the toggle on, the composer gains a **paperclip** button. Attach
one image to a question and the chat model answers with text **and**
image:
- **Server-stored, path-persisted.** On send the file is first uploaded
to `POST /api/chat-images` (capped at `BOR_CHAT_IMAGE_MAX_MB`,
default 10 MiB — over is a 413) and stored under `BOR_CHAT_IMAGE_DIR`.
Saved and shared chats carry the returned *path* — never base64.
- **Current turn only.** The model receives a multimodal message for
this question; prior turns' images are never replayed into history
(the text of a prior turn stands alone).
- **Rendered everywhere.** The image renders in the user bubble, after
a page reload, and on the shared chat page; a failed load degrades to
a small "image unavailable" note (never a broken-image icon).
- **Blocked on failure.** If the image upload fails, the send is
blocked with a banner — the question is never sent without the image
the user attached.
- **Toggle off** ⇒ the attach control is absent, and an API request
carrying an image gets an error frame with the hint: *Enable
`BOR_IMAGES` in the server's `.env` (and restart) to ask with an
image.*
A question image is **never indexed as a document** — it is turn-local
storage, not a source.
---
## Caching & deploys
A deploy is a commit — and the browser must see it **without a hard refresh**.
@@ -402,40 +481,131 @@ asset references of the known pages in flight.
## Configuration reference
Everything is an environment variable (the `BOR_` prefix, plus
`DEBUGPY*`) read from `.env`. **Every variable in
[`.env.example`](.env.example) is documented below** — copy that file
to `.env` and adjust.
### App & branding
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_APP_NAME` | `Brain of Reese` | Display name everywhere (page titles, header brand, status labels) |
| `BOR_INPUT_PLACEHOLDER` | `Ask me anything…` | Chat composer placeholder |
| `BOR_FOOTER_TEXT` | `Powered by self-hosted models` | Footer line on every page |
| `BOR_ENVIRONMENT` | `development` | `development` or `production` (the prod compose profile sets `production`) |
| `BOR_LOG_LEVEL` | `INFO` | App log level |
| `BOR_STATIC_DIR` | `frontend` | Where static files are served from (the container image sets `/app/static`) |
### Database & concurrency
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_DATABASE_URL` | local compose URL | SQLAlchemy URL (psycopg) |
| `BOR_DB_POOL_SIZE` | `5` | DB connection pool size |
| `BOR_DB_POOL_MAX_OVERFLOW` | `10` | Overflow connections above `BOR_DB_POOL_SIZE` |
| `BOR_DB_POOL_RECYCLE` | `3600` | Recycle pool connections after N seconds |
| `BOR_CHAT_MAX_CONCURRENT` | `10` | Max concurrent `/api/chat` turns |
### LLM (aipi)
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_LLM_BASE_URL` | `https://aipi.reeseapps.com/v1` | OpenAI-compatible endpoint |
| `BOR_LLM_API_KEY` | — (falls back to `$AIPI_KEY`) | aipi API key |
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model |
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` |
| `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model |
| `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview |
| `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) |
| `BOR_TOP_N_DOCS` | `2` | Full documents fed to the LLM |
| `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off |
| `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries |
| `BOR_LLM_TIMEOUT` | `120` | HTTP timeout (s) for LLM API calls |
| `BOR_STREAM_THINKING` | `1` | Stream the model's thinking as `thinking` SSE events (`0` = suppress) |
### RAG & prompt budgets
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_TOP_N_DOCS` | `2` | Ceiling for the cited tier, not a quota (phase 113) — a single strong doc yields one citation |
| `BOR_RELEVANCE_THRESHOLD` | `0.62` | Answer when best cosine ≥ this **or** a lexical hit corroborated by cosine ≥ `BOR_LEXICAL_SUPPORT_FLOOR`; otherwise honest deflection |
| `BOR_LEXICAL_SUPPORT_FLOOR` | `0.35` | Best-cosine floor an FTS hit must clear to flip the gate HIGH (A8 revised 2026-09-14) |
| `BOR_SOURCE_USEFULNESS_FLOOR` | `0.35` | Cosine bar for a citation slot (phase 113): a doc earns the cited tier only when its best-chunk cosine clears it; `0` = no bar |
| `BOR_RELATED_MAX_DOCS` | `2` | Cap on the related-doc tier (the de-emphasized "nearby docs" row — never a citation chip); `0` = no related docs |
| `BOR_SUGGESTED_DOCS` | `5` | Cap on the "start here" suggestion tier (phase 118): top-N doc summaries seeded into the grounded prompt; no cosine floor; ≥ 1 |
| `BOR_SUGGESTION_PREVIEW_CHARS` | `400` | NULL-summary fallback preview in a suggestion block (phase 118): first N content chars + truncation marker; > 0 |
| `BOR_MAX_OUTPUT_TOKENS` | `32768` | Max answer length in tokens (answers must not be cut off) |
| `BOR_STEERING_MAX_CHARS` | `8000` | Char budget for the `<tuning>` (steering notes) prompt section |
| `BOR_SUMMARY_MAX_CHARS` | `12000` | Cap on document content sent to the `lite` summary model |
| `BOR_KB_OVERVIEW_MAX_CHARS` | `4000` | Char budget for the `<knowledge_base>` prompt section |
| `BOR_OVERVIEW_INPUT_MAX_CHARS` | `40000` | Cap on the document list sent to `lite` for KB overview generation |
| `BOR_FOLDER_SUMMARY_INPUT_MAX_CHARS` | `8000` | Cap on a folder's document list sent to `lite` for folder summaries (phase 94) |
### Chunking & embedding
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_CHUNK_TARGET_CHARS` | `2000` | Target chunk size (chars) |
| `BOR_CHUNK_OVERLAP_CHARS` | `200` | Overlap between adjacent chunks (chars) |
| `BOR_EMBED_BATCH_SIZE` | `16` | Chunks per embedding API call |
| `BOR_EMBED_QUESTION_MAX_CHARS` | `1200` | Chat question-embed prefix cap (phase 114): the embed step embeds at most this many chars of the question — the full question still reaches the LLM prompt |
### Hybrid retrieval
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_HYBRID_VECTOR_CANDIDATES` | `100` | Cosine list width for the RRF fusion |
| `BOR_HYBRID_LEXICAL_CANDIDATES` | `30` | FTS list width for the RRF fusion |
| `BOR_RRF_K` | `60` | RRF damping constant (`1/(k + rank)`) |
| `BOR_RECENCY_BOOST` | `0.0007` | Max additive fused-score bonus a zero-age doc gets (phase 106); `0` = off |
| `BOR_RECENCY_HALF_LIFE_DAYS` | `365` | Recency decay timescale, days (phase 106) — the boost decays by e^-1 (~0.37x) per timescale of age |
| `BOR_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a name-hit doc's best fused score gets (phase 119); `0` = off |
### Agent tools
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools) |
| `BOR_IMPORT_EXTENSIONS` | csv (see below) | Importable formats (may only narrow the A9 set) |
| `BOR_READ_MAX_CHARS` | `128000` | Cap on a `read` tool result (chars); over-cap reads truncate + a grep-pointer notice (phase 95) |
### Import & sources
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_IMPORT_EXTENSIONS` | A9 family | Importable text formats — the A9 default is `md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2`; any well-formed extension is allowed. Images are **not** added here — see `BOR_IMAGES` |
| `BOR_GIT_SOURCES` | — (empty) | CSV of git repo URLs — **fallback while the admin Git sources page's list is empty** |
| `BOR_SOURCES_DIR` | `~/bor-sources` | Where git repos are cloned/pulled |
| `BOR_UPLOAD_DIR` | `~/bor-sources/uploads` | Where uploaded source archives are unpacked |
| `BOR_UPLOAD_MAX_MB` | `512` | Cap (MiB) for uploaded source archives (zip-bomb guard) |
| `BOR_STEERING_MAX_CHARS` | `8000` | Char budget for the `<tuning>` prompt section |
| `BOR_SUMMARY_MAX_CHARS` | `12000` | Cap on document content sent to the `lite` summary model |
| `BOR_KB_OVERVIEW_MAX_CHARS` | `4000` | Char budget for the `<knowledge_base>` prompt section |
| `BOR_OVERVIEW_INPUT_MAX_CHARS` | `40000` | Cap on the document list sent to `lite` for KB overview generation |
| `BOR_SUGGESTIONS` | built-in list | JSON seed for onboarding chips (shown before the first saved question) |
### Image support (phases 122/123)
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_IMAGES` | `0` | Master switch for image support — **off by default; enable only when `BOR_LLM_CHAT_MODEL` supports vision**. Gates both image documents and question images |
| `BOR_IMAGE_EXTENSIONS` | `png,jpg,jpeg,webp,gif,bmp` | Extensions indexed as image documents (case-insensitive; separate from `BOR_IMPORT_EXTENSIONS`) |
| `BOR_IMAGE_DIR` | `~/bor-sources/images` | Persistent home for the served image bytes (uploads are replaced, checkouts re-cloned) |
| `BOR_CHAT_IMAGE_DIR` | `~/bor-sources/chat-images` | Where question images are stored (saved/shared chats carry the path, never base64) |
| `BOR_CHAT_IMAGE_MAX_MB` | `10` | Question-image upload cap (MiB; must be > 0) |
### Docs push
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_DOCS_REPO` | — (empty) | The git repo chat answers can be committed to (URL or local path); while empty "Save as doc" is hidden and the push endpoint 409s |
| `BOR_DOCS_BRANCH` | `bor-docs` | Branch pushes land on (`git push --ff-only` — you open the PR yourself) |
| `BOR_DOCS_BASE_BRANCH` | `main` | Branch each push is cut fresh from |
| `BOR_DOCS_WORK_DIR` | `~/bor-docs` | Where `BOR_DOCS_REPO` is checked out on the server |
### Admin & debugging
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_ADMIN_PASSWORD` | *(required)* | The single admin's password — app refuses to start when empty |
| `BOR_SESSION_SECRET` | *(required)* | Signing key for the `bor_session` cookie |
| `BOR_SESSION_MAX_AGE` | `43200` | Session-cookie lifetime in seconds (12 h, sliding) |
| `DEBUGPY` | `0` | `1` ⇒ attach-on-demand debugpy on `DEBUGPY_PORT` (default 5678) |
| `BOR_LOG_LEVEL` | `INFO` | App log level |
| `DEBUGPY` | `0` | `1` ⇒ attach-on-demand debugpy on `DEBUGPY_PORT` |
| `DEBUGPY_PORT` | `5678` | debugpy listen port |
### Customizing the look
@@ -576,6 +746,10 @@ served locally (no CDN), `BOR_ENVIRONMENT=production`.
- **Embedding dimension mismatch** — aipi changed models; run
`uv run python -m scripts.llm_probe`, update `BOR_EMBEDDING_DIM`, then
drop + recreate the chunks table.
- **No attach button / images ignored on import** — image support is
off (the default). Set `BOR_IMAGES=1` in `.env` and restart — only
when `BOR_LLM_CHAT_MODEL` supports vision. Turning the toggle off
never destroys existing image documents (syncs prune-guard them).
- **Honest deflection** (the amber deflection bubble) — every question
passes the honesty gate: deflection happens
when the best cosine similarity is below `BOR_RELEVANCE_THRESHOLD`