ducoterra
03d26255c6
Phase 49 (owner request, chat 2026-08-28: "The git sources page should remove local directory and should instead accept a tarball or zipfile upload which it will unpack and scan … reuploading the same tarball should not create a new folder, but should unpack and overwrite the previously unpacked content" — design confirmed in the same conversation): * POST /api/git-sources/upload (admin-only, require_admin): accepts .tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap (bounds BOTH the compressed upload and the total extracted bytes — zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink escape and device/FIFO members rejected), and atomically swaps the content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the archive suffix — no missing window, a failed upload never touches the existing folder/row/KB). The git_sources row is upserted by path (kind='local', no duplicates, added_at preserved), the models are checked fail-fast (503 sanitized when down — the folder/row stay committed and the next sync/re-upload retries idempotently), and the source is scanned synchronously in the request (single-source import_sources prune=True + change-gated KB overview), answering 200 with the sync-style counts. One upload at a time (409); the request session is released before the scan so a concurrent TRUNCATE cannot deadlock against it. * app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES, archive_source_name (safe-name derivation), unpack_archive (guarded zip/tar extraction with the extracted-byte cap, no partial state), swap_in (atomic replace with restore-on-failure) — fully unit-tested. * app/config.py + .env.example: BOR_UPLOAD_DIR (default ~/bor-sources/uploads, deliberately separate from the git checkouts) and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at startup on <= 0). * python-multipart added to the dependencies — FastAPI's required multipart parser (an A2 implementation detail, phase locked decision). * The Sources page: the phase-38 "Add a local directory" form is removed; #archive-upload-form takes its place (labeled file input, "Upload & scan" button, the §7.4 never-stale lifecycle, inline role=alert error, role=status count line); hint + table caption updated. The POST /api/git-sources kind=local API contract is UNCHANGED — a plain directory is still registrable via the API, and existing Local rows list/remove/sync exactly as before. * The phase-38 story E2E (test_local_directory_sources.py) is rewritten API-driven — the form it drove is gone; its acceptance stands. * The story E2E (test_archive_upload_sources.py): the swap, upload→scan→list (the deterministic "Uploading…" in-flight state, the Local row, /api/docs + the RAG catalog), same-filename re-upload (in-place replace, prune, no duplicate row, v2-only folder), the 422 inline error + recovery (the form is not wedged), and the anonymous gate + 403. * README: the archive-upload section (formats, naming rule, in-place replace, both new settings), the local-directory form removal noted, config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB. Gates: unit+integration green, app/ coverage 99%, the story E2E green in isolation, the regression suites (git sources admin, local directory sources, sync button, import documents, nav rename, smoke, shared header) green in isolation, ruff + pyright clean. Note: per this phase's file-level staging, frontend/assets/styles.css also carries the small same-day in-flight owner rework already in the working tree (the .sign-in-mobile companion rule for the phase-48 mobile sign-in copy); the phase-49 change is the upload form's block.
91 lines
4.9 KiB
Bash
91 lines
4.9 KiB
Bash
# Brain of Reese — environment configuration
|
|
# Copy to `.env` and adjust: cp .env.example .env
|
|
# (`.env` is gitignored; never commit secrets.)
|
|
|
|
# --- App ---
|
|
BOR_ENVIRONMENT=development
|
|
# BOR_APP_NAME=Brain of Reese # display name on all pages — titles, header brand, status labels, aria text (phase 39)
|
|
# BOR_LOG_LEVEL=INFO
|
|
# BOR_STATIC_DIR=frontend # dev default; container sets /app/static
|
|
|
|
# --- Database (matches `podman compose` db service) ---
|
|
BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
|
|
|
|
# --- LLM (self-hosted, OpenAI-compatible "aipi") ---
|
|
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
|
|
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
|
|
BOR_LLM_CHAT_MODEL=turbo
|
|
BOR_LLM_EMBED_MODEL=embed
|
|
BOR_LLM_SUMMARY_MODEL=lite # one-shot completions: document summaries (phase 30), KB overview (phase 31)
|
|
BOR_EMBEDDING_DIM=768 # verified 2026-08 via scripts/llm_probe.py
|
|
BOR_STREAM_THINKING=1 # stream the model's thinking as `thinking` SSE events (0 to suppress)
|
|
|
|
# --- RAG tuning ---
|
|
BOR_TOP_N_DOCS=2
|
|
BOR_RELEVANCE_THRESHOLD=0.62 # answer when best cosine >= this OR an FTS hit; else honest deflection
|
|
BOR_MAX_OUTPUT_TOKENS=32768 # max answer length in tokens (answers must not be cut off)
|
|
BOR_STEERING_MAX_CHARS=8000 # char budget for the <tuning> (steering notes) prompt section
|
|
BOR_SUMMARY_MAX_CHARS=12000 # cap on document content sent to the lite summary model (phase 30)
|
|
BOR_KB_OVERVIEW_MAX_CHARS=4000 # char budget for the <knowledge_base> prompt section (phase 31)
|
|
BOR_OVERVIEW_INPUT_MAX_CHARS=40000 # cap on the document list sent to the lite model for the KB outline (phase 31)
|
|
BOR_CHUNK_TARGET_CHARS=2000
|
|
BOR_CHUNK_OVERLAP_CHARS=200
|
|
BOR_EMBED_BATCH_SIZE=16
|
|
|
|
# --- Hybrid retrieval (vector + Postgres FTS, RRF-fused) ---
|
|
BOR_HYBRID_VECTOR_CANDIDATES=100 # cosine list width for the fusion
|
|
BOR_HYBRID_LEXICAL_CANDIDATES=30 # FTS list width for the fusion
|
|
BOR_RRF_K=60 # Reciprocal Rank Fusion damping constant
|
|
|
|
# --- Agent document tools (phase 37: grounded turns may list + read) ---
|
|
# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools)
|
|
|
|
# --- Import scope (A9 formats; may only narrow, never widen) ---
|
|
# BOR_IMPORT_EXTENSIONS=md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2
|
|
# BOR_SUGGESTIONS=["How is my Kubernetes cluster set up?"] # JSON list of onboarding chips
|
|
|
|
# --- Import sources (git; phase 28, admin-managed since phase 35) ---
|
|
# Comma-separated git repo URLs; import_docs clones each (first run) or
|
|
# pulls it (subsequent runs) into BOR_SOURCES_DIR/<repo-name>/ and indexes
|
|
# the result. Auth via URL (e.g. an https token) or SSH keys.
|
|
#
|
|
# Phase 35 (owner permission 2026-08-26): the PRIMARY way to manage the
|
|
# list is the admin Git sources page (http://localhost:8000/git-sources.html)
|
|
# — rows stored in Postgres (git_sources table, migration 0006). This
|
|
# variable is the EMPTY-TABLE FALLBACK: it only applies while the admin
|
|
# list is empty; once the page has stored any source, this variable is
|
|
# ignored (the page is the source of truth). Empty table + empty variable
|
|
# + no local rows = no sources (import_docs falls back to --source / the
|
|
# old ~/Homelab + ~/Deployments defaults; the UI Sync button fails loudly
|
|
# with "no sources configured (git or local)").
|
|
#
|
|
# Phase 38: this env fallback is GIT-ONLY. Local directory sources (an
|
|
# existing, non-git directory on the server) have NO env var — the DB is
|
|
# the local-source registry: add them on the same admin page (the
|
|
# "Add a local directory" form, kind 'local' + path, migration 0007).
|
|
# BOR_GIT_SOURCES=https://github.com/user/homelab.git,https://github.com/user/deployments.git
|
|
# BOR_SOURCES_DIR=~/bor-sources
|
|
|
|
# Phase 49: uploaded source archives (the Sources page upload form).
|
|
# An uploaded .tar / .tar.gz / .tgz / .zip is unpacked to
|
|
# BOR_UPLOAD_DIR/<name>/ where <name> is the filename minus the archive
|
|
# suffix (homelab.tar.gz -> homelab/). Re-uploading the same name
|
|
# replaces the folder's content IN PLACE — one folder, one row, no
|
|
# missing window; a failed upload never touches the existing folder,
|
|
# row, or KB. Deliberately separate from BOR_SOURCES_DIR (git checkouts).
|
|
# BOR_UPLOAD_DIR=~/bor-sources/uploads
|
|
# BOR_UPLOAD_MAX_MB=512 # caps BOTH the compressed upload and the total
|
|
# extracted bytes (zip-bomb guard); must be > 0
|
|
|
|
# --- Admin & sign-in (single-admin password login; BOTH required) ---
|
|
# The app refuses to start while either is empty (names the missing
|
|
# variable(s) — README "Admin & sign-in"). Generate the secret with:
|
|
# python -c 'import secrets;print(secrets.token_hex(32))'
|
|
BOR_ADMIN_PASSWORD=
|
|
BOR_SESSION_SECRET=
|
|
# BOR_SESSION_MAX_AGE=43200 # signed-cookie lifetime, seconds (default 12 h, sliding)
|
|
|
|
# --- Debugging (0/1 — 1 enables attach-on-demand debugpy on port 5678) ---
|
|
DEBUGPY=0
|
|
# DEBUGPY_PORT=5678
|