Files
brain-of-reese/app/main.py
T
ducoterra 2ac37455f0
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 14s
phase: 167_api_payload_diet
All gates green. Final report:

**Verified (phase 167 final pass — all 6 tasks already in `complete/`)**
- Live re-measure of the headline criterion on the **1,000-doc live-replica corpus** (restored `data/bor_live_kb_replica.dump.sql` into a scratch DB `bor_audit`, alembic 0011→0027, dropped after; the shared dev DB was never touched): `GET /api/docs/tree` default depth=1 = **8,474 identity → 1,128 B with `Content-Encoding: gzip`** (< 40 KB bar; ~2 MB defect fixed), repeat load **304 / 0 bytes**; depth=8 265,147→23,515; node drill 20,844→1,065; `/api/docs` 246,366→45,104; `/api/documents/content` 892,124→60,776 (`LOOK`, accepted-with-why). Reproduces `AUDIT.md` to within 1 byte.
- Transport contracts proven live: `POST /api/chat` **no** `Content-Encoding` + frames arriving live + budget line `… enc=identity duration_ms=7380 stream=1`; HTML still `no-cache` with the `?v=` rewrite (and gzipped); A20 headers on 200 **and** 304; ETag identical across identity/gzip; `depth=0|9|abc`→422; anonymous tree/node→403; unknown folder/source→404 `unknown node`.

**Gates (all 100% green)**
- `uv run pytest` → **3980 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (`compression.py`, `http_cache.py`, `response_budget.py` 100%; `api/docs.py` 99%, one defensive dedupe line).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Phase + regression E2E **in isolation** (`--junitxml`, exit codes): `test_kb_tree_lazy_load` 11, `test_kb_tree` 8, `test_kb_tree_nav` 4, `test_navbar_refresh` 7, `test_document_dates` 6, `test_git_source_dates` 4, `test_sync_summary_visibility` 3, `test_table_ux_polish` 6, `test_sources_midstream_bug` 6, `test_ls_tree_drilldown` 3, `test_smoke` 3 — **0 failures/errors** (the earlier `-q -q` run had simply suppressed pytest's summary line).

**Documentation** — README.md + AGENTS.md reviewed in full (1,436 / 80 lines):
- Fixed (pre-existing, phase-167-exposed): README *Caching & deploys* claimed "One Starlette middleware (`app/core/caching.py`) applies the rule" while the section now describes three transport layers → rewritten to name caching/compression/http_cache, the real outermost→innermost order, and why `bytes=` is uncompressed.
- AGENTS.md rule 5 (UI Structure Check) gained the phase-167 clause: a lazily-drilled level keeps the current level rendered, its in-flight `role="status"` / `role="alert"`+Retry row lives in the same bounded scroll panel, and stat cards read server `totals` (never a walk of a depth-limited tree).
- Cross-checked and consistent: every `BOR_*` in `.env.example` is in the README table, every `Settings` field is documented, all 15 README-cited API routes exist in the app, no stale "one fetch / drill client-side" claim remains anywhere.

**Notable decisions / deviations**
- `compression.py` adds a documented `_RebufferSmallResponses` seam beyond the D1 sketch (the caching layer's `BaseHTTPMiddleware` re-framing otherwise defeats Starlette's `minimum_size` exemption and inflates tiny responses) — integration-pinned, math still Starlette's.
- A node-route **404 = vanished folder → honest reset to top level** (per `00_phase.md` D4/D6), not task-06's "alert + Retry"; 500 shows alert+Retry with the level preserved — both pinned, nothing weakened.
- Retrieval battery not measurable (shared E2E DB holds no corpus), but `git diff --name-only -- app/rag` is empty, so retrieval is unchanged by construction. `AUDIT.md` left byte-identical (it lives under `todo/`, off-limits to this pass); no git index/history commands run.

**Next pending phase:** none — `167_api_payload_diet` is the only entry in `.agents/phases/todo/`; the harness moves it to `complete/` and commits.
2026-10-08 21:23:49 -04:00

210 lines
9.5 KiB
Python

"""Brain of Reese — application entrypoint.
Boots logging + conditional debugpy, then creates the FastAPI app:
API routes first (so they win over the catch-all), and the static frontend
mounted last. No CDN: everything the browser needs is served by this
process from local files (see PLAN §UI/UX — No External Dependencies).
Phase 16 (A10 revised): before anything is served, admin auth must be
configured (fail-loud), and the app wraps every route in Starlette's
SessionMiddleware — a signed ``bor_session`` cookie is the only session
state in the system.
"""
from __future__ import annotations
import logging
from pathlib import Path
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from starlette.middleware.sessions import SessionMiddleware
from starlette.responses import FileResponse
from app.api.auth import router as auth_router
from app.api.chat import router as chat_router
from app.api.chat_images import router as chat_images_router
from app.api.chats import (
public_router as chats_public_router,
)
from app.api.chats import (
router as chats_router,
)
from app.api.chats import (
shared_page_router as chats_shared_page_router,
)
from app.api.config import router as config_router
from app.api.doc_drafts import router as doc_drafts_router
from app.api.docs import router as docs_router
from app.api.git_sources import router as git_sources_router
from app.api.health import router as health_router
from app.api.steering import router as steering_router
from app.api.suggestions import router as suggestions_router
from app.api.sync import router as sync_router
from app.api.tokens import router as tokens_router
from app.api.ui_settings import router as ui_settings_router
from app.config import get_settings
from app.core.auth import ensure_admin_configured
from app.core.caching import configure_caching
from app.core.compression import SelectiveGZipMiddleware
from app.core.debugging import configure_debugging
from app.core.logging import configure_logging
from app.core.response_budget import ResponseBudgetMiddleware
from app.core.security_headers import SecurityHeadersMiddleware
configure_logging()
configure_debugging()
settings = get_settings()
logger = logging.getLogger("app")
def _shell_routes(app: FastAPI, static_dir: Path, paths: tuple[str, ...]) -> None:
"""Phase 76: the navbar views are views of ONE shell document.
Every registered path serves ``frontend/index.html`` (the shell)
instead of its own page file: the client-side router
(``frontend/assets/router.js``) reads ``location.pathname`` at boot
and shows the matching view, so a direct load of e.g.
``/tuning.html`` deep-links to the Tuning view. Registered AFTER the
API routers and BEFORE the static catch-all mount (routes-first),
so the phase-33 caching middleware — which wraps the whole app and
already lists every one of these paths in ``HTML_PAGES`` — applies
the no-cache + ``?v=<token>`` contract to the response untouched.
The list is driven by the caller: tasks 02/03 fold the remaining
views in by extending the tuple (task 03 lands History — all four
non-chat navbar views are in; the old per-view ``.html`` files are
deleted in the same change as their shell route lands — one source
of truth).
"""
shell_file = static_dir / "index.html"
async def _shell_view() -> FileResponse:
return FileResponse(shell_file, media_type="text/html")
for path in paths:
# GET (document loads, the browser path) + HEAD — the pre-fold
# static file answered both, so the shell route keeps that
# method parity (the body is the same FileResponse; HEAD ships
# headers only).
app.api_route(
path, methods=["GET", "HEAD"], include_in_schema=False
)(_shell_view)
def create_app() -> FastAPI:
# Fail loud BEFORE serving anything (phase 16): missing
# BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET raises at boot, naming the
# variable(s) — the app never starts in a half-authenticated state.
ensure_admin_configured(settings)
app = FastAPI(title=settings.app_name, version=settings.app_version)
# Signed single-admin session cookie (Starlette middleware, itsdangerous
# signer — no server-side store, no new services). Homelab HTTP: same_site
# is "lax" and https_only stays off (documented in the README).
app.add_middleware(
SessionMiddleware,
secret_key=settings.session_secret,
session_cookie=settings.session_cookie,
max_age=settings.session_max_age,
same_site="lax",
https_only=False,
)
# API routes first so they take precedence over the catch-all static mount.
app.include_router(health_router, prefix="/api")
app.include_router(config_router, prefix="/api")
app.include_router(auth_router, prefix="/api")
app.include_router(suggestions_router, prefix="/api")
app.include_router(docs_router, prefix="/api")
app.include_router(git_sources_router, prefix="/api")
app.include_router(chat_router, prefix="/api")
# Phase 123: the question-image upload/serve pair (POST is
# user-gated like the chat turn; GET is public like saved-chat
# content — the uuid filename is the credential).
app.include_router(chat_images_router, prefix="/api")
app.include_router(steering_router, prefix="/api")
app.include_router(sync_router, prefix="/api")
app.include_router(chats_router, prefix="/api")
app.include_router(doc_drafts_router, prefix="/api")
# Phase 79: the admin token surface (create/list/revoke) — admin-only
# (router-wide require_admin; a token USER stays 403 here, task 03).
app.include_router(tokens_router, prefix="/api")
# Phase 91 (task 01): the admin UI-settings surface (GET/PUT the
# single ui_settings row — the Theme tab's persistence) — admin-only
# (router-wide require_admin; anonymous AND token users stay 403).
app.include_router(ui_settings_router, prefix="/api")
# Phase 51: the anonymous shared-chat read — NO admin dependency.
# /api/shared/<token> is the JSON snapshot; /shared/<token> (the
# page route below, registered without a prefix) is the page.
app.include_router(chats_public_router, prefix="/api")
app.include_router(chats_shared_page_router) # no prefix — /shared/<token>
# Cache busting (phase 33): the five HTML pages revalidate (no-cache)
# with ?v=<token> asset refs; /assets/* becomes immutable for a year.
# Added after the session middleware, so it wraps the whole app
# (including the static catch-all below); /api/* — the SSE chat
# stream in particular — passes through untouched.
configure_caching(app)
# Phase 167 (task 05, TODO L5): the byte budget — ONE
# ``api method=… path=… status=… bytes=… enc=… duration_ms=…`` line per
# /api/* response (INFO at/over BOR_LOG_LARGE_RESPONSE_BYTES, DEBUG
# below), so "which endpoint is heavy, right now" is a
# ``grep ' :: api '`` away. Position: added BEFORE the gzip call below,
# i.e. INSIDE it (SecurityHeaders → gzip → response-budget → caching →
# session → routes), so ``bytes=`` is the bytes the ROUTE produced
# (uncompressed) while ``enc=`` reports what went on the wire — the two
# together answer "how fat is it, and how fat did the client see it".
# Pure ASGI (never BaseHTTPMiddleware): the SSE stream is counted as it
# passes and logged once, when the stream ends.
app.add_middleware(ResponseBudgetMiddleware)
# Phase 167 (TODO L5): selective gzip — every response except the
# POST /api/chat SSE stream (skipped by EXACT path, A15) compresses
# when it is >= compress_min_bytes (the 2 MB /api/docs/tree defect).
# 0 = the feature is OFF: no middleware is added at all (the
# kill-switch convention). Position: added AFTER configure_caching but
# BEFORE the SecurityHeaders line below, so SecurityHeaders stays
# OUTERMOST (A20, the last add_middleware is outermost) and gzip sits
# outside the caching layer — an already-rewritten HTML page compresses
# too, and /api/* still passes the caching layer byte-identically
# before compression.
if settings.compress_min_bytes > 0:
app.add_middleware(
SelectiveGZipMiddleware, minimum_size=settings.compress_min_bytes
)
# Phase 82: security headers (CSP + no-framing + nosniff) — outermost on
# purpose (last add_middleware): every response carries them, including
# the static catch-all's 404s below (audit SEC-04).
app.add_middleware(SecurityHeadersMiddleware)
static_dir = Path(settings.static_dir).resolve()
if static_dir.is_dir():
# Phase 76: the folded navbar views serve the shell — the
# router picks the view from the pathname. Task 01 landed
# Tuning; task 02 folds RAG + Sources; task 03 lands History
# (list-driven — all four non-chat navbar views are in);
# phase 79 task 06 folds the sixth view (Tokens).
_shell_routes(
app,
static_dir,
(
"/tuning.html",
"/sources.html",
"/git-sources.html",
"/history.html",
"/tokens.html", # phase 79 task 06: the Tokens view
"/theme.html", # phase 91 task 04: the Theme view (shell route)
),
)
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
else:
logger.warning("static dir %s not found — serving API only", static_dir)
return app
app = create_app()