All gates green. Final report: **Verified (phase 167 final pass — all 6 tasks already in `complete/`)** - Live re-measure of the headline criterion on the **1,000-doc live-replica corpus** (restored `data/bor_live_kb_replica.dump.sql` into a scratch DB `bor_audit`, alembic 0011→0027, dropped after; the shared dev DB was never touched): `GET /api/docs/tree` default depth=1 = **8,474 identity → 1,128 B with `Content-Encoding: gzip`** (< 40 KB bar; ~2 MB defect fixed), repeat load **304 / 0 bytes**; depth=8 265,147→23,515; node drill 20,844→1,065; `/api/docs` 246,366→45,104; `/api/documents/content` 892,124→60,776 (`LOOK`, accepted-with-why). Reproduces `AUDIT.md` to within 1 byte. - Transport contracts proven live: `POST /api/chat` **no** `Content-Encoding` + frames arriving live + budget line `… enc=identity duration_ms=7380 stream=1`; HTML still `no-cache` with the `?v=` rewrite (and gzipped); A20 headers on 200 **and** 304; ETag identical across identity/gzip; `depth=0|9|abc`→422; anonymous tree/node→403; unknown folder/source→404 `unknown node`. **Gates (all 100% green)** - `uv run pytest` → **3980 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (`compression.py`, `http_cache.py`, `response_budget.py` 100%; `api/docs.py` 99%, one defensive dedupe line). - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Phase + regression E2E **in isolation** (`--junitxml`, exit codes): `test_kb_tree_lazy_load` 11, `test_kb_tree` 8, `test_kb_tree_nav` 4, `test_navbar_refresh` 7, `test_document_dates` 6, `test_git_source_dates` 4, `test_sync_summary_visibility` 3, `test_table_ux_polish` 6, `test_sources_midstream_bug` 6, `test_ls_tree_drilldown` 3, `test_smoke` 3 — **0 failures/errors** (the earlier `-q -q` run had simply suppressed pytest's summary line). **Documentation** — README.md + AGENTS.md reviewed in full (1,436 / 80 lines): - Fixed (pre-existing, phase-167-exposed): README *Caching & deploys* claimed "One Starlette middleware (`app/core/caching.py`) applies the rule" while the section now describes three transport layers → rewritten to name caching/compression/http_cache, the real outermost→innermost order, and why `bytes=` is uncompressed. - AGENTS.md rule 5 (UI Structure Check) gained the phase-167 clause: a lazily-drilled level keeps the current level rendered, its in-flight `role="status"` / `role="alert"`+Retry row lives in the same bounded scroll panel, and stat cards read server `totals` (never a walk of a depth-limited tree). - Cross-checked and consistent: every `BOR_*` in `.env.example` is in the README table, every `Settings` field is documented, all 15 README-cited API routes exist in the app, no stale "one fetch / drill client-side" claim remains anywhere. **Notable decisions / deviations** - `compression.py` adds a documented `_RebufferSmallResponses` seam beyond the D1 sketch (the caching layer's `BaseHTTPMiddleware` re-framing otherwise defeats Starlette's `minimum_size` exemption and inflates tiny responses) — integration-pinned, math still Starlette's. - A node-route **404 = vanished folder → honest reset to top level** (per `00_phase.md` D4/D6), not task-06's "alert + Retry"; 500 shows alert+Retry with the level preserved — both pinned, nothing weakened. - Retrieval battery not measurable (shared E2E DB holds no corpus), but `git diff --name-only -- app/rag` is empty, so retrieval is unchanged by construction. `AUDIT.md` left byte-identical (it lives under `todo/`, off-limits to this pass); no git index/history commands run. **Next pending phase:** none — `167_api_payload_diet` is the only entry in `.agents/phases/todo/`; the harness moves it to `complete/` and commits.
210 lines
9.5 KiB
Python
210 lines
9.5 KiB
Python
"""Brain of Reese — application entrypoint.
|
|
|
|
Boots logging + conditional debugpy, then creates the FastAPI app:
|
|
API routes first (so they win over the catch-all), and the static frontend
|
|
mounted last. No CDN: everything the browser needs is served by this
|
|
process from local files (see PLAN §UI/UX — No External Dependencies).
|
|
|
|
Phase 16 (A10 revised): before anything is served, admin auth must be
|
|
configured (fail-loud), and the app wraps every route in Starlette's
|
|
SessionMiddleware — a signed ``bor_session`` cookie is the only session
|
|
state in the system.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from pathlib import Path
|
|
|
|
from fastapi import FastAPI
|
|
from fastapi.staticfiles import StaticFiles
|
|
from starlette.middleware.sessions import SessionMiddleware
|
|
from starlette.responses import FileResponse
|
|
|
|
from app.api.auth import router as auth_router
|
|
from app.api.chat import router as chat_router
|
|
from app.api.chat_images import router as chat_images_router
|
|
from app.api.chats import (
|
|
public_router as chats_public_router,
|
|
)
|
|
from app.api.chats import (
|
|
router as chats_router,
|
|
)
|
|
from app.api.chats import (
|
|
shared_page_router as chats_shared_page_router,
|
|
)
|
|
from app.api.config import router as config_router
|
|
from app.api.doc_drafts import router as doc_drafts_router
|
|
from app.api.docs import router as docs_router
|
|
from app.api.git_sources import router as git_sources_router
|
|
from app.api.health import router as health_router
|
|
from app.api.steering import router as steering_router
|
|
from app.api.suggestions import router as suggestions_router
|
|
from app.api.sync import router as sync_router
|
|
from app.api.tokens import router as tokens_router
|
|
from app.api.ui_settings import router as ui_settings_router
|
|
from app.config import get_settings
|
|
from app.core.auth import ensure_admin_configured
|
|
from app.core.caching import configure_caching
|
|
from app.core.compression import SelectiveGZipMiddleware
|
|
from app.core.debugging import configure_debugging
|
|
from app.core.logging import configure_logging
|
|
from app.core.response_budget import ResponseBudgetMiddleware
|
|
from app.core.security_headers import SecurityHeadersMiddleware
|
|
|
|
configure_logging()
|
|
configure_debugging()
|
|
|
|
settings = get_settings()
|
|
logger = logging.getLogger("app")
|
|
|
|
|
|
def _shell_routes(app: FastAPI, static_dir: Path, paths: tuple[str, ...]) -> None:
|
|
"""Phase 76: the navbar views are views of ONE shell document.
|
|
|
|
Every registered path serves ``frontend/index.html`` (the shell)
|
|
instead of its own page file: the client-side router
|
|
(``frontend/assets/router.js``) reads ``location.pathname`` at boot
|
|
and shows the matching view, so a direct load of e.g.
|
|
``/tuning.html`` deep-links to the Tuning view. Registered AFTER the
|
|
API routers and BEFORE the static catch-all mount (routes-first),
|
|
so the phase-33 caching middleware — which wraps the whole app and
|
|
already lists every one of these paths in ``HTML_PAGES`` — applies
|
|
the no-cache + ``?v=<token>`` contract to the response untouched.
|
|
The list is driven by the caller: tasks 02/03 fold the remaining
|
|
views in by extending the tuple (task 03 lands History — all four
|
|
non-chat navbar views are in; the old per-view ``.html`` files are
|
|
deleted in the same change as their shell route lands — one source
|
|
of truth).
|
|
"""
|
|
shell_file = static_dir / "index.html"
|
|
|
|
async def _shell_view() -> FileResponse:
|
|
return FileResponse(shell_file, media_type="text/html")
|
|
|
|
for path in paths:
|
|
# GET (document loads, the browser path) + HEAD — the pre-fold
|
|
# static file answered both, so the shell route keeps that
|
|
# method parity (the body is the same FileResponse; HEAD ships
|
|
# headers only).
|
|
app.api_route(
|
|
path, methods=["GET", "HEAD"], include_in_schema=False
|
|
)(_shell_view)
|
|
|
|
|
|
def create_app() -> FastAPI:
|
|
# Fail loud BEFORE serving anything (phase 16): missing
|
|
# BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET raises at boot, naming the
|
|
# variable(s) — the app never starts in a half-authenticated state.
|
|
ensure_admin_configured(settings)
|
|
|
|
app = FastAPI(title=settings.app_name, version=settings.app_version)
|
|
|
|
# Signed single-admin session cookie (Starlette middleware, itsdangerous
|
|
# signer — no server-side store, no new services). Homelab HTTP: same_site
|
|
# is "lax" and https_only stays off (documented in the README).
|
|
app.add_middleware(
|
|
SessionMiddleware,
|
|
secret_key=settings.session_secret,
|
|
session_cookie=settings.session_cookie,
|
|
max_age=settings.session_max_age,
|
|
same_site="lax",
|
|
https_only=False,
|
|
)
|
|
|
|
# API routes first so they take precedence over the catch-all static mount.
|
|
app.include_router(health_router, prefix="/api")
|
|
app.include_router(config_router, prefix="/api")
|
|
app.include_router(auth_router, prefix="/api")
|
|
app.include_router(suggestions_router, prefix="/api")
|
|
app.include_router(docs_router, prefix="/api")
|
|
app.include_router(git_sources_router, prefix="/api")
|
|
app.include_router(chat_router, prefix="/api")
|
|
# Phase 123: the question-image upload/serve pair (POST is
|
|
# user-gated like the chat turn; GET is public like saved-chat
|
|
# content — the uuid filename is the credential).
|
|
app.include_router(chat_images_router, prefix="/api")
|
|
app.include_router(steering_router, prefix="/api")
|
|
app.include_router(sync_router, prefix="/api")
|
|
app.include_router(chats_router, prefix="/api")
|
|
app.include_router(doc_drafts_router, prefix="/api")
|
|
# Phase 79: the admin token surface (create/list/revoke) — admin-only
|
|
# (router-wide require_admin; a token USER stays 403 here, task 03).
|
|
app.include_router(tokens_router, prefix="/api")
|
|
# Phase 91 (task 01): the admin UI-settings surface (GET/PUT the
|
|
# single ui_settings row — the Theme tab's persistence) — admin-only
|
|
# (router-wide require_admin; anonymous AND token users stay 403).
|
|
app.include_router(ui_settings_router, prefix="/api")
|
|
# Phase 51: the anonymous shared-chat read — NO admin dependency.
|
|
# /api/shared/<token> is the JSON snapshot; /shared/<token> (the
|
|
# page route below, registered without a prefix) is the page.
|
|
app.include_router(chats_public_router, prefix="/api")
|
|
app.include_router(chats_shared_page_router) # no prefix — /shared/<token>
|
|
|
|
# Cache busting (phase 33): the five HTML pages revalidate (no-cache)
|
|
# with ?v=<token> asset refs; /assets/* becomes immutable for a year.
|
|
# Added after the session middleware, so it wraps the whole app
|
|
# (including the static catch-all below); /api/* — the SSE chat
|
|
# stream in particular — passes through untouched.
|
|
configure_caching(app)
|
|
|
|
# Phase 167 (task 05, TODO L5): the byte budget — ONE
|
|
# ``api method=… path=… status=… bytes=… enc=… duration_ms=…`` line per
|
|
# /api/* response (INFO at/over BOR_LOG_LARGE_RESPONSE_BYTES, DEBUG
|
|
# below), so "which endpoint is heavy, right now" is a
|
|
# ``grep ' :: api '`` away. Position: added BEFORE the gzip call below,
|
|
# i.e. INSIDE it (SecurityHeaders → gzip → response-budget → caching →
|
|
# session → routes), so ``bytes=`` is the bytes the ROUTE produced
|
|
# (uncompressed) while ``enc=`` reports what went on the wire — the two
|
|
# together answer "how fat is it, and how fat did the client see it".
|
|
# Pure ASGI (never BaseHTTPMiddleware): the SSE stream is counted as it
|
|
# passes and logged once, when the stream ends.
|
|
app.add_middleware(ResponseBudgetMiddleware)
|
|
|
|
# Phase 167 (TODO L5): selective gzip — every response except the
|
|
# POST /api/chat SSE stream (skipped by EXACT path, A15) compresses
|
|
# when it is >= compress_min_bytes (the 2 MB /api/docs/tree defect).
|
|
# 0 = the feature is OFF: no middleware is added at all (the
|
|
# kill-switch convention). Position: added AFTER configure_caching but
|
|
# BEFORE the SecurityHeaders line below, so SecurityHeaders stays
|
|
# OUTERMOST (A20, the last add_middleware is outermost) and gzip sits
|
|
# outside the caching layer — an already-rewritten HTML page compresses
|
|
# too, and /api/* still passes the caching layer byte-identically
|
|
# before compression.
|
|
if settings.compress_min_bytes > 0:
|
|
app.add_middleware(
|
|
SelectiveGZipMiddleware, minimum_size=settings.compress_min_bytes
|
|
)
|
|
|
|
# Phase 82: security headers (CSP + no-framing + nosniff) — outermost on
|
|
# purpose (last add_middleware): every response carries them, including
|
|
# the static catch-all's 404s below (audit SEC-04).
|
|
app.add_middleware(SecurityHeadersMiddleware)
|
|
|
|
static_dir = Path(settings.static_dir).resolve()
|
|
if static_dir.is_dir():
|
|
# Phase 76: the folded navbar views serve the shell — the
|
|
# router picks the view from the pathname. Task 01 landed
|
|
# Tuning; task 02 folds RAG + Sources; task 03 lands History
|
|
# (list-driven — all four non-chat navbar views are in);
|
|
# phase 79 task 06 folds the sixth view (Tokens).
|
|
_shell_routes(
|
|
app,
|
|
static_dir,
|
|
(
|
|
"/tuning.html",
|
|
"/sources.html",
|
|
"/git-sources.html",
|
|
"/history.html",
|
|
"/tokens.html", # phase 79 task 06: the Tokens view
|
|
"/theme.html", # phase 91 task 04: the Theme view (shell route)
|
|
),
|
|
)
|
|
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
|
|
else:
|
|
logger.warning("static dir %s not found — serving API only", static_dir)
|
|
|
|
return app
|
|
|
|
|
|
app = create_app()
|