phase: 168_admin_settings_panel
**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
This commit is contained in:
@@ -0,0 +1,347 @@
|
||||
"""The admin Settings panel's read/write surface (phase 168, task 04).
|
||||
|
||||
``TODO.md`` L3: "…can edit anything in .env and save it to the
|
||||
database. The database value should override the env var. The change
|
||||
should take effect on clicking 'save'."
|
||||
|
||||
The whole router sits behind :func:`app.core.auth.require_admin`
|
||||
(router-wide ``dependencies`` — the :mod:`app.api.ui_settings` pattern):
|
||||
anonymous callers AND token users get 403 on every route (only the
|
||||
admin retunes the deployment).
|
||||
|
||||
Routes (under ``/api`` via the ``main`` registration):
|
||||
|
||||
* ``GET /api/admin/settings`` — the whole catalogue: one item per
|
||||
``Settings`` field, in ``GROUPS`` order (the stable order the E2E
|
||||
pins against), each with its effective value in env-native string
|
||||
form, a ``db``/``env``/``default`` source badge, and the three
|
||||
capability flags. Reads only; creates nothing.
|
||||
* ``PUT /api/admin/settings/{key}`` — set one value (the body's string
|
||||
is stored as-is in ``app_settings``). Fail loud BEFORE the write
|
||||
(LOCKED A5): the FULL candidate set (every overridable stored row +
|
||||
this key/value, layered over the env base) is run through the
|
||||
real ``Settings`` validators — a bad shape OR a cross-field rule
|
||||
(e.g. ``lexical_support_floor`` above a DB-overridden
|
||||
``relevance_threshold``) is a 422 that names the field and stores
|
||||
NOTHING. Unknown key → 404; a non-overridable key (LOCKED A3) → 422
|
||||
naming the env var; a blank value for a secret → 422 (an empty API
|
||||
key / session secret is a self-lockout; ``DELETE`` is the explicit
|
||||
clear). A successful write commits, calls ``invalidate_settings()``
|
||||
(task 03 makes it live — no restart) and returns the refreshed item.
|
||||
* ``DELETE /api/admin/settings/{key}`` — reset to env: deletes the row
|
||||
(idempotent — deleting an absent key is a success, not a 404),
|
||||
invalidates, returns the item with ``source="env"``/``"default"``.
|
||||
* ``DELETE /api/admin/settings`` — "Clear all overrides": deletes every
|
||||
row, invalidates, ``{"cleared": rowcount}``.
|
||||
|
||||
Three invariants (the unit tests pin them):
|
||||
|
||||
1. **Validate-before-store** — the write path never persists a value
|
||||
the ``Settings`` validators reject (A5);
|
||||
2. **Masked secrets** (LOCKED A4) — a ``secret`` field reports
|
||||
``masked``/``has_value`` only, never the plaintext, in no response
|
||||
body; every write logs one line carrying the key and the action,
|
||||
NEVER the value (AGENTS rule 10);
|
||||
3. **Never write ``.env``** (LOCKED A1) — this module contains no file
|
||||
writes, never shells out and never ``eval``s a value; the env file
|
||||
stays the base, the DB row wins, the reset deletes the row.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import Any, Literal, get_origin
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import delete, func, text
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings, get_settings, invalidate_settings
|
||||
from app.core.auth import require_admin
|
||||
from app.core.settings_catalog import CATALOG, GROUPS, env_var
|
||||
from app.core.settings_overlay import LIST_FIELDS, _coerce, enabled
|
||||
from app.db import get_db
|
||||
from app.models import AppSetting
|
||||
from app.schemas import (
|
||||
AdminSettingIn,
|
||||
AdminSettingItem,
|
||||
AdminSettingsCleared,
|
||||
AdminSettingsOut,
|
||||
AdminSettingsTotals,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("app.api.admin_settings")
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/admin/settings",
|
||||
tags=["admin-settings"],
|
||||
dependencies=[Depends(require_admin)], # phase 168: the settings panel is admin-only
|
||||
)
|
||||
|
||||
|
||||
def _field_type(key: str) -> Literal["int", "float", "bool", "str", "json"]:
|
||||
"""The panel control kind for one field, from its annotation
|
||||
(``bool`` before ``int`` — ``isinstance(True, int)`` is ``True``;
|
||||
the one ``list[str]`` field is ``json``, K2)."""
|
||||
annotation = Settings.model_fields[key].annotation
|
||||
if get_origin(annotation) in (list, tuple):
|
||||
return "json"
|
||||
if annotation is bool:
|
||||
return "bool"
|
||||
if annotation is int:
|
||||
return "int"
|
||||
if annotation is float:
|
||||
return "float"
|
||||
return "str"
|
||||
|
||||
|
||||
def _display(field_name: str, value: Any) -> str:
|
||||
"""Stringify one effective value in the env-native string form
|
||||
(D1: stored strings, one coercion place — the router's display is
|
||||
the mirror of ``settings_overlay._coerce``): bool →
|
||||
``"true"``/``"false"``, the one ``list[str]`` field → a JSON array
|
||||
(what the env spelling would carry, K2), everything else
|
||||
``str()``."""
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
if field_name in LIST_FIELDS:
|
||||
return json.dumps(value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def _rows(db: Session) -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings`` — the whole table in one
|
||||
round trip (the ``ui_settings`` "display fact, never a query
|
||||
surface" note)."""
|
||||
return {key: value for key, value in db.execute(text("SELECT key, value FROM app_settings"))}
|
||||
|
||||
|
||||
def _item(
|
||||
key: str,
|
||||
rows: dict[str, str],
|
||||
effective: Settings,
|
||||
) -> AdminSettingItem:
|
||||
"""One catalogue entry for one field (the stable builder for all
|
||||
three read paths — GET, the PUT/DELETE returns, and the error
|
||||
paths' neighbours)."""
|
||||
meta = CATALOG[key]
|
||||
value = getattr(effective, key)
|
||||
#: Only an OVERRIDABLE row counts as an override: a hand-inserted
|
||||
#: ``database_url`` / ``embedding_dim`` row is never in effect
|
||||
#: (the overlay drops it, LOCKED A3) — it is not an override.
|
||||
has_override = meta.overridable and key in rows
|
||||
#: ``source="db"`` means "in effect": a row the overlay actually
|
||||
#: applies. With the kill switch off, no row is in effect, so the
|
||||
#: badge falls back to env/default (the panel's ``overlay_enabled``
|
||||
#: banner explains the rest — D6).
|
||||
in_effect = has_override and enabled()
|
||||
if in_effect:
|
||||
source: str = "db"
|
||||
elif os.environ.get(env_var(key)) is not None:
|
||||
source = "env"
|
||||
else:
|
||||
source = "default"
|
||||
if meta.secret:
|
||||
# LOCKED A4 — write-only: the plaintext appears in NO response;
|
||||
# the read side reports has_value (is a non-empty value in
|
||||
# effect at all) and nothing else about it.
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=True,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=None,
|
||||
masked=True,
|
||||
has_value=bool(value),
|
||||
has_override=has_override,
|
||||
db_value=None,
|
||||
default="",
|
||||
)
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=False,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=_display(key, value),
|
||||
masked=False,
|
||||
has_value=bool(_display(key, value)),
|
||||
has_override=has_override,
|
||||
db_value=rows.get(key) if has_override else None,
|
||||
default=_display(key, Settings.model_fields[key].default),
|
||||
)
|
||||
|
||||
|
||||
def _validate_candidate(candidate: dict[str, str]) -> None:
|
||||
"""Validate the FULL candidate set before anything is stored
|
||||
(LOCKED A5, D5): every overridable stored row + the new
|
||||
key/value, layered over the env base, through the SAME coercion
|
||||
(``settings_overlay._coerce`` — one place for the string →
|
||||
field-value decision) and the REAL ``Settings`` validators. A bad
|
||||
shape OR a cross-field rule is a 422 that names the field — and
|
||||
nothing is written (the caller writes only on return)."""
|
||||
kwargs: dict[str, Any] = {}
|
||||
for key, raw in candidate.items():
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {key}: {exc}"
|
||||
) from exc
|
||||
try:
|
||||
Settings(**kwargs)
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
field = str(err["loc"][0])
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {field}: {err['msg']}"
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get("", response_model=AdminSettingsOut)
|
||||
def get_catalogue(
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingsOut:
|
||||
"""The whole catalogue (75 items, ``GROUPS`` order — the stable
|
||||
order the E2E pins against).
|
||||
|
||||
Reads the table once, resolves the effective ``Settings`` through
|
||||
``get_settings()`` (the overlay — DB-over-env, task 02), and
|
||||
emits one item per field. GETs log nothing here: the phase-167
|
||||
``ResponseBudgetMiddleware`` already writes the
|
||||
``api method=… bytes=…`` line, and the whole catalogue must stay
|
||||
under ``BOR_LOG_LARGE_RESPONSE_BYTES``' INFO bar (75 items ≈ 20 KB
|
||||
— verified in task 06 with the audit script).
|
||||
"""
|
||||
rows = _rows(db)
|
||||
effective = get_settings()
|
||||
items = [
|
||||
_item(key, rows, effective)
|
||||
for group in GROUPS
|
||||
for key, meta in CATALOG.items()
|
||||
if meta.group == group
|
||||
]
|
||||
return AdminSettingsOut(
|
||||
items=items,
|
||||
totals=AdminSettingsTotals(
|
||||
count=len(items),
|
||||
overridden=sum(1 for item in items if item.source == "db"),
|
||||
),
|
||||
overlay_enabled=enabled(),
|
||||
)
|
||||
|
||||
|
||||
@router.put("/{key}", response_model=AdminSettingItem)
|
||||
def set_value(
|
||||
key: str,
|
||||
payload: AdminSettingIn,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Set one value (store it in ``app_settings`` — env-native string
|
||||
form, D1), make it live (``invalidate_settings()`` — task 03) and
|
||||
return the refreshed item.
|
||||
|
||||
The write path (LOCKED A5, A3, A4): unknown key → 404; a
|
||||
non-overridable key → 422 naming the env var (A3: the engine is
|
||||
built from ``database_url`` — an override is a chicken-and-egg
|
||||
brick; ``chunks.embedding`` is fixed at ``embedding_dim`` by A6,
|
||||
so a dim override would corrupt retrieval); a blank value for a
|
||||
secret → 422 (a self-lockout; ``DELETE`` is the explicit clear);
|
||||
then the FULL candidate set is validated before anything is
|
||||
stored (a 422 names the field and writes nothing).
|
||||
"""
|
||||
meta = CATALOG.get(key)
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
if not meta.overridable:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=f"{env_var(key)} is not settable from the database",
|
||||
)
|
||||
if meta.secret and not payload.value.strip():
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"{env_var(key)} must not be blank — use Reset to fall "
|
||||
"back to the environment"
|
||||
),
|
||||
)
|
||||
# The candidate set: the overridable stored rows + this key/value
|
||||
# (the same drop rules ``settings_overlay.merge`` applies on the
|
||||
# read side — an orphan / non-overridable row can never affect
|
||||
# validation or the effective set).
|
||||
rows = _rows(db)
|
||||
candidate = {
|
||||
k: v for k, v in rows.items() if k in CATALOG and CATALOG[k].overridable
|
||||
}
|
||||
candidate[key] = payload.value
|
||||
_validate_candidate(candidate)
|
||||
# The write: upsert on the PK (INSERT … ON CONFLICT (key) DO
|
||||
# UPDATE — the row's updated_at moves with the save).
|
||||
stmt = pg_insert(AppSetting).values(key=key, value=payload.value)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["key"],
|
||||
set_={"value": stmt.excluded.value, "updated_at": func.now()},
|
||||
)
|
||||
db.execute(stmt)
|
||||
db.commit()
|
||||
# One line per write, never the value (AGENTS rule 10, A4).
|
||||
logger.info("settings action=set key=%s status=200", key)
|
||||
# Make it live (task 03): clear the cache + run the runtime reset
|
||||
# hooks, then report the refreshed item from the fresh settings.
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("/{key}", response_model=AdminSettingItem)
|
||||
def reset_value(
|
||||
key: str,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Reset one key to env (LOCKED A1: the reset is a row DELETE —
|
||||
never an empty value, so the resolver never distinguishes "no
|
||||
override" from "override to empty").
|
||||
|
||||
Idempotent: deleting a key with NO ROW is a success, not a 404
|
||||
(the panel's per-row Reset is safe to double-click — the env
|
||||
value is already in effect; an unknown *setting* key is a 404,
|
||||
like the PUT — a typo in the key is a client bug, and the item
|
||||
shape needs the catalog metadata, so "nothing to reset" can only
|
||||
mean "the row is already gone"). A successful delete logs one
|
||||
value-free line; a no-op double Reset logs nothing.
|
||||
"""
|
||||
if key not in CATALOG:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
result = db.execute(
|
||||
delete(AppSetting).where(AppSetting.key == key).returning(AppSetting.key)
|
||||
)
|
||||
db.commit()
|
||||
if len(result.all()):
|
||||
logger.info("settings action=clear key=%s", key)
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("", response_model=AdminSettingsCleared)
|
||||
def reset_all(db: Session = Depends(get_db)) -> AdminSettingsCleared: # noqa: B008
|
||||
"""The panel's "Clear all overrides": delete every row (the env
|
||||
values win again for all 75 settings), invalidate (task 03) and
|
||||
report how many rows went."""
|
||||
result = db.execute(delete(AppSetting).returning(AppSetting.key))
|
||||
db.commit()
|
||||
count = len(result.all())
|
||||
logger.info("settings action=clear_all count=%d", count)
|
||||
invalidate_settings()
|
||||
return AdminSettingsCleared(cleared=count)
|
||||
@@ -326,6 +326,22 @@ def get_llm() -> LLMClient:
|
||||
return _llm
|
||||
|
||||
|
||||
def reset_runtime_caches() -> None:
|
||||
"""Drop the settings-derived singletons (phase 168, task 03).
|
||||
|
||||
Called by ``invalidate_settings()`` after the admin saves. The next
|
||||
turn rebuilds the client from the CURRENT ``get_settings()`` — an
|
||||
in-flight turn keeps the client it started with (its ``AsyncOpenAI``
|
||||
transport is mid-request; swapping it would kill the stream), which
|
||||
is the same accepted-semantics note as A12's in-process state.
|
||||
``_chat_active`` is deliberately untouched: it counts live turns, and
|
||||
the new ``chat_max_concurrent`` applies as permits are taken.
|
||||
"""
|
||||
global _llm, _chat_semaphore
|
||||
_llm = None
|
||||
_chat_semaphore = None
|
||||
|
||||
|
||||
def _whole_line_number(value: Any) -> bool:
|
||||
"""A ``read`` range argument's whole-line-number guard (phase 131,
|
||||
task 04).
|
||||
|
||||
+34
-1
@@ -905,4 +905,37 @@ class Settings(BaseSettings):
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
return Settings()
|
||||
"""Effective settings: env/.env with the admin's DB overrides layered
|
||||
on top (phase 168, TODO L3: "The database value should override
|
||||
the env var").
|
||||
|
||||
The DB half lives in :mod:`app.core.settings_overlay` — imported
|
||||
LAZILY, on purpose: ``app.core.*`` imports ``app.config`` at module
|
||||
scope (and ``app.db`` → ``app.config`` runs at import time), so a
|
||||
module-scope import here would deadlock the import graph. The
|
||||
``@lru_cache`` is unchanged from pre-phase — every existing
|
||||
``get_settings.cache_clear()`` call site keeps working (they only
|
||||
retune the environment, so skipping the reset hooks there is
|
||||
correct), and :func:`invalidate_settings` is the save-side wrapper
|
||||
that also runs the runtime reset hooks (phase 168, task 03).
|
||||
"""
|
||||
from app.core.settings_overlay import resolve_settings
|
||||
|
||||
return resolve_settings()
|
||||
|
||||
|
||||
def invalidate_settings() -> None:
|
||||
"""Save-side settings refresh (phase 168, task 02/03).
|
||||
|
||||
Clears the ``get_settings`` cache (the next resolution re-reads the
|
||||
DB overrides — no restart) and runs the registered runtime reset
|
||||
hooks (task 03: the in-process singletons — the chat LLM client
|
||||
and semaphore — rebuild from the fresh settings). The six existing
|
||||
``get_settings.cache_clear()`` test call sites stay the env-only
|
||||
retune path: they re-resolve without running the hooks, which is
|
||||
exactly what a test that only re-tunes the environment wants.
|
||||
"""
|
||||
get_settings.cache_clear()
|
||||
from app.core.settings_overlay import run_invalidators
|
||||
|
||||
run_invalidators()
|
||||
|
||||
@@ -177,6 +177,7 @@ HTML_PAGES: tuple[str, ...] = (
|
||||
"/history.html", # phase 50: the admin saved-chats page
|
||||
"/tokens.html", # phase 79 task 06: the admin tokens page (shell route)
|
||||
"/theme.html", # phase 91 task 04: the admin theme page (shell route)
|
||||
"/settings.html", # phase 168: the admin settings view (shell route)
|
||||
# phase 51: the shared page's STATIC path (the static mount serves
|
||||
# shared.html at /shared.html as well as the real route serves the
|
||||
# dynamic /shared/<token> — both must carry the no-cache + ?v=
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
"""The admin Settings panel's catalogue (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "I want an admin panel that can edit anything in .env
|
||||
and save it to the database."
|
||||
|
||||
This module is the single source of truth for what the panel shows and
|
||||
what the DB-over-env overlay (``app.core.settings_overlay``) is allowed
|
||||
to do with each of the ``Settings`` fields:
|
||||
|
||||
* :data:`CATALOG` maps every ``Settings`` field name to its
|
||||
presentation metadata (``group`` + ``label``) and its capability
|
||||
flags:
|
||||
|
||||
- ``secret`` — write-only (LOCKED A4): the API reports ``is_set``,
|
||||
never the plaintext; a saved value is masked on read.
|
||||
- ``restart_required`` — boot-wired (LOCKED A4): the middleware /
|
||||
engine / logging / static mount read it once at boot, so the
|
||||
override applies on the NEXT boot (the panel shows the badge).
|
||||
- ``overridable`` — ``False`` for LOCKED A3's ``database_url``
|
||||
(it builds the very engine that would read the override — a
|
||||
chicken-and-egg brick) and ``embedding_dim`` (A6 fixes
|
||||
``chunks.embedding`` at table-creation; a dim override would
|
||||
corrupt retrieval).
|
||||
|
||||
The panel (task 05) is GENERATED from this catalogue — never
|
||||
hand-written — so a new ``Settings`` field appears there only once
|
||||
it has a :class:`SettingMeta`; the completeness pin
|
||||
(``tests/unit/test_settings_catalog.py``: ``set(CATALOG) ==
|
||||
set(Settings.model_fields)``) fails the build the moment a field is
|
||||
added without metadata, so it can never silently disappear from the
|
||||
panel.
|
||||
* :data:`GROUPS` is the display order of the groups — it mirrors
|
||||
``app/config.py``'s own section comments (App, HTTP response
|
||||
compression / byte budget, UI customization, Database, LLM, Chat
|
||||
history, RAG tuning, …).
|
||||
* :data:`SECRETS`, :data:`RESTART_REQUIRED` and
|
||||
:data:`NON_OVERRIDABLE` are DERIVED from :data:`CATALOG` — never
|
||||
re-listed by hand, so they cannot drift.
|
||||
|
||||
:func:`env_var` keeps the ``BOR_`` prefix a pure presentation concern
|
||||
(D1): the stored key is the bare ``Settings`` field name
|
||||
(``relevance_threshold``), ``"BOR_" + key.upper()`` is what the panel
|
||||
and the API show, and a stored prefix could double up — so it is
|
||||
derived, never stored.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SettingMeta:
|
||||
"""One settings field's panel metadata + capability flags."""
|
||||
|
||||
key: str
|
||||
group: str
|
||||
label: str
|
||||
#: Write-only (LOCKED A4) — masked in responses, ``is_set`` only.
|
||||
secret: bool = False
|
||||
#: Boot-wired (LOCKED A4) — applies on the next boot.
|
||||
restart_required: bool = False
|
||||
#: ``False`` = read-only row (LOCKED A3).
|
||||
overridable: bool = True
|
||||
|
||||
|
||||
#: The ``BOR_`` prefix is presentation only (D1) — see :func:`env_var`.
|
||||
ENV_PREFIX = "BOR_"
|
||||
|
||||
#: Display order of the groups (mirrors ``app/config.py``'s section
|
||||
#: comments); :data:`CATALOG` is keyed in the same order.
|
||||
GROUPS: tuple[str, ...] = (
|
||||
"App & branding",
|
||||
"HTTP & logging",
|
||||
"Database",
|
||||
"LLM (aipi)",
|
||||
"Chat history",
|
||||
"Retrieval & honesty",
|
||||
"Chunking & embedding",
|
||||
"Prompt budgets",
|
||||
"Agent tools",
|
||||
"Import & sources",
|
||||
"Images",
|
||||
"PDF",
|
||||
"Docs push",
|
||||
"Admin & security",
|
||||
)
|
||||
|
||||
#: Every ``Settings`` field, in :data:`GROUPS` display order. Completeness
|
||||
#: is pinned by ``tests/unit/test_settings_catalog.py`` (the diff is in
|
||||
#: the failure message — a new setting without metadata fails the build).
|
||||
CATALOG: dict[str, SettingMeta] = {
|
||||
# --- App & branding ---
|
||||
"app_name": SettingMeta("app_name", "App & branding", "App name"),
|
||||
"app_version": SettingMeta("app_version", "App & branding", "App version"),
|
||||
"environment": SettingMeta("environment", "App & branding", "Environment"),
|
||||
"input_placeholder": SettingMeta(
|
||||
"input_placeholder", "App & branding", "Input placeholder"
|
||||
),
|
||||
"footer_text": SettingMeta("footer_text", "App & branding", "Footer text"),
|
||||
"suggestions": SettingMeta(
|
||||
"suggestions", "App & branding", "Suggestions (JSON array)"
|
||||
),
|
||||
# --- HTTP & logging ---
|
||||
"log_level": SettingMeta(
|
||||
"log_level", "HTTP & logging", "Log level", restart_required=True
|
||||
),
|
||||
"log_large_response_bytes": SettingMeta(
|
||||
"log_large_response_bytes",
|
||||
"HTTP & logging",
|
||||
"Log large response bytes",
|
||||
),
|
||||
"compress_min_bytes": SettingMeta(
|
||||
"compress_min_bytes", "HTTP & logging", "Compress min bytes",
|
||||
restart_required=True,
|
||||
),
|
||||
"static_dir": SettingMeta(
|
||||
"static_dir", "HTTP & logging", "Static dir", restart_required=True
|
||||
),
|
||||
# --- Database ---
|
||||
"database_url": SettingMeta(
|
||||
"database_url", "Database", "Database URL",
|
||||
secret=True, overridable=False,
|
||||
),
|
||||
"db_pool_size": SettingMeta(
|
||||
"db_pool_size", "Database", "DB pool size", restart_required=True
|
||||
),
|
||||
"db_pool_max_overflow": SettingMeta(
|
||||
"db_pool_max_overflow", "Database", "DB pool max overflow",
|
||||
restart_required=True,
|
||||
),
|
||||
"db_pool_recycle": SettingMeta(
|
||||
"db_pool_recycle", "Database", "DB pool recycle", restart_required=True
|
||||
),
|
||||
"chat_max_concurrent": SettingMeta(
|
||||
"chat_max_concurrent", "Database", "Chat max concurrent"
|
||||
),
|
||||
# --- LLM (aipi) ---
|
||||
"llm_base_url": SettingMeta(
|
||||
"llm_base_url", "LLM (aipi)", "LLM base URL"
|
||||
),
|
||||
"llm_api_key": SettingMeta(
|
||||
"llm_api_key", "LLM (aipi)", "LLM API key", secret=True
|
||||
),
|
||||
"llm_chat_model": SettingMeta(
|
||||
"llm_chat_model", "LLM (aipi)", "LLM chat model"
|
||||
),
|
||||
"llm_embed_model": SettingMeta(
|
||||
"llm_embed_model", "LLM (aipi)", "LLM embed model"
|
||||
),
|
||||
"llm_summary_model": SettingMeta(
|
||||
"llm_summary_model", "LLM (aipi)", "LLM summary model"
|
||||
),
|
||||
"stream_thinking": SettingMeta(
|
||||
"stream_thinking", "LLM (aipi)", "Stream thinking"
|
||||
),
|
||||
"llm_retries": SettingMeta("llm_retries", "LLM (aipi)", "LLM retries"),
|
||||
"llm_retry_delay": SettingMeta(
|
||||
"llm_retry_delay", "LLM (aipi)", "LLM retry delay"
|
||||
),
|
||||
"llm_timeout": SettingMeta("llm_timeout", "LLM (aipi)", "LLM timeout (s)"),
|
||||
# --- Chat history ---
|
||||
"history_max_turns": SettingMeta(
|
||||
"history_max_turns", "Chat history", "History max turns"
|
||||
),
|
||||
"history_max_chars": SettingMeta(
|
||||
"history_max_chars", "Chat history", "History max chars"
|
||||
),
|
||||
# --- Retrieval & honesty ---
|
||||
"embedding_dim": SettingMeta(
|
||||
"embedding_dim", "Retrieval & honesty", "Embedding dim",
|
||||
overridable=False,
|
||||
),
|
||||
"top_n_docs": SettingMeta("top_n_docs", "Retrieval & honesty", "Top N docs"),
|
||||
"relevance_threshold": SettingMeta(
|
||||
"relevance_threshold", "Retrieval & honesty", "Relevance threshold"
|
||||
),
|
||||
"lexical_support_floor": SettingMeta(
|
||||
"lexical_support_floor", "Retrieval & honesty", "Lexical support floor"
|
||||
),
|
||||
"source_usefulness_floor": SettingMeta(
|
||||
"source_usefulness_floor",
|
||||
"Retrieval & honesty",
|
||||
"Source usefulness floor",
|
||||
),
|
||||
"related_max_docs": SettingMeta(
|
||||
"related_max_docs", "Retrieval & honesty", "Related max docs"
|
||||
),
|
||||
"suggested_docs": SettingMeta(
|
||||
"suggested_docs", "Retrieval & honesty", "Suggested docs"
|
||||
),
|
||||
"suggestion_preview_chars": SettingMeta(
|
||||
"suggestion_preview_chars",
|
||||
"Retrieval & honesty",
|
||||
"Suggestion preview chars",
|
||||
),
|
||||
"hybrid_vector_candidates": SettingMeta(
|
||||
"hybrid_vector_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid vector candidates",
|
||||
),
|
||||
"hybrid_lexical_candidates": SettingMeta(
|
||||
"hybrid_lexical_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid lexical candidates",
|
||||
),
|
||||
"rrf_k": SettingMeta("rrf_k", "Retrieval & honesty", "RRF k"),
|
||||
"recency_boost": SettingMeta(
|
||||
"recency_boost", "Retrieval & honesty", "Recency boost"
|
||||
),
|
||||
"recency_half_life_days": SettingMeta(
|
||||
"recency_half_life_days",
|
||||
"Retrieval & honesty",
|
||||
"Recency half-life days",
|
||||
),
|
||||
"name_hit_bonus": SettingMeta(
|
||||
"name_hit_bonus", "Retrieval & honesty", "Name hit bonus"
|
||||
),
|
||||
# --- Chunking & embedding ---
|
||||
"chunk_target_chars": SettingMeta(
|
||||
"chunk_target_chars", "Chunking & embedding", "Chunk target chars"
|
||||
),
|
||||
"chunk_overlap_chars": SettingMeta(
|
||||
"chunk_overlap_chars", "Chunking & embedding", "Chunk overlap chars"
|
||||
),
|
||||
"embed_batch_size": SettingMeta(
|
||||
"embed_batch_size", "Chunking & embedding", "Embed batch size"
|
||||
),
|
||||
"embed_question_max_chars": SettingMeta(
|
||||
"embed_question_max_chars",
|
||||
"Chunking & embedding",
|
||||
"Embed question max chars",
|
||||
),
|
||||
# --- Prompt budgets ---
|
||||
"max_output_tokens": SettingMeta(
|
||||
"max_output_tokens", "Prompt budgets", "Max output tokens"
|
||||
),
|
||||
"steering_max_chars": SettingMeta(
|
||||
"steering_max_chars", "Prompt budgets", "Steering max chars"
|
||||
),
|
||||
"summary_max_chars": SettingMeta(
|
||||
"summary_max_chars", "Prompt budgets", "Summary max chars"
|
||||
),
|
||||
"kb_overview_max_chars": SettingMeta(
|
||||
"kb_overview_max_chars", "Prompt budgets", "KB overview max chars"
|
||||
),
|
||||
"overview_input_max_chars": SettingMeta(
|
||||
"overview_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Overview input max chars",
|
||||
),
|
||||
"folder_summary_input_max_chars": SettingMeta(
|
||||
"folder_summary_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Folder summary input max chars",
|
||||
),
|
||||
# --- Agent tools ---
|
||||
"agent_max_rounds": SettingMeta(
|
||||
"agent_max_rounds", "Agent tools", "Agent max rounds"
|
||||
),
|
||||
"read_max_chars": SettingMeta(
|
||||
"read_max_chars", "Agent tools", "Read max chars"
|
||||
),
|
||||
# --- Import & sources ---
|
||||
"import_extensions": SettingMeta(
|
||||
"import_extensions", "Import & sources", "Import extensions (CSV)"
|
||||
),
|
||||
"git_sources": SettingMeta(
|
||||
"git_sources", "Import & sources", "Git sources (CSV)"
|
||||
),
|
||||
"sources_dir": SettingMeta(
|
||||
"sources_dir", "Import & sources", "Sources dir"
|
||||
),
|
||||
"upload_dir": SettingMeta("upload_dir", "Import & sources", "Upload dir"),
|
||||
"upload_max_mb": SettingMeta(
|
||||
"upload_max_mb", "Import & sources", "Upload max MB"
|
||||
),
|
||||
# --- Images ---
|
||||
"images": SettingMeta("images", "Images", "Images"),
|
||||
"image_extensions": SettingMeta(
|
||||
"image_extensions", "Images", "Image extensions (CSV)"
|
||||
),
|
||||
"image_dir": SettingMeta("image_dir", "Images", "Image dir"),
|
||||
"chat_image_dir": SettingMeta(
|
||||
"chat_image_dir", "Images", "Chat image dir"
|
||||
),
|
||||
"chat_image_max_mb": SettingMeta(
|
||||
"chat_image_max_mb", "Images", "Chat image max MB"
|
||||
),
|
||||
"answer_images": SettingMeta("answer_images", "Images", "Answer images"),
|
||||
# --- PDF ---
|
||||
"pdf_pictorial": SettingMeta("pdf_pictorial", "PDF", "PDF pictorial"),
|
||||
"pdf_pictorial_min_chars": SettingMeta(
|
||||
"pdf_pictorial_min_chars", "PDF", "PDF pictorial min chars"
|
||||
),
|
||||
"pdf_image_skip_blank": SettingMeta(
|
||||
"pdf_image_skip_blank", "PDF", "PDF image skip blank"
|
||||
),
|
||||
"pdf_page_dir": SettingMeta("pdf_page_dir", "PDF", "PDF page dir"),
|
||||
# --- Docs push ---
|
||||
"docs_repo": SettingMeta("docs_repo", "Docs push", "Docs repo"),
|
||||
"docs_branch": SettingMeta("docs_branch", "Docs push", "Docs branch"),
|
||||
"docs_base_branch": SettingMeta(
|
||||
"docs_base_branch", "Docs push", "Docs base branch"
|
||||
),
|
||||
"docs_work_dir": SettingMeta("docs_work_dir", "Docs push", "Docs work dir"),
|
||||
# --- Admin & security ---
|
||||
"admin_password": SettingMeta(
|
||||
"admin_password", "Admin & security", "Admin password", secret=True
|
||||
),
|
||||
"session_secret": SettingMeta(
|
||||
"session_secret", "Admin & security", "Session secret",
|
||||
secret=True, restart_required=True,
|
||||
),
|
||||
"session_max_age": SettingMeta(
|
||||
"session_max_age", "Admin & security", "Session max age",
|
||||
restart_required=True,
|
||||
),
|
||||
"session_cookie": SettingMeta(
|
||||
"session_cookie", "Admin & security", "Session cookie",
|
||||
restart_required=True,
|
||||
),
|
||||
}
|
||||
|
||||
#: The masked set (LOCKED A4: write-only; ``database_url`` is read-only
|
||||
#: but its URL carries credentials, so it is masked too) — derived,
|
||||
#: never re-listed by hand.
|
||||
SECRETS: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.secret
|
||||
)
|
||||
|
||||
#: The boot-wired set (LOCKED A4: logging, the gzip middleware, the
|
||||
#: static mount, ``create_engine``, ``SessionMiddleware``) — derived.
|
||||
RESTART_REQUIRED: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.restart_required
|
||||
)
|
||||
|
||||
#: The read-only set (LOCKED A3) — derived.
|
||||
NON_OVERRIDABLE: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if not meta.overridable
|
||||
)
|
||||
|
||||
|
||||
def env_var(key: str) -> str:
|
||||
"""The env var for one bare setting key — presentation only (D1).
|
||||
|
||||
The stored key is the bare ``Settings`` field name; the ``BOR_``
|
||||
form is what the panel and the API show, never what is stored
|
||||
(a stored prefix could double up).
|
||||
"""
|
||||
return ENV_PREFIX + key.upper()
|
||||
@@ -0,0 +1,281 @@
|
||||
"""The DB-over-env settings overlay (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "…save it to the database. The database value should
|
||||
override the env var."
|
||||
|
||||
:func:`resolve_settings` layers the ``app_settings`` rows (phase 168
|
||||
task 01: bare ``Settings`` field name → env-native string) over the
|
||||
environment and hands every existing consumer one valid ``Settings`` —
|
||||
through :func:`app.config.get_settings` (which delegates here, lazily,
|
||||
under its unchanged ``lru_cache``). No consumer changes, no restart
|
||||
(D2). The kill switch is env-only: ``BOR_SETTINGS_DB=0`` (read
|
||||
straight from ``os.environ`` per call — it is not a ``Settings`` field,
|
||||
so the escape hatch cannot disable itself).
|
||||
|
||||
Fail loud on write, fail soft on read (LOCKED A5, D5): a write
|
||||
validates the FULL candidate set (task 04); a read that finds an
|
||||
unusable row (hand-edited, or a validator changed in a later release)
|
||||
skips THAT key, logs ``settings override ignored key=… reason=…`` and
|
||||
applies the rest — a DB value must never brick boot, because a
|
||||
bricked app cannot be un-bricked from the panel.
|
||||
|
||||
The overlay's engine is a lazily-created ``NullPool`` engine built from
|
||||
``Settings().database_url`` (never the app's real pool): importing
|
||||
``app.db`` from here would be a cycle (``app.db`` → ``app.config`` at
|
||||
import time), and the pool whose size this table can change must not
|
||||
be the one that reads the table. ``NullPool`` = one connection per
|
||||
resolution, none held.
|
||||
|
||||
A missing ``app_settings`` table (fresh deployment, migrations not
|
||||
applied) means "no overrides" + one DEBUG line per process (K3 — a
|
||||
pre-migration deployment is a normal state); any other DB failure is
|
||||
one WARNING per process naming the exception class ("settings
|
||||
overrides unavailable — env values in effect"). Both "once per process"
|
||||
guards are module-level flags, reset by :func:`reset_state_for_tests`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any, get_origin
|
||||
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.engine import Engine
|
||||
from sqlalchemy.exc import ProgrammingError, SQLAlchemyError
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
from app.config import Settings
|
||||
from app.core.settings_catalog import CATALOG
|
||||
|
||||
logger = logging.getLogger("app.settings")
|
||||
|
||||
#: ``BOR_SETTINGS_DB`` — env-only, read per call from ``os.environ`` so
|
||||
#: tests can flip it; unset/``1`` = on, the house bool off-spellings
|
||||
#: are off. It is NOT a ``Settings`` field, so the escape hatch cannot
|
||||
#: disable itself from the database (D6).
|
||||
_DISABLED_SPELLOS = frozenset({"0", "false", "no", "off"})
|
||||
|
||||
_engine: Engine | None = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators: list[Callable[[], None]] = []
|
||||
|
||||
#: The one ``list[str]`` field (derived — a new list field would join
|
||||
#: automatically); every other field is a scalar or CSV string (K2).
|
||||
LIST_FIELDS: frozenset[str] = frozenset(
|
||||
name
|
||||
for name, field in Settings.model_fields.items()
|
||||
if get_origin(field.annotation) in (list, tuple)
|
||||
)
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
"""The ``BOR_SETTINGS_DB`` kill switch — read per call (D6)."""
|
||||
value = os.environ.get("BOR_SETTINGS_DB", "1")
|
||||
return value.strip().lower() not in _DISABLED_SPELLOS
|
||||
|
||||
|
||||
def _get_engine() -> Engine:
|
||||
"""The overlay's lazily-created ``NullPool`` engine (see module
|
||||
docstring — never the app's real pool: import cycle + the pool
|
||||
whose size this table can change must not read the table)."""
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(
|
||||
Settings().database_url, poolclass=NullPool, pool_pre_ping=True
|
||||
)
|
||||
return _engine
|
||||
|
||||
|
||||
def load_overrides() -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings ORDER BY key`` — fail-soft.
|
||||
|
||||
A missing table (pre-migration deployment) is one DEBUG line per
|
||||
process (K3); any other DB failure is one WARNING per process
|
||||
naming the exception class. Both return ``{}`` — the env values
|
||||
stay in effect, and the NEXT resolution retries (a boot-time miss
|
||||
is never a permanently cached env-only view — the ``create_app``
|
||||
prime, task 02 step 4).
|
||||
"""
|
||||
global _warned_db_failure, _debugged_missing_table
|
||||
try:
|
||||
with _get_engine().connect() as conn:
|
||||
rows = conn.execute(text("SELECT key, value FROM app_settings ORDER BY key"))
|
||||
return {key: value for key, value in rows}
|
||||
except ProgrammingError:
|
||||
# Undefined table (psycopg ``UndefinedTable`` → SQLAlchemy
|
||||
# ProgrammingError): the pre-migration deployment — DEBUG (K3).
|
||||
if not _debugged_missing_table:
|
||||
_debugged_missing_table = True
|
||||
logger.debug(
|
||||
"app_settings table missing — no DB overrides in effect "
|
||||
"(pre-migration deployment, phase 168 K3)"
|
||||
)
|
||||
return {}
|
||||
except (SQLAlchemyError, OSError) as exc:
|
||||
if not _warned_db_failure:
|
||||
_warned_db_failure = True
|
||||
logger.warning(
|
||||
"settings overrides unavailable — env values in effect (%s)",
|
||||
type(exc).__name__,
|
||||
)
|
||||
return {}
|
||||
|
||||
|
||||
def _coerce(field_name: str, raw: str) -> Any:
|
||||
"""Coerce one env-native string row value for the candidate set.
|
||||
|
||||
The one ``list[str]`` field (``suggestions``) is ``json.loads``-ed
|
||||
— it recovers exactly what the env spelling (a JSON array, K2)
|
||||
wrote. Every other field is the raw string: pydantic-settings
|
||||
lax-coerces ``str`` → ``int`` / ``float`` / ``bool``, and init
|
||||
kwargs rank ABOVE env in pydantic-settings — precisely "the
|
||||
database value overrides the env var" (D2). A malformed
|
||||
``suggestions`` JSON raises :class:`ValueError`, which
|
||||
:func:`merge` turns into a per-key skip (LOCKED A5: one bad row
|
||||
costs one setting, never the whole table). The return type is
|
||||
``Any`` on purpose — the value is whatever pydantic will
|
||||
lax-coerce, decided in exactly one place (the ``Settings``
|
||||
validators).
|
||||
"""
|
||||
if field_name in LIST_FIELDS:
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"not a JSON array ({exc.msg})") from exc
|
||||
return raw
|
||||
|
||||
|
||||
def merge(
|
||||
rows: Mapping[str, str],
|
||||
) -> tuple[Settings, list[tuple[str, str]]]:
|
||||
"""Layer ``rows`` over the env base (``Settings()``) and validate.
|
||||
|
||||
Returns ``(settings, skipped)`` — ``skipped`` is the
|
||||
``(key, reason)`` pairs the read-side fail-soft (LOCKED A5)
|
||||
dropped so one unusable row costs one setting, not the whole
|
||||
table. Dropped keys: an orphan row (not a ``Settings`` field —
|
||||
the setting was removed in a later release), a LOCKED A3 key
|
||||
(a hand-inserted ``database_url`` / ``embedding_dim`` row must
|
||||
never hijack the engine / the fixed 768-dim table), a malformed
|
||||
``suggestions`` JSON, and any value the real ``Settings``
|
||||
validators reject (retried per field — ``error["loc"][0]``). Each
|
||||
dropped key is logged once at ERROR
|
||||
(``settings override ignored key=… reason=…``).
|
||||
"""
|
||||
kwargs: dict[str, Any] = {}
|
||||
skipped: list[tuple[str, str]] = []
|
||||
for key, raw in rows.items():
|
||||
if key not in Settings.model_fields:
|
||||
skipped.append((key, "unknown setting (orphan row from a removed field)"))
|
||||
continue
|
||||
if not CATALOG[key].overridable:
|
||||
skipped.append((key, "not DB-overridable (LOCKED A3 — read-only)"))
|
||||
continue
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
skipped.append((key, str(exc)))
|
||||
while True:
|
||||
try:
|
||||
settings = Settings(**kwargs)
|
||||
break
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
bad = str(err["loc"][0])
|
||||
if bad in kwargs:
|
||||
# The offending row is an override — drop it and retry
|
||||
# (LOCKED A5: one bad row costs one setting).
|
||||
kwargs.pop(bad)
|
||||
skipped.append((bad, err["msg"]))
|
||||
continue
|
||||
# The validator blames a field with NO override — its value
|
||||
# comes from env/.env and collides with a cross-field
|
||||
# override (e.g. the env lexical_support_floor above a
|
||||
# DB-overridden relevance_threshold). The env base alone is
|
||||
# always valid (the app booted with it pre-phase), so try
|
||||
# dropping the override(s) until the candidate set is valid
|
||||
# again — one unresolvable collision costs the colliding
|
||||
# row, the rest of the table stands.
|
||||
resolved = False
|
||||
for candidate in list(kwargs):
|
||||
trial = dict(kwargs)
|
||||
trial.pop(candidate)
|
||||
try:
|
||||
settings = Settings(**trial)
|
||||
except ValidationError:
|
||||
continue
|
||||
skipped.append(
|
||||
(
|
||||
candidate,
|
||||
f"cross-field collision with env/base {bad}: {err['msg']} "
|
||||
"— override skipped, env value for the other field stands",
|
||||
)
|
||||
)
|
||||
kwargs = trial
|
||||
resolved = True
|
||||
break
|
||||
if not resolved:
|
||||
# Nothing to drop (no overrides at all) or no single drop
|
||||
# heals the set: the env base alone would boot (pre-phase
|
||||
# proof), so return it — overrides skipped, boot intact.
|
||||
skipped.append(
|
||||
(
|
||||
bad,
|
||||
f"unresolvable cross-field collision: {err['msg']} — "
|
||||
"all overrides skipped, env values in effect",
|
||||
)
|
||||
)
|
||||
settings = Settings()
|
||||
break
|
||||
for key, reason in skipped:
|
||||
logger.error("settings override ignored key=%s reason=%s", key, reason)
|
||||
return settings, skipped
|
||||
|
||||
|
||||
def resolve_settings() -> Settings:
|
||||
"""The whole overlay: env base + kill switch + DB rows.
|
||||
|
||||
Never calls :func:`app.config.get_settings` (recursion guard — that
|
||||
resolver delegates HERE): ``Settings()`` is the env-only entry
|
||||
point, and the kill switch short-circuits before any DB access.
|
||||
"""
|
||||
if not enabled():
|
||||
return Settings()
|
||||
settings, _skipped = merge(load_overrides())
|
||||
return settings
|
||||
|
||||
|
||||
def register_invalidator(fn: Callable[[], None]) -> None:
|
||||
"""Register a runtime reset hook (task 03: the chat LLM client +
|
||||
semaphore). Hooks run on every :func:`app.config.invalidate_settings`
|
||||
— after the cache clear — so the next resolution rebuilds the
|
||||
singletons from the fresh settings."""
|
||||
_invalidators.append(fn)
|
||||
|
||||
|
||||
def run_invalidators() -> None:
|
||||
"""Run the registered reset hooks, in order. A hook that raises is
|
||||
logged, never propagated (a failed reset must not undo a
|
||||
successful save — LOCKED A5's "the save stands")."""
|
||||
for fn in list(_invalidators):
|
||||
try:
|
||||
fn()
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"settings invalidator failed — the save stands, this "
|
||||
"runtime reset did not run"
|
||||
)
|
||||
|
||||
|
||||
def reset_state_for_tests() -> None:
|
||||
"""Clear the module engine + the once-per-process flags + the hook
|
||||
list (the unit suites flip the kill switch and stub the engine)."""
|
||||
global _engine, _warned_db_failure, _debugged_missing_table
|
||||
_engine = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators.clear()
|
||||
+37
-1
@@ -20,7 +20,9 @@ from fastapi.staticfiles import StaticFiles
|
||||
from starlette.middleware.sessions import SessionMiddleware
|
||||
from starlette.responses import FileResponse
|
||||
|
||||
from app.api.admin_settings import router as admin_settings_router
|
||||
from app.api.auth import router as auth_router
|
||||
from app.api.chat import reset_runtime_caches
|
||||
from app.api.chat import router as chat_router
|
||||
from app.api.chat_images import router as chat_images_router
|
||||
from app.api.chats import (
|
||||
@@ -42,7 +44,7 @@ from app.api.suggestions import router as suggestions_router
|
||||
from app.api.sync import router as sync_router
|
||||
from app.api.tokens import router as tokens_router
|
||||
from app.api.ui_settings import router as ui_settings_router
|
||||
from app.config import get_settings
|
||||
from app.config import get_settings, invalidate_settings
|
||||
from app.core.auth import ensure_admin_configured
|
||||
from app.core.caching import configure_caching
|
||||
from app.core.compression import SelectiveGZipMiddleware
|
||||
@@ -50,6 +52,8 @@ from app.core.debugging import configure_debugging
|
||||
from app.core.logging import configure_logging
|
||||
from app.core.response_budget import ResponseBudgetMiddleware
|
||||
from app.core.security_headers import SecurityHeadersMiddleware
|
||||
from app.core.settings_overlay import enabled as settings_db_enabled
|
||||
from app.core.settings_overlay import load_overrides, register_invalidator
|
||||
|
||||
configure_logging()
|
||||
configure_debugging()
|
||||
@@ -97,6 +101,33 @@ def create_app() -> FastAPI:
|
||||
# variable(s) — the app never starts in a half-authenticated state.
|
||||
ensure_admin_configured(settings)
|
||||
|
||||
# Phase 168 (task 02, D2/D3): prime the DB-over-env resolver now
|
||||
# that auth is verified. The module-level ``settings`` above stays
|
||||
# the pre-boot value the middleware wiring below uses (LOCKED A4:
|
||||
# those boot-wired settings apply on the NEXT boot); every consumer
|
||||
# that calls get_settings() from here on resolves DB-over-env — no
|
||||
# restart. invalidate_settings() runs the runtime reset hooks
|
||||
# (task 03) and clears the cache; a boot-time DB miss is fail-soft
|
||||
# (one WARNING, env values in effect) and the next resolution
|
||||
# retries, so the prime can never cache an env-only view forever.
|
||||
invalidate_settings()
|
||||
if settings_db_enabled():
|
||||
get_settings()
|
||||
logger.info("settings overrides loaded count=%d", len(load_overrides()))
|
||||
else:
|
||||
get_settings()
|
||||
logger.info("settings overrides loaded source=env-only disabled")
|
||||
|
||||
# Phase 168 (task 03): the settings panel's Save must retune the
|
||||
# running process, and app/core must never import app/api (PLAN §3's
|
||||
# direction) — so create_app(), the composition root, wires the
|
||||
# reset hook. It runs on every invalidate_settings() after the cache
|
||||
# clear: the next chat turn rebuilds the shared LLMClient and the
|
||||
# concurrency semaphore from the CURRENT settings (no restart), while
|
||||
# in-flight turns keep the client they started with (A12 semantics).
|
||||
register_invalidator(reset_runtime_caches)
|
||||
logger.info("settings invalidator registered name=chat_runtime_caches")
|
||||
|
||||
app = FastAPI(title=settings.app_name, version=settings.app_version)
|
||||
|
||||
# Signed single-admin session cookie (Starlette middleware, itsdangerous
|
||||
@@ -134,6 +165,10 @@ def create_app() -> FastAPI:
|
||||
# single ui_settings row — the Theme tab's persistence) — admin-only
|
||||
# (router-wide require_admin; anonymous AND token users stay 403).
|
||||
app.include_router(ui_settings_router, prefix="/api")
|
||||
# Phase 168 (task 04): the admin settings panel surface (read/write
|
||||
# the app_settings overrides — DB-over-env, live on Save) — admin-only
|
||||
# (router-wide require_admin; a token user stays 403).
|
||||
app.include_router(admin_settings_router, prefix="/api")
|
||||
# Phase 51: the anonymous shared-chat read — NO admin dependency.
|
||||
# /api/shared/<token> is the JSON snapshot; /shared/<token> (the
|
||||
# page route below, registered without a prefix) is the page.
|
||||
@@ -197,6 +232,7 @@ def create_app() -> FastAPI:
|
||||
"/history.html",
|
||||
"/tokens.html", # phase 79 task 06: the Tokens view
|
||||
"/theme.html", # phase 91 task 04: the Theme view (shell route)
|
||||
"/settings.html", # phase 168 task 05: the Settings view (shell route)
|
||||
),
|
||||
)
|
||||
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
|
||||
|
||||
@@ -70,6 +70,13 @@ Data model — see ``.agents/PLAN.md`` §Data Model:
|
||||
``lite`` summary the drill-down
|
||||
``ls`` shows next to each folder (phase 94;
|
||||
``folder_path = ""`` = the source root).
|
||||
* ``app_settings`` — one row per admin setting override (phase
|
||||
168, TODO.md L3): the Settings panel persists
|
||||
what used to be ``.env``-only ``BOR_*`` knobs
|
||||
here, keyed by the BARE ``Settings`` field
|
||||
name, the value in its env-native string form;
|
||||
the row WINS over the environment (resolved
|
||||
inside ``app.config.get_settings``).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -649,3 +656,44 @@ class FolderSummary(Base):
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), onupdate=func.now()
|
||||
)
|
||||
|
||||
|
||||
class AppSetting(Base):
|
||||
"""One admin setting override (phase 168, task 01).
|
||||
|
||||
``TODO.md`` L3: the admin Settings panel persists what used to be
|
||||
``.env``-only knobs here, and the row WINS over the environment
|
||||
(resolved by ``app.core.settings_overlay``, consumed by
|
||||
``app.config.get_settings``). The narrow key/value shape is the
|
||||
whole design (phase 168 D1): one row per setting, the BARE field
|
||||
name as the key, the value in the SAME textual form the env var
|
||||
would carry — so "anything in .env" is editable for all 75
|
||||
settings with ZERO per-field DDL (no typed columns, no wide row),
|
||||
and coercion/validation stays in exactly one place (the pydantic
|
||||
validators, phase 168 A5). A DELETE of the row is the reset
|
||||
("use the env value again") — never a NULL/empty value, so the
|
||||
resolver never has to distinguish "no override" from "override to
|
||||
empty".
|
||||
|
||||
No index beyond the PK: the whole table is read by primary key
|
||||
order in ONE ``SELECT`` per settings resolution (the
|
||||
``ui_settings`` "display fact, never a query surface" note, 0027).
|
||||
"""
|
||||
|
||||
__tablename__ = "app_settings"
|
||||
|
||||
#: The BARE ``Settings`` field name (``relevance_threshold``) — never
|
||||
#: the ``BOR_…`` form: the prefix is presentation
|
||||
#: (``"BOR_" + key.upper()``), so a stored prefix could double up.
|
||||
key: Mapped[str] = mapped_column(String(100), primary_key=True)
|
||||
#: The value in the SAME textual form the env var would carry
|
||||
#: (``"0.55"``, ``"true"``, ``"md,txt"``, a JSON array for the one
|
||||
#: ``list[str]`` field) — coercion and validation belong to
|
||||
#: pydantic, in exactly one place (phase 168 D1/A5). NOT NULL:
|
||||
#: "clear the override" is a DELETE, never an empty value.
|
||||
value: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
#: When the admin last saved it (the ``folder_summaries`` /
|
||||
#: ``kb_overview.updated_at`` house style: server-side now()).
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
@@ -1990,3 +1990,93 @@ class UiSettingsOut(BaseModel):
|
||||
accent_bg: str
|
||||
accent_ink: str
|
||||
accent_line: str
|
||||
|
||||
|
||||
class AdminSettingItem(BaseModel):
|
||||
"""One row of the admin settings catalogue
|
||||
(``GET``/``PUT``/``DELETE /api/admin/settings[/{key}]``, phase 168,
|
||||
task 04).
|
||||
|
||||
One entry per ``Settings`` field (75 today — the catalogue is
|
||||
completeness-pinned, ``tests/unit/test_settings_catalog.py``), in
|
||||
``GROUPS`` order: the field's metadata (``group`` + ``label`` + the
|
||||
three capability flags), where its EFFECTIVE value comes from
|
||||
(``source``: ``db`` = an overridable row is in effect, ``env`` = the
|
||||
``BOR_*`` var is in the process environment, ``default`` = the code
|
||||
default), and the value itself — ``value`` in the SAME env-native
|
||||
string form the env var would carry (bool → ``"true"``/``"false"``,
|
||||
the one ``list[str]`` field → a JSON array, everything else
|
||||
``str()``; the stringification lives in the router, D1), ``None``
|
||||
while masked. LOCKED A4: a ``secret`` field is write-only —
|
||||
``masked=True``, ``value=None``, ``db_value=None`` and
|
||||
``has_value`` (is a non-empty value in effect at all) are all the
|
||||
read side ever reports; the plaintext appears in no response.
|
||||
"""
|
||||
|
||||
key: str
|
||||
env_var: str
|
||||
group: str
|
||||
label: str
|
||||
type: Literal["int", "float", "bool", "str", "json"]
|
||||
secret: bool
|
||||
restart_required: bool
|
||||
overridable: bool
|
||||
source: Literal["db", "env", "default"]
|
||||
#: The effective value in env-native string form (``None`` masked).
|
||||
value: str | None
|
||||
#: Locked A4: ``True`` for a write-only field — ``value`` is ``None``
|
||||
#: while masked and the plaintext is in no response.
|
||||
masked: bool
|
||||
#: For a masked field: is a non-empty value in effect at all.
|
||||
has_value: bool
|
||||
#: An (allowed) row exists in ``app_settings`` for this key.
|
||||
has_override: bool
|
||||
#: The stored string while it is a stored non-secret override; ``None``
|
||||
#: otherwise (a masked or non-overridable row never exposes its text).
|
||||
db_value: str | None
|
||||
#: The code default, stringified (``""`` for a masked field).
|
||||
default: str
|
||||
|
||||
|
||||
class AdminSettingsTotals(BaseModel):
|
||||
"""The catalogue's footer counts (phase 168, task 04): how many
|
||||
settings there are and how many an ``app_settings`` row currently
|
||||
overrides (in effect — ``source == "db"``)."""
|
||||
|
||||
count: int
|
||||
overridden: int
|
||||
|
||||
|
||||
class AdminSettingsOut(BaseModel):
|
||||
"""``GET /api/admin/settings`` response (phase 168, task 04): the
|
||||
whole catalogue — one :class:`AdminSettingItem` per ``Settings``
|
||||
field, in ``GROUPS`` order (the stable order the E2E pins
|
||||
against) — plus the footer counts and the ``BOR_SETTINGS_DB``
|
||||
state: while ``overlay_enabled`` is ``false`` the panel shows a
|
||||
banner instead of pretending a save took effect (the kill switch
|
||||
disables the mechanism, not the panel — D6)."""
|
||||
|
||||
items: list[AdminSettingItem]
|
||||
totals: AdminSettingsTotals
|
||||
overlay_enabled: bool
|
||||
|
||||
|
||||
class AdminSettingIn(BaseModel):
|
||||
"""``PUT /api/admin/settings/{key}`` body (phase 168, task 04):
|
||||
always a string — the value in the env-native string form (D1
|
||||
stores strings; coercion and validation belong to pydantic's
|
||||
``Settings`` validators, in exactly one place, A5). An empty
|
||||
string for a non-secret field is a legitimate value (e.g. clearing
|
||||
``git_sources``); an empty string for a SECRET field is a 422 from
|
||||
the router (an empty API key / session secret is a self-lockout —
|
||||
``DELETE`` is the explicit clear)."""
|
||||
|
||||
value: str
|
||||
|
||||
|
||||
class AdminSettingsCleared(BaseModel):
|
||||
"""``DELETE /api/admin/settings`` response (phase 168, task 04):
|
||||
how many ``app_settings`` rows the "Clear all overrides" action
|
||||
deleted."""
|
||||
|
||||
cleared: int
|
||||
|
||||
Reference in New Issue
Block a user