phase: 168_admin_settings_panel
**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
This commit is contained in:
@@ -0,0 +1,347 @@
|
||||
"""The admin Settings panel's read/write surface (phase 168, task 04).
|
||||
|
||||
``TODO.md`` L3: "…can edit anything in .env and save it to the
|
||||
database. The database value should override the env var. The change
|
||||
should take effect on clicking 'save'."
|
||||
|
||||
The whole router sits behind :func:`app.core.auth.require_admin`
|
||||
(router-wide ``dependencies`` — the :mod:`app.api.ui_settings` pattern):
|
||||
anonymous callers AND token users get 403 on every route (only the
|
||||
admin retunes the deployment).
|
||||
|
||||
Routes (under ``/api`` via the ``main`` registration):
|
||||
|
||||
* ``GET /api/admin/settings`` — the whole catalogue: one item per
|
||||
``Settings`` field, in ``GROUPS`` order (the stable order the E2E
|
||||
pins against), each with its effective value in env-native string
|
||||
form, a ``db``/``env``/``default`` source badge, and the three
|
||||
capability flags. Reads only; creates nothing.
|
||||
* ``PUT /api/admin/settings/{key}`` — set one value (the body's string
|
||||
is stored as-is in ``app_settings``). Fail loud BEFORE the write
|
||||
(LOCKED A5): the FULL candidate set (every overridable stored row +
|
||||
this key/value, layered over the env base) is run through the
|
||||
real ``Settings`` validators — a bad shape OR a cross-field rule
|
||||
(e.g. ``lexical_support_floor`` above a DB-overridden
|
||||
``relevance_threshold``) is a 422 that names the field and stores
|
||||
NOTHING. Unknown key → 404; a non-overridable key (LOCKED A3) → 422
|
||||
naming the env var; a blank value for a secret → 422 (an empty API
|
||||
key / session secret is a self-lockout; ``DELETE`` is the explicit
|
||||
clear). A successful write commits, calls ``invalidate_settings()``
|
||||
(task 03 makes it live — no restart) and returns the refreshed item.
|
||||
* ``DELETE /api/admin/settings/{key}`` — reset to env: deletes the row
|
||||
(idempotent — deleting an absent key is a success, not a 404),
|
||||
invalidates, returns the item with ``source="env"``/``"default"``.
|
||||
* ``DELETE /api/admin/settings`` — "Clear all overrides": deletes every
|
||||
row, invalidates, ``{"cleared": rowcount}``.
|
||||
|
||||
Three invariants (the unit tests pin them):
|
||||
|
||||
1. **Validate-before-store** — the write path never persists a value
|
||||
the ``Settings`` validators reject (A5);
|
||||
2. **Masked secrets** (LOCKED A4) — a ``secret`` field reports
|
||||
``masked``/``has_value`` only, never the plaintext, in no response
|
||||
body; every write logs one line carrying the key and the action,
|
||||
NEVER the value (AGENTS rule 10);
|
||||
3. **Never write ``.env``** (LOCKED A1) — this module contains no file
|
||||
writes, never shells out and never ``eval``s a value; the env file
|
||||
stays the base, the DB row wins, the reset deletes the row.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import Any, Literal, get_origin
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import delete, func, text
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings, get_settings, invalidate_settings
|
||||
from app.core.auth import require_admin
|
||||
from app.core.settings_catalog import CATALOG, GROUPS, env_var
|
||||
from app.core.settings_overlay import LIST_FIELDS, _coerce, enabled
|
||||
from app.db import get_db
|
||||
from app.models import AppSetting
|
||||
from app.schemas import (
|
||||
AdminSettingIn,
|
||||
AdminSettingItem,
|
||||
AdminSettingsCleared,
|
||||
AdminSettingsOut,
|
||||
AdminSettingsTotals,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("app.api.admin_settings")
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/admin/settings",
|
||||
tags=["admin-settings"],
|
||||
dependencies=[Depends(require_admin)], # phase 168: the settings panel is admin-only
|
||||
)
|
||||
|
||||
|
||||
def _field_type(key: str) -> Literal["int", "float", "bool", "str", "json"]:
|
||||
"""The panel control kind for one field, from its annotation
|
||||
(``bool`` before ``int`` — ``isinstance(True, int)`` is ``True``;
|
||||
the one ``list[str]`` field is ``json``, K2)."""
|
||||
annotation = Settings.model_fields[key].annotation
|
||||
if get_origin(annotation) in (list, tuple):
|
||||
return "json"
|
||||
if annotation is bool:
|
||||
return "bool"
|
||||
if annotation is int:
|
||||
return "int"
|
||||
if annotation is float:
|
||||
return "float"
|
||||
return "str"
|
||||
|
||||
|
||||
def _display(field_name: str, value: Any) -> str:
|
||||
"""Stringify one effective value in the env-native string form
|
||||
(D1: stored strings, one coercion place — the router's display is
|
||||
the mirror of ``settings_overlay._coerce``): bool →
|
||||
``"true"``/``"false"``, the one ``list[str]`` field → a JSON array
|
||||
(what the env spelling would carry, K2), everything else
|
||||
``str()``."""
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
if field_name in LIST_FIELDS:
|
||||
return json.dumps(value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def _rows(db: Session) -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings`` — the whole table in one
|
||||
round trip (the ``ui_settings`` "display fact, never a query
|
||||
surface" note)."""
|
||||
return {key: value for key, value in db.execute(text("SELECT key, value FROM app_settings"))}
|
||||
|
||||
|
||||
def _item(
|
||||
key: str,
|
||||
rows: dict[str, str],
|
||||
effective: Settings,
|
||||
) -> AdminSettingItem:
|
||||
"""One catalogue entry for one field (the stable builder for all
|
||||
three read paths — GET, the PUT/DELETE returns, and the error
|
||||
paths' neighbours)."""
|
||||
meta = CATALOG[key]
|
||||
value = getattr(effective, key)
|
||||
#: Only an OVERRIDABLE row counts as an override: a hand-inserted
|
||||
#: ``database_url`` / ``embedding_dim`` row is never in effect
|
||||
#: (the overlay drops it, LOCKED A3) — it is not an override.
|
||||
has_override = meta.overridable and key in rows
|
||||
#: ``source="db"`` means "in effect": a row the overlay actually
|
||||
#: applies. With the kill switch off, no row is in effect, so the
|
||||
#: badge falls back to env/default (the panel's ``overlay_enabled``
|
||||
#: banner explains the rest — D6).
|
||||
in_effect = has_override and enabled()
|
||||
if in_effect:
|
||||
source: str = "db"
|
||||
elif os.environ.get(env_var(key)) is not None:
|
||||
source = "env"
|
||||
else:
|
||||
source = "default"
|
||||
if meta.secret:
|
||||
# LOCKED A4 — write-only: the plaintext appears in NO response;
|
||||
# the read side reports has_value (is a non-empty value in
|
||||
# effect at all) and nothing else about it.
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=True,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=None,
|
||||
masked=True,
|
||||
has_value=bool(value),
|
||||
has_override=has_override,
|
||||
db_value=None,
|
||||
default="",
|
||||
)
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=False,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=_display(key, value),
|
||||
masked=False,
|
||||
has_value=bool(_display(key, value)),
|
||||
has_override=has_override,
|
||||
db_value=rows.get(key) if has_override else None,
|
||||
default=_display(key, Settings.model_fields[key].default),
|
||||
)
|
||||
|
||||
|
||||
def _validate_candidate(candidate: dict[str, str]) -> None:
|
||||
"""Validate the FULL candidate set before anything is stored
|
||||
(LOCKED A5, D5): every overridable stored row + the new
|
||||
key/value, layered over the env base, through the SAME coercion
|
||||
(``settings_overlay._coerce`` — one place for the string →
|
||||
field-value decision) and the REAL ``Settings`` validators. A bad
|
||||
shape OR a cross-field rule is a 422 that names the field — and
|
||||
nothing is written (the caller writes only on return)."""
|
||||
kwargs: dict[str, Any] = {}
|
||||
for key, raw in candidate.items():
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {key}: {exc}"
|
||||
) from exc
|
||||
try:
|
||||
Settings(**kwargs)
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
field = str(err["loc"][0])
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {field}: {err['msg']}"
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get("", response_model=AdminSettingsOut)
|
||||
def get_catalogue(
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingsOut:
|
||||
"""The whole catalogue (75 items, ``GROUPS`` order — the stable
|
||||
order the E2E pins against).
|
||||
|
||||
Reads the table once, resolves the effective ``Settings`` through
|
||||
``get_settings()`` (the overlay — DB-over-env, task 02), and
|
||||
emits one item per field. GETs log nothing here: the phase-167
|
||||
``ResponseBudgetMiddleware`` already writes the
|
||||
``api method=… bytes=…`` line, and the whole catalogue must stay
|
||||
under ``BOR_LOG_LARGE_RESPONSE_BYTES``' INFO bar (75 items ≈ 20 KB
|
||||
— verified in task 06 with the audit script).
|
||||
"""
|
||||
rows = _rows(db)
|
||||
effective = get_settings()
|
||||
items = [
|
||||
_item(key, rows, effective)
|
||||
for group in GROUPS
|
||||
for key, meta in CATALOG.items()
|
||||
if meta.group == group
|
||||
]
|
||||
return AdminSettingsOut(
|
||||
items=items,
|
||||
totals=AdminSettingsTotals(
|
||||
count=len(items),
|
||||
overridden=sum(1 for item in items if item.source == "db"),
|
||||
),
|
||||
overlay_enabled=enabled(),
|
||||
)
|
||||
|
||||
|
||||
@router.put("/{key}", response_model=AdminSettingItem)
|
||||
def set_value(
|
||||
key: str,
|
||||
payload: AdminSettingIn,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Set one value (store it in ``app_settings`` — env-native string
|
||||
form, D1), make it live (``invalidate_settings()`` — task 03) and
|
||||
return the refreshed item.
|
||||
|
||||
The write path (LOCKED A5, A3, A4): unknown key → 404; a
|
||||
non-overridable key → 422 naming the env var (A3: the engine is
|
||||
built from ``database_url`` — an override is a chicken-and-egg
|
||||
brick; ``chunks.embedding`` is fixed at ``embedding_dim`` by A6,
|
||||
so a dim override would corrupt retrieval); a blank value for a
|
||||
secret → 422 (a self-lockout; ``DELETE`` is the explicit clear);
|
||||
then the FULL candidate set is validated before anything is
|
||||
stored (a 422 names the field and writes nothing).
|
||||
"""
|
||||
meta = CATALOG.get(key)
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
if not meta.overridable:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=f"{env_var(key)} is not settable from the database",
|
||||
)
|
||||
if meta.secret and not payload.value.strip():
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"{env_var(key)} must not be blank — use Reset to fall "
|
||||
"back to the environment"
|
||||
),
|
||||
)
|
||||
# The candidate set: the overridable stored rows + this key/value
|
||||
# (the same drop rules ``settings_overlay.merge`` applies on the
|
||||
# read side — an orphan / non-overridable row can never affect
|
||||
# validation or the effective set).
|
||||
rows = _rows(db)
|
||||
candidate = {
|
||||
k: v for k, v in rows.items() if k in CATALOG and CATALOG[k].overridable
|
||||
}
|
||||
candidate[key] = payload.value
|
||||
_validate_candidate(candidate)
|
||||
# The write: upsert on the PK (INSERT … ON CONFLICT (key) DO
|
||||
# UPDATE — the row's updated_at moves with the save).
|
||||
stmt = pg_insert(AppSetting).values(key=key, value=payload.value)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["key"],
|
||||
set_={"value": stmt.excluded.value, "updated_at": func.now()},
|
||||
)
|
||||
db.execute(stmt)
|
||||
db.commit()
|
||||
# One line per write, never the value (AGENTS rule 10, A4).
|
||||
logger.info("settings action=set key=%s status=200", key)
|
||||
# Make it live (task 03): clear the cache + run the runtime reset
|
||||
# hooks, then report the refreshed item from the fresh settings.
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("/{key}", response_model=AdminSettingItem)
|
||||
def reset_value(
|
||||
key: str,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Reset one key to env (LOCKED A1: the reset is a row DELETE —
|
||||
never an empty value, so the resolver never distinguishes "no
|
||||
override" from "override to empty").
|
||||
|
||||
Idempotent: deleting a key with NO ROW is a success, not a 404
|
||||
(the panel's per-row Reset is safe to double-click — the env
|
||||
value is already in effect; an unknown *setting* key is a 404,
|
||||
like the PUT — a typo in the key is a client bug, and the item
|
||||
shape needs the catalog metadata, so "nothing to reset" can only
|
||||
mean "the row is already gone"). A successful delete logs one
|
||||
value-free line; a no-op double Reset logs nothing.
|
||||
"""
|
||||
if key not in CATALOG:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
result = db.execute(
|
||||
delete(AppSetting).where(AppSetting.key == key).returning(AppSetting.key)
|
||||
)
|
||||
db.commit()
|
||||
if len(result.all()):
|
||||
logger.info("settings action=clear key=%s", key)
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("", response_model=AdminSettingsCleared)
|
||||
def reset_all(db: Session = Depends(get_db)) -> AdminSettingsCleared: # noqa: B008
|
||||
"""The panel's "Clear all overrides": delete every row (the env
|
||||
values win again for all 75 settings), invalidate (task 03) and
|
||||
report how many rows went."""
|
||||
result = db.execute(delete(AppSetting).returning(AppSetting.key))
|
||||
db.commit()
|
||||
count = len(result.all())
|
||||
logger.info("settings action=clear_all count=%d", count)
|
||||
invalidate_settings()
|
||||
return AdminSettingsCleared(cleared=count)
|
||||
@@ -326,6 +326,22 @@ def get_llm() -> LLMClient:
|
||||
return _llm
|
||||
|
||||
|
||||
def reset_runtime_caches() -> None:
|
||||
"""Drop the settings-derived singletons (phase 168, task 03).
|
||||
|
||||
Called by ``invalidate_settings()`` after the admin saves. The next
|
||||
turn rebuilds the client from the CURRENT ``get_settings()`` — an
|
||||
in-flight turn keeps the client it started with (its ``AsyncOpenAI``
|
||||
transport is mid-request; swapping it would kill the stream), which
|
||||
is the same accepted-semantics note as A12's in-process state.
|
||||
``_chat_active`` is deliberately untouched: it counts live turns, and
|
||||
the new ``chat_max_concurrent`` applies as permits are taken.
|
||||
"""
|
||||
global _llm, _chat_semaphore
|
||||
_llm = None
|
||||
_chat_semaphore = None
|
||||
|
||||
|
||||
def _whole_line_number(value: Any) -> bool:
|
||||
"""A ``read`` range argument's whole-line-number guard (phase 131,
|
||||
task 04).
|
||||
|
||||
Reference in New Issue
Block a user