phase: 168_admin_settings_panel
**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
This commit is contained in:
@@ -177,6 +177,7 @@ HTML_PAGES: tuple[str, ...] = (
|
||||
"/history.html", # phase 50: the admin saved-chats page
|
||||
"/tokens.html", # phase 79 task 06: the admin tokens page (shell route)
|
||||
"/theme.html", # phase 91 task 04: the admin theme page (shell route)
|
||||
"/settings.html", # phase 168: the admin settings view (shell route)
|
||||
# phase 51: the shared page's STATIC path (the static mount serves
|
||||
# shared.html at /shared.html as well as the real route serves the
|
||||
# dynamic /shared/<token> — both must carry the no-cache + ?v=
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
"""The admin Settings panel's catalogue (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "I want an admin panel that can edit anything in .env
|
||||
and save it to the database."
|
||||
|
||||
This module is the single source of truth for what the panel shows and
|
||||
what the DB-over-env overlay (``app.core.settings_overlay``) is allowed
|
||||
to do with each of the ``Settings`` fields:
|
||||
|
||||
* :data:`CATALOG` maps every ``Settings`` field name to its
|
||||
presentation metadata (``group`` + ``label``) and its capability
|
||||
flags:
|
||||
|
||||
- ``secret`` — write-only (LOCKED A4): the API reports ``is_set``,
|
||||
never the plaintext; a saved value is masked on read.
|
||||
- ``restart_required`` — boot-wired (LOCKED A4): the middleware /
|
||||
engine / logging / static mount read it once at boot, so the
|
||||
override applies on the NEXT boot (the panel shows the badge).
|
||||
- ``overridable`` — ``False`` for LOCKED A3's ``database_url``
|
||||
(it builds the very engine that would read the override — a
|
||||
chicken-and-egg brick) and ``embedding_dim`` (A6 fixes
|
||||
``chunks.embedding`` at table-creation; a dim override would
|
||||
corrupt retrieval).
|
||||
|
||||
The panel (task 05) is GENERATED from this catalogue — never
|
||||
hand-written — so a new ``Settings`` field appears there only once
|
||||
it has a :class:`SettingMeta`; the completeness pin
|
||||
(``tests/unit/test_settings_catalog.py``: ``set(CATALOG) ==
|
||||
set(Settings.model_fields)``) fails the build the moment a field is
|
||||
added without metadata, so it can never silently disappear from the
|
||||
panel.
|
||||
* :data:`GROUPS` is the display order of the groups — it mirrors
|
||||
``app/config.py``'s own section comments (App, HTTP response
|
||||
compression / byte budget, UI customization, Database, LLM, Chat
|
||||
history, RAG tuning, …).
|
||||
* :data:`SECRETS`, :data:`RESTART_REQUIRED` and
|
||||
:data:`NON_OVERRIDABLE` are DERIVED from :data:`CATALOG` — never
|
||||
re-listed by hand, so they cannot drift.
|
||||
|
||||
:func:`env_var` keeps the ``BOR_`` prefix a pure presentation concern
|
||||
(D1): the stored key is the bare ``Settings`` field name
|
||||
(``relevance_threshold``), ``"BOR_" + key.upper()`` is what the panel
|
||||
and the API show, and a stored prefix could double up — so it is
|
||||
derived, never stored.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SettingMeta:
|
||||
"""One settings field's panel metadata + capability flags."""
|
||||
|
||||
key: str
|
||||
group: str
|
||||
label: str
|
||||
#: Write-only (LOCKED A4) — masked in responses, ``is_set`` only.
|
||||
secret: bool = False
|
||||
#: Boot-wired (LOCKED A4) — applies on the next boot.
|
||||
restart_required: bool = False
|
||||
#: ``False`` = read-only row (LOCKED A3).
|
||||
overridable: bool = True
|
||||
|
||||
|
||||
#: The ``BOR_`` prefix is presentation only (D1) — see :func:`env_var`.
|
||||
ENV_PREFIX = "BOR_"
|
||||
|
||||
#: Display order of the groups (mirrors ``app/config.py``'s section
|
||||
#: comments); :data:`CATALOG` is keyed in the same order.
|
||||
GROUPS: tuple[str, ...] = (
|
||||
"App & branding",
|
||||
"HTTP & logging",
|
||||
"Database",
|
||||
"LLM (aipi)",
|
||||
"Chat history",
|
||||
"Retrieval & honesty",
|
||||
"Chunking & embedding",
|
||||
"Prompt budgets",
|
||||
"Agent tools",
|
||||
"Import & sources",
|
||||
"Images",
|
||||
"PDF",
|
||||
"Docs push",
|
||||
"Admin & security",
|
||||
)
|
||||
|
||||
#: Every ``Settings`` field, in :data:`GROUPS` display order. Completeness
|
||||
#: is pinned by ``tests/unit/test_settings_catalog.py`` (the diff is in
|
||||
#: the failure message — a new setting without metadata fails the build).
|
||||
CATALOG: dict[str, SettingMeta] = {
|
||||
# --- App & branding ---
|
||||
"app_name": SettingMeta("app_name", "App & branding", "App name"),
|
||||
"app_version": SettingMeta("app_version", "App & branding", "App version"),
|
||||
"environment": SettingMeta("environment", "App & branding", "Environment"),
|
||||
"input_placeholder": SettingMeta(
|
||||
"input_placeholder", "App & branding", "Input placeholder"
|
||||
),
|
||||
"footer_text": SettingMeta("footer_text", "App & branding", "Footer text"),
|
||||
"suggestions": SettingMeta(
|
||||
"suggestions", "App & branding", "Suggestions (JSON array)"
|
||||
),
|
||||
# --- HTTP & logging ---
|
||||
"log_level": SettingMeta(
|
||||
"log_level", "HTTP & logging", "Log level", restart_required=True
|
||||
),
|
||||
"log_large_response_bytes": SettingMeta(
|
||||
"log_large_response_bytes",
|
||||
"HTTP & logging",
|
||||
"Log large response bytes",
|
||||
),
|
||||
"compress_min_bytes": SettingMeta(
|
||||
"compress_min_bytes", "HTTP & logging", "Compress min bytes",
|
||||
restart_required=True,
|
||||
),
|
||||
"static_dir": SettingMeta(
|
||||
"static_dir", "HTTP & logging", "Static dir", restart_required=True
|
||||
),
|
||||
# --- Database ---
|
||||
"database_url": SettingMeta(
|
||||
"database_url", "Database", "Database URL",
|
||||
secret=True, overridable=False,
|
||||
),
|
||||
"db_pool_size": SettingMeta(
|
||||
"db_pool_size", "Database", "DB pool size", restart_required=True
|
||||
),
|
||||
"db_pool_max_overflow": SettingMeta(
|
||||
"db_pool_max_overflow", "Database", "DB pool max overflow",
|
||||
restart_required=True,
|
||||
),
|
||||
"db_pool_recycle": SettingMeta(
|
||||
"db_pool_recycle", "Database", "DB pool recycle", restart_required=True
|
||||
),
|
||||
"chat_max_concurrent": SettingMeta(
|
||||
"chat_max_concurrent", "Database", "Chat max concurrent"
|
||||
),
|
||||
# --- LLM (aipi) ---
|
||||
"llm_base_url": SettingMeta(
|
||||
"llm_base_url", "LLM (aipi)", "LLM base URL"
|
||||
),
|
||||
"llm_api_key": SettingMeta(
|
||||
"llm_api_key", "LLM (aipi)", "LLM API key", secret=True
|
||||
),
|
||||
"llm_chat_model": SettingMeta(
|
||||
"llm_chat_model", "LLM (aipi)", "LLM chat model"
|
||||
),
|
||||
"llm_embed_model": SettingMeta(
|
||||
"llm_embed_model", "LLM (aipi)", "LLM embed model"
|
||||
),
|
||||
"llm_summary_model": SettingMeta(
|
||||
"llm_summary_model", "LLM (aipi)", "LLM summary model"
|
||||
),
|
||||
"stream_thinking": SettingMeta(
|
||||
"stream_thinking", "LLM (aipi)", "Stream thinking"
|
||||
),
|
||||
"llm_retries": SettingMeta("llm_retries", "LLM (aipi)", "LLM retries"),
|
||||
"llm_retry_delay": SettingMeta(
|
||||
"llm_retry_delay", "LLM (aipi)", "LLM retry delay"
|
||||
),
|
||||
"llm_timeout": SettingMeta("llm_timeout", "LLM (aipi)", "LLM timeout (s)"),
|
||||
# --- Chat history ---
|
||||
"history_max_turns": SettingMeta(
|
||||
"history_max_turns", "Chat history", "History max turns"
|
||||
),
|
||||
"history_max_chars": SettingMeta(
|
||||
"history_max_chars", "Chat history", "History max chars"
|
||||
),
|
||||
# --- Retrieval & honesty ---
|
||||
"embedding_dim": SettingMeta(
|
||||
"embedding_dim", "Retrieval & honesty", "Embedding dim",
|
||||
overridable=False,
|
||||
),
|
||||
"top_n_docs": SettingMeta("top_n_docs", "Retrieval & honesty", "Top N docs"),
|
||||
"relevance_threshold": SettingMeta(
|
||||
"relevance_threshold", "Retrieval & honesty", "Relevance threshold"
|
||||
),
|
||||
"lexical_support_floor": SettingMeta(
|
||||
"lexical_support_floor", "Retrieval & honesty", "Lexical support floor"
|
||||
),
|
||||
"source_usefulness_floor": SettingMeta(
|
||||
"source_usefulness_floor",
|
||||
"Retrieval & honesty",
|
||||
"Source usefulness floor",
|
||||
),
|
||||
"related_max_docs": SettingMeta(
|
||||
"related_max_docs", "Retrieval & honesty", "Related max docs"
|
||||
),
|
||||
"suggested_docs": SettingMeta(
|
||||
"suggested_docs", "Retrieval & honesty", "Suggested docs"
|
||||
),
|
||||
"suggestion_preview_chars": SettingMeta(
|
||||
"suggestion_preview_chars",
|
||||
"Retrieval & honesty",
|
||||
"Suggestion preview chars",
|
||||
),
|
||||
"hybrid_vector_candidates": SettingMeta(
|
||||
"hybrid_vector_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid vector candidates",
|
||||
),
|
||||
"hybrid_lexical_candidates": SettingMeta(
|
||||
"hybrid_lexical_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid lexical candidates",
|
||||
),
|
||||
"rrf_k": SettingMeta("rrf_k", "Retrieval & honesty", "RRF k"),
|
||||
"recency_boost": SettingMeta(
|
||||
"recency_boost", "Retrieval & honesty", "Recency boost"
|
||||
),
|
||||
"recency_half_life_days": SettingMeta(
|
||||
"recency_half_life_days",
|
||||
"Retrieval & honesty",
|
||||
"Recency half-life days",
|
||||
),
|
||||
"name_hit_bonus": SettingMeta(
|
||||
"name_hit_bonus", "Retrieval & honesty", "Name hit bonus"
|
||||
),
|
||||
# --- Chunking & embedding ---
|
||||
"chunk_target_chars": SettingMeta(
|
||||
"chunk_target_chars", "Chunking & embedding", "Chunk target chars"
|
||||
),
|
||||
"chunk_overlap_chars": SettingMeta(
|
||||
"chunk_overlap_chars", "Chunking & embedding", "Chunk overlap chars"
|
||||
),
|
||||
"embed_batch_size": SettingMeta(
|
||||
"embed_batch_size", "Chunking & embedding", "Embed batch size"
|
||||
),
|
||||
"embed_question_max_chars": SettingMeta(
|
||||
"embed_question_max_chars",
|
||||
"Chunking & embedding",
|
||||
"Embed question max chars",
|
||||
),
|
||||
# --- Prompt budgets ---
|
||||
"max_output_tokens": SettingMeta(
|
||||
"max_output_tokens", "Prompt budgets", "Max output tokens"
|
||||
),
|
||||
"steering_max_chars": SettingMeta(
|
||||
"steering_max_chars", "Prompt budgets", "Steering max chars"
|
||||
),
|
||||
"summary_max_chars": SettingMeta(
|
||||
"summary_max_chars", "Prompt budgets", "Summary max chars"
|
||||
),
|
||||
"kb_overview_max_chars": SettingMeta(
|
||||
"kb_overview_max_chars", "Prompt budgets", "KB overview max chars"
|
||||
),
|
||||
"overview_input_max_chars": SettingMeta(
|
||||
"overview_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Overview input max chars",
|
||||
),
|
||||
"folder_summary_input_max_chars": SettingMeta(
|
||||
"folder_summary_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Folder summary input max chars",
|
||||
),
|
||||
# --- Agent tools ---
|
||||
"agent_max_rounds": SettingMeta(
|
||||
"agent_max_rounds", "Agent tools", "Agent max rounds"
|
||||
),
|
||||
"read_max_chars": SettingMeta(
|
||||
"read_max_chars", "Agent tools", "Read max chars"
|
||||
),
|
||||
# --- Import & sources ---
|
||||
"import_extensions": SettingMeta(
|
||||
"import_extensions", "Import & sources", "Import extensions (CSV)"
|
||||
),
|
||||
"git_sources": SettingMeta(
|
||||
"git_sources", "Import & sources", "Git sources (CSV)"
|
||||
),
|
||||
"sources_dir": SettingMeta(
|
||||
"sources_dir", "Import & sources", "Sources dir"
|
||||
),
|
||||
"upload_dir": SettingMeta("upload_dir", "Import & sources", "Upload dir"),
|
||||
"upload_max_mb": SettingMeta(
|
||||
"upload_max_mb", "Import & sources", "Upload max MB"
|
||||
),
|
||||
# --- Images ---
|
||||
"images": SettingMeta("images", "Images", "Images"),
|
||||
"image_extensions": SettingMeta(
|
||||
"image_extensions", "Images", "Image extensions (CSV)"
|
||||
),
|
||||
"image_dir": SettingMeta("image_dir", "Images", "Image dir"),
|
||||
"chat_image_dir": SettingMeta(
|
||||
"chat_image_dir", "Images", "Chat image dir"
|
||||
),
|
||||
"chat_image_max_mb": SettingMeta(
|
||||
"chat_image_max_mb", "Images", "Chat image max MB"
|
||||
),
|
||||
"answer_images": SettingMeta("answer_images", "Images", "Answer images"),
|
||||
# --- PDF ---
|
||||
"pdf_pictorial": SettingMeta("pdf_pictorial", "PDF", "PDF pictorial"),
|
||||
"pdf_pictorial_min_chars": SettingMeta(
|
||||
"pdf_pictorial_min_chars", "PDF", "PDF pictorial min chars"
|
||||
),
|
||||
"pdf_image_skip_blank": SettingMeta(
|
||||
"pdf_image_skip_blank", "PDF", "PDF image skip blank"
|
||||
),
|
||||
"pdf_page_dir": SettingMeta("pdf_page_dir", "PDF", "PDF page dir"),
|
||||
# --- Docs push ---
|
||||
"docs_repo": SettingMeta("docs_repo", "Docs push", "Docs repo"),
|
||||
"docs_branch": SettingMeta("docs_branch", "Docs push", "Docs branch"),
|
||||
"docs_base_branch": SettingMeta(
|
||||
"docs_base_branch", "Docs push", "Docs base branch"
|
||||
),
|
||||
"docs_work_dir": SettingMeta("docs_work_dir", "Docs push", "Docs work dir"),
|
||||
# --- Admin & security ---
|
||||
"admin_password": SettingMeta(
|
||||
"admin_password", "Admin & security", "Admin password", secret=True
|
||||
),
|
||||
"session_secret": SettingMeta(
|
||||
"session_secret", "Admin & security", "Session secret",
|
||||
secret=True, restart_required=True,
|
||||
),
|
||||
"session_max_age": SettingMeta(
|
||||
"session_max_age", "Admin & security", "Session max age",
|
||||
restart_required=True,
|
||||
),
|
||||
"session_cookie": SettingMeta(
|
||||
"session_cookie", "Admin & security", "Session cookie",
|
||||
restart_required=True,
|
||||
),
|
||||
}
|
||||
|
||||
#: The masked set (LOCKED A4: write-only; ``database_url`` is read-only
|
||||
#: but its URL carries credentials, so it is masked too) — derived,
|
||||
#: never re-listed by hand.
|
||||
SECRETS: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.secret
|
||||
)
|
||||
|
||||
#: The boot-wired set (LOCKED A4: logging, the gzip middleware, the
|
||||
#: static mount, ``create_engine``, ``SessionMiddleware``) — derived.
|
||||
RESTART_REQUIRED: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.restart_required
|
||||
)
|
||||
|
||||
#: The read-only set (LOCKED A3) — derived.
|
||||
NON_OVERRIDABLE: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if not meta.overridable
|
||||
)
|
||||
|
||||
|
||||
def env_var(key: str) -> str:
|
||||
"""The env var for one bare setting key — presentation only (D1).
|
||||
|
||||
The stored key is the bare ``Settings`` field name; the ``BOR_``
|
||||
form is what the panel and the API show, never what is stored
|
||||
(a stored prefix could double up).
|
||||
"""
|
||||
return ENV_PREFIX + key.upper()
|
||||
@@ -0,0 +1,281 @@
|
||||
"""The DB-over-env settings overlay (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "…save it to the database. The database value should
|
||||
override the env var."
|
||||
|
||||
:func:`resolve_settings` layers the ``app_settings`` rows (phase 168
|
||||
task 01: bare ``Settings`` field name → env-native string) over the
|
||||
environment and hands every existing consumer one valid ``Settings`` —
|
||||
through :func:`app.config.get_settings` (which delegates here, lazily,
|
||||
under its unchanged ``lru_cache``). No consumer changes, no restart
|
||||
(D2). The kill switch is env-only: ``BOR_SETTINGS_DB=0`` (read
|
||||
straight from ``os.environ`` per call — it is not a ``Settings`` field,
|
||||
so the escape hatch cannot disable itself).
|
||||
|
||||
Fail loud on write, fail soft on read (LOCKED A5, D5): a write
|
||||
validates the FULL candidate set (task 04); a read that finds an
|
||||
unusable row (hand-edited, or a validator changed in a later release)
|
||||
skips THAT key, logs ``settings override ignored key=… reason=…`` and
|
||||
applies the rest — a DB value must never brick boot, because a
|
||||
bricked app cannot be un-bricked from the panel.
|
||||
|
||||
The overlay's engine is a lazily-created ``NullPool`` engine built from
|
||||
``Settings().database_url`` (never the app's real pool): importing
|
||||
``app.db`` from here would be a cycle (``app.db`` → ``app.config`` at
|
||||
import time), and the pool whose size this table can change must not
|
||||
be the one that reads the table. ``NullPool`` = one connection per
|
||||
resolution, none held.
|
||||
|
||||
A missing ``app_settings`` table (fresh deployment, migrations not
|
||||
applied) means "no overrides" + one DEBUG line per process (K3 — a
|
||||
pre-migration deployment is a normal state); any other DB failure is
|
||||
one WARNING per process naming the exception class ("settings
|
||||
overrides unavailable — env values in effect"). Both "once per process"
|
||||
guards are module-level flags, reset by :func:`reset_state_for_tests`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any, get_origin
|
||||
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.engine import Engine
|
||||
from sqlalchemy.exc import ProgrammingError, SQLAlchemyError
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
from app.config import Settings
|
||||
from app.core.settings_catalog import CATALOG
|
||||
|
||||
logger = logging.getLogger("app.settings")
|
||||
|
||||
#: ``BOR_SETTINGS_DB`` — env-only, read per call from ``os.environ`` so
|
||||
#: tests can flip it; unset/``1`` = on, the house bool off-spellings
|
||||
#: are off. It is NOT a ``Settings`` field, so the escape hatch cannot
|
||||
#: disable itself from the database (D6).
|
||||
_DISABLED_SPELLOS = frozenset({"0", "false", "no", "off"})
|
||||
|
||||
_engine: Engine | None = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators: list[Callable[[], None]] = []
|
||||
|
||||
#: The one ``list[str]`` field (derived — a new list field would join
|
||||
#: automatically); every other field is a scalar or CSV string (K2).
|
||||
LIST_FIELDS: frozenset[str] = frozenset(
|
||||
name
|
||||
for name, field in Settings.model_fields.items()
|
||||
if get_origin(field.annotation) in (list, tuple)
|
||||
)
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
"""The ``BOR_SETTINGS_DB`` kill switch — read per call (D6)."""
|
||||
value = os.environ.get("BOR_SETTINGS_DB", "1")
|
||||
return value.strip().lower() not in _DISABLED_SPELLOS
|
||||
|
||||
|
||||
def _get_engine() -> Engine:
|
||||
"""The overlay's lazily-created ``NullPool`` engine (see module
|
||||
docstring — never the app's real pool: import cycle + the pool
|
||||
whose size this table can change must not read the table)."""
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(
|
||||
Settings().database_url, poolclass=NullPool, pool_pre_ping=True
|
||||
)
|
||||
return _engine
|
||||
|
||||
|
||||
def load_overrides() -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings ORDER BY key`` — fail-soft.
|
||||
|
||||
A missing table (pre-migration deployment) is one DEBUG line per
|
||||
process (K3); any other DB failure is one WARNING per process
|
||||
naming the exception class. Both return ``{}`` — the env values
|
||||
stay in effect, and the NEXT resolution retries (a boot-time miss
|
||||
is never a permanently cached env-only view — the ``create_app``
|
||||
prime, task 02 step 4).
|
||||
"""
|
||||
global _warned_db_failure, _debugged_missing_table
|
||||
try:
|
||||
with _get_engine().connect() as conn:
|
||||
rows = conn.execute(text("SELECT key, value FROM app_settings ORDER BY key"))
|
||||
return {key: value for key, value in rows}
|
||||
except ProgrammingError:
|
||||
# Undefined table (psycopg ``UndefinedTable`` → SQLAlchemy
|
||||
# ProgrammingError): the pre-migration deployment — DEBUG (K3).
|
||||
if not _debugged_missing_table:
|
||||
_debugged_missing_table = True
|
||||
logger.debug(
|
||||
"app_settings table missing — no DB overrides in effect "
|
||||
"(pre-migration deployment, phase 168 K3)"
|
||||
)
|
||||
return {}
|
||||
except (SQLAlchemyError, OSError) as exc:
|
||||
if not _warned_db_failure:
|
||||
_warned_db_failure = True
|
||||
logger.warning(
|
||||
"settings overrides unavailable — env values in effect (%s)",
|
||||
type(exc).__name__,
|
||||
)
|
||||
return {}
|
||||
|
||||
|
||||
def _coerce(field_name: str, raw: str) -> Any:
|
||||
"""Coerce one env-native string row value for the candidate set.
|
||||
|
||||
The one ``list[str]`` field (``suggestions``) is ``json.loads``-ed
|
||||
— it recovers exactly what the env spelling (a JSON array, K2)
|
||||
wrote. Every other field is the raw string: pydantic-settings
|
||||
lax-coerces ``str`` → ``int`` / ``float`` / ``bool``, and init
|
||||
kwargs rank ABOVE env in pydantic-settings — precisely "the
|
||||
database value overrides the env var" (D2). A malformed
|
||||
``suggestions`` JSON raises :class:`ValueError`, which
|
||||
:func:`merge` turns into a per-key skip (LOCKED A5: one bad row
|
||||
costs one setting, never the whole table). The return type is
|
||||
``Any`` on purpose — the value is whatever pydantic will
|
||||
lax-coerce, decided in exactly one place (the ``Settings``
|
||||
validators).
|
||||
"""
|
||||
if field_name in LIST_FIELDS:
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"not a JSON array ({exc.msg})") from exc
|
||||
return raw
|
||||
|
||||
|
||||
def merge(
|
||||
rows: Mapping[str, str],
|
||||
) -> tuple[Settings, list[tuple[str, str]]]:
|
||||
"""Layer ``rows`` over the env base (``Settings()``) and validate.
|
||||
|
||||
Returns ``(settings, skipped)`` — ``skipped`` is the
|
||||
``(key, reason)`` pairs the read-side fail-soft (LOCKED A5)
|
||||
dropped so one unusable row costs one setting, not the whole
|
||||
table. Dropped keys: an orphan row (not a ``Settings`` field —
|
||||
the setting was removed in a later release), a LOCKED A3 key
|
||||
(a hand-inserted ``database_url`` / ``embedding_dim`` row must
|
||||
never hijack the engine / the fixed 768-dim table), a malformed
|
||||
``suggestions`` JSON, and any value the real ``Settings``
|
||||
validators reject (retried per field — ``error["loc"][0]``). Each
|
||||
dropped key is logged once at ERROR
|
||||
(``settings override ignored key=… reason=…``).
|
||||
"""
|
||||
kwargs: dict[str, Any] = {}
|
||||
skipped: list[tuple[str, str]] = []
|
||||
for key, raw in rows.items():
|
||||
if key not in Settings.model_fields:
|
||||
skipped.append((key, "unknown setting (orphan row from a removed field)"))
|
||||
continue
|
||||
if not CATALOG[key].overridable:
|
||||
skipped.append((key, "not DB-overridable (LOCKED A3 — read-only)"))
|
||||
continue
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
skipped.append((key, str(exc)))
|
||||
while True:
|
||||
try:
|
||||
settings = Settings(**kwargs)
|
||||
break
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
bad = str(err["loc"][0])
|
||||
if bad in kwargs:
|
||||
# The offending row is an override — drop it and retry
|
||||
# (LOCKED A5: one bad row costs one setting).
|
||||
kwargs.pop(bad)
|
||||
skipped.append((bad, err["msg"]))
|
||||
continue
|
||||
# The validator blames a field with NO override — its value
|
||||
# comes from env/.env and collides with a cross-field
|
||||
# override (e.g. the env lexical_support_floor above a
|
||||
# DB-overridden relevance_threshold). The env base alone is
|
||||
# always valid (the app booted with it pre-phase), so try
|
||||
# dropping the override(s) until the candidate set is valid
|
||||
# again — one unresolvable collision costs the colliding
|
||||
# row, the rest of the table stands.
|
||||
resolved = False
|
||||
for candidate in list(kwargs):
|
||||
trial = dict(kwargs)
|
||||
trial.pop(candidate)
|
||||
try:
|
||||
settings = Settings(**trial)
|
||||
except ValidationError:
|
||||
continue
|
||||
skipped.append(
|
||||
(
|
||||
candidate,
|
||||
f"cross-field collision with env/base {bad}: {err['msg']} "
|
||||
"— override skipped, env value for the other field stands",
|
||||
)
|
||||
)
|
||||
kwargs = trial
|
||||
resolved = True
|
||||
break
|
||||
if not resolved:
|
||||
# Nothing to drop (no overrides at all) or no single drop
|
||||
# heals the set: the env base alone would boot (pre-phase
|
||||
# proof), so return it — overrides skipped, boot intact.
|
||||
skipped.append(
|
||||
(
|
||||
bad,
|
||||
f"unresolvable cross-field collision: {err['msg']} — "
|
||||
"all overrides skipped, env values in effect",
|
||||
)
|
||||
)
|
||||
settings = Settings()
|
||||
break
|
||||
for key, reason in skipped:
|
||||
logger.error("settings override ignored key=%s reason=%s", key, reason)
|
||||
return settings, skipped
|
||||
|
||||
|
||||
def resolve_settings() -> Settings:
|
||||
"""The whole overlay: env base + kill switch + DB rows.
|
||||
|
||||
Never calls :func:`app.config.get_settings` (recursion guard — that
|
||||
resolver delegates HERE): ``Settings()`` is the env-only entry
|
||||
point, and the kill switch short-circuits before any DB access.
|
||||
"""
|
||||
if not enabled():
|
||||
return Settings()
|
||||
settings, _skipped = merge(load_overrides())
|
||||
return settings
|
||||
|
||||
|
||||
def register_invalidator(fn: Callable[[], None]) -> None:
|
||||
"""Register a runtime reset hook (task 03: the chat LLM client +
|
||||
semaphore). Hooks run on every :func:`app.config.invalidate_settings`
|
||||
— after the cache clear — so the next resolution rebuilds the
|
||||
singletons from the fresh settings."""
|
||||
_invalidators.append(fn)
|
||||
|
||||
|
||||
def run_invalidators() -> None:
|
||||
"""Run the registered reset hooks, in order. A hook that raises is
|
||||
logged, never propagated (a failed reset must not undo a
|
||||
successful save — LOCKED A5's "the save stands")."""
|
||||
for fn in list(_invalidators):
|
||||
try:
|
||||
fn()
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"settings invalidator failed — the save stands, this "
|
||||
"runtime reset did not run"
|
||||
)
|
||||
|
||||
|
||||
def reset_state_for_tests() -> None:
|
||||
"""Clear the module engine + the once-per-process flags + the hook
|
||||
list (the unit suites flip the kill switch and stub the engine)."""
|
||||
global _engine, _warned_db_failure, _debugged_missing_table
|
||||
_engine = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators.clear()
|
||||
Reference in New Issue
Block a user