phase: 168_admin_settings_panel
**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
This commit is contained in:
@@ -0,0 +1,350 @@
|
||||
"""The admin Settings panel's catalogue (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "I want an admin panel that can edit anything in .env
|
||||
and save it to the database."
|
||||
|
||||
This module is the single source of truth for what the panel shows and
|
||||
what the DB-over-env overlay (``app.core.settings_overlay``) is allowed
|
||||
to do with each of the ``Settings`` fields:
|
||||
|
||||
* :data:`CATALOG` maps every ``Settings`` field name to its
|
||||
presentation metadata (``group`` + ``label``) and its capability
|
||||
flags:
|
||||
|
||||
- ``secret`` — write-only (LOCKED A4): the API reports ``is_set``,
|
||||
never the plaintext; a saved value is masked on read.
|
||||
- ``restart_required`` — boot-wired (LOCKED A4): the middleware /
|
||||
engine / logging / static mount read it once at boot, so the
|
||||
override applies on the NEXT boot (the panel shows the badge).
|
||||
- ``overridable`` — ``False`` for LOCKED A3's ``database_url``
|
||||
(it builds the very engine that would read the override — a
|
||||
chicken-and-egg brick) and ``embedding_dim`` (A6 fixes
|
||||
``chunks.embedding`` at table-creation; a dim override would
|
||||
corrupt retrieval).
|
||||
|
||||
The panel (task 05) is GENERATED from this catalogue — never
|
||||
hand-written — so a new ``Settings`` field appears there only once
|
||||
it has a :class:`SettingMeta`; the completeness pin
|
||||
(``tests/unit/test_settings_catalog.py``: ``set(CATALOG) ==
|
||||
set(Settings.model_fields)``) fails the build the moment a field is
|
||||
added without metadata, so it can never silently disappear from the
|
||||
panel.
|
||||
* :data:`GROUPS` is the display order of the groups — it mirrors
|
||||
``app/config.py``'s own section comments (App, HTTP response
|
||||
compression / byte budget, UI customization, Database, LLM, Chat
|
||||
history, RAG tuning, …).
|
||||
* :data:`SECRETS`, :data:`RESTART_REQUIRED` and
|
||||
:data:`NON_OVERRIDABLE` are DERIVED from :data:`CATALOG` — never
|
||||
re-listed by hand, so they cannot drift.
|
||||
|
||||
:func:`env_var` keeps the ``BOR_`` prefix a pure presentation concern
|
||||
(D1): the stored key is the bare ``Settings`` field name
|
||||
(``relevance_threshold``), ``"BOR_" + key.upper()`` is what the panel
|
||||
and the API show, and a stored prefix could double up — so it is
|
||||
derived, never stored.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SettingMeta:
|
||||
"""One settings field's panel metadata + capability flags."""
|
||||
|
||||
key: str
|
||||
group: str
|
||||
label: str
|
||||
#: Write-only (LOCKED A4) — masked in responses, ``is_set`` only.
|
||||
secret: bool = False
|
||||
#: Boot-wired (LOCKED A4) — applies on the next boot.
|
||||
restart_required: bool = False
|
||||
#: ``False`` = read-only row (LOCKED A3).
|
||||
overridable: bool = True
|
||||
|
||||
|
||||
#: The ``BOR_`` prefix is presentation only (D1) — see :func:`env_var`.
|
||||
ENV_PREFIX = "BOR_"
|
||||
|
||||
#: Display order of the groups (mirrors ``app/config.py``'s section
|
||||
#: comments); :data:`CATALOG` is keyed in the same order.
|
||||
GROUPS: tuple[str, ...] = (
|
||||
"App & branding",
|
||||
"HTTP & logging",
|
||||
"Database",
|
||||
"LLM (aipi)",
|
||||
"Chat history",
|
||||
"Retrieval & honesty",
|
||||
"Chunking & embedding",
|
||||
"Prompt budgets",
|
||||
"Agent tools",
|
||||
"Import & sources",
|
||||
"Images",
|
||||
"PDF",
|
||||
"Docs push",
|
||||
"Admin & security",
|
||||
)
|
||||
|
||||
#: Every ``Settings`` field, in :data:`GROUPS` display order. Completeness
|
||||
#: is pinned by ``tests/unit/test_settings_catalog.py`` (the diff is in
|
||||
#: the failure message — a new setting without metadata fails the build).
|
||||
CATALOG: dict[str, SettingMeta] = {
|
||||
# --- App & branding ---
|
||||
"app_name": SettingMeta("app_name", "App & branding", "App name"),
|
||||
"app_version": SettingMeta("app_version", "App & branding", "App version"),
|
||||
"environment": SettingMeta("environment", "App & branding", "Environment"),
|
||||
"input_placeholder": SettingMeta(
|
||||
"input_placeholder", "App & branding", "Input placeholder"
|
||||
),
|
||||
"footer_text": SettingMeta("footer_text", "App & branding", "Footer text"),
|
||||
"suggestions": SettingMeta(
|
||||
"suggestions", "App & branding", "Suggestions (JSON array)"
|
||||
),
|
||||
# --- HTTP & logging ---
|
||||
"log_level": SettingMeta(
|
||||
"log_level", "HTTP & logging", "Log level", restart_required=True
|
||||
),
|
||||
"log_large_response_bytes": SettingMeta(
|
||||
"log_large_response_bytes",
|
||||
"HTTP & logging",
|
||||
"Log large response bytes",
|
||||
),
|
||||
"compress_min_bytes": SettingMeta(
|
||||
"compress_min_bytes", "HTTP & logging", "Compress min bytes",
|
||||
restart_required=True,
|
||||
),
|
||||
"static_dir": SettingMeta(
|
||||
"static_dir", "HTTP & logging", "Static dir", restart_required=True
|
||||
),
|
||||
# --- Database ---
|
||||
"database_url": SettingMeta(
|
||||
"database_url", "Database", "Database URL",
|
||||
secret=True, overridable=False,
|
||||
),
|
||||
"db_pool_size": SettingMeta(
|
||||
"db_pool_size", "Database", "DB pool size", restart_required=True
|
||||
),
|
||||
"db_pool_max_overflow": SettingMeta(
|
||||
"db_pool_max_overflow", "Database", "DB pool max overflow",
|
||||
restart_required=True,
|
||||
),
|
||||
"db_pool_recycle": SettingMeta(
|
||||
"db_pool_recycle", "Database", "DB pool recycle", restart_required=True
|
||||
),
|
||||
"chat_max_concurrent": SettingMeta(
|
||||
"chat_max_concurrent", "Database", "Chat max concurrent"
|
||||
),
|
||||
# --- LLM (aipi) ---
|
||||
"llm_base_url": SettingMeta(
|
||||
"llm_base_url", "LLM (aipi)", "LLM base URL"
|
||||
),
|
||||
"llm_api_key": SettingMeta(
|
||||
"llm_api_key", "LLM (aipi)", "LLM API key", secret=True
|
||||
),
|
||||
"llm_chat_model": SettingMeta(
|
||||
"llm_chat_model", "LLM (aipi)", "LLM chat model"
|
||||
),
|
||||
"llm_embed_model": SettingMeta(
|
||||
"llm_embed_model", "LLM (aipi)", "LLM embed model"
|
||||
),
|
||||
"llm_summary_model": SettingMeta(
|
||||
"llm_summary_model", "LLM (aipi)", "LLM summary model"
|
||||
),
|
||||
"stream_thinking": SettingMeta(
|
||||
"stream_thinking", "LLM (aipi)", "Stream thinking"
|
||||
),
|
||||
"llm_retries": SettingMeta("llm_retries", "LLM (aipi)", "LLM retries"),
|
||||
"llm_retry_delay": SettingMeta(
|
||||
"llm_retry_delay", "LLM (aipi)", "LLM retry delay"
|
||||
),
|
||||
"llm_timeout": SettingMeta("llm_timeout", "LLM (aipi)", "LLM timeout (s)"),
|
||||
# --- Chat history ---
|
||||
"history_max_turns": SettingMeta(
|
||||
"history_max_turns", "Chat history", "History max turns"
|
||||
),
|
||||
"history_max_chars": SettingMeta(
|
||||
"history_max_chars", "Chat history", "History max chars"
|
||||
),
|
||||
# --- Retrieval & honesty ---
|
||||
"embedding_dim": SettingMeta(
|
||||
"embedding_dim", "Retrieval & honesty", "Embedding dim",
|
||||
overridable=False,
|
||||
),
|
||||
"top_n_docs": SettingMeta("top_n_docs", "Retrieval & honesty", "Top N docs"),
|
||||
"relevance_threshold": SettingMeta(
|
||||
"relevance_threshold", "Retrieval & honesty", "Relevance threshold"
|
||||
),
|
||||
"lexical_support_floor": SettingMeta(
|
||||
"lexical_support_floor", "Retrieval & honesty", "Lexical support floor"
|
||||
),
|
||||
"source_usefulness_floor": SettingMeta(
|
||||
"source_usefulness_floor",
|
||||
"Retrieval & honesty",
|
||||
"Source usefulness floor",
|
||||
),
|
||||
"related_max_docs": SettingMeta(
|
||||
"related_max_docs", "Retrieval & honesty", "Related max docs"
|
||||
),
|
||||
"suggested_docs": SettingMeta(
|
||||
"suggested_docs", "Retrieval & honesty", "Suggested docs"
|
||||
),
|
||||
"suggestion_preview_chars": SettingMeta(
|
||||
"suggestion_preview_chars",
|
||||
"Retrieval & honesty",
|
||||
"Suggestion preview chars",
|
||||
),
|
||||
"hybrid_vector_candidates": SettingMeta(
|
||||
"hybrid_vector_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid vector candidates",
|
||||
),
|
||||
"hybrid_lexical_candidates": SettingMeta(
|
||||
"hybrid_lexical_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid lexical candidates",
|
||||
),
|
||||
"rrf_k": SettingMeta("rrf_k", "Retrieval & honesty", "RRF k"),
|
||||
"recency_boost": SettingMeta(
|
||||
"recency_boost", "Retrieval & honesty", "Recency boost"
|
||||
),
|
||||
"recency_half_life_days": SettingMeta(
|
||||
"recency_half_life_days",
|
||||
"Retrieval & honesty",
|
||||
"Recency half-life days",
|
||||
),
|
||||
"name_hit_bonus": SettingMeta(
|
||||
"name_hit_bonus", "Retrieval & honesty", "Name hit bonus"
|
||||
),
|
||||
# --- Chunking & embedding ---
|
||||
"chunk_target_chars": SettingMeta(
|
||||
"chunk_target_chars", "Chunking & embedding", "Chunk target chars"
|
||||
),
|
||||
"chunk_overlap_chars": SettingMeta(
|
||||
"chunk_overlap_chars", "Chunking & embedding", "Chunk overlap chars"
|
||||
),
|
||||
"embed_batch_size": SettingMeta(
|
||||
"embed_batch_size", "Chunking & embedding", "Embed batch size"
|
||||
),
|
||||
"embed_question_max_chars": SettingMeta(
|
||||
"embed_question_max_chars",
|
||||
"Chunking & embedding",
|
||||
"Embed question max chars",
|
||||
),
|
||||
# --- Prompt budgets ---
|
||||
"max_output_tokens": SettingMeta(
|
||||
"max_output_tokens", "Prompt budgets", "Max output tokens"
|
||||
),
|
||||
"steering_max_chars": SettingMeta(
|
||||
"steering_max_chars", "Prompt budgets", "Steering max chars"
|
||||
),
|
||||
"summary_max_chars": SettingMeta(
|
||||
"summary_max_chars", "Prompt budgets", "Summary max chars"
|
||||
),
|
||||
"kb_overview_max_chars": SettingMeta(
|
||||
"kb_overview_max_chars", "Prompt budgets", "KB overview max chars"
|
||||
),
|
||||
"overview_input_max_chars": SettingMeta(
|
||||
"overview_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Overview input max chars",
|
||||
),
|
||||
"folder_summary_input_max_chars": SettingMeta(
|
||||
"folder_summary_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Folder summary input max chars",
|
||||
),
|
||||
# --- Agent tools ---
|
||||
"agent_max_rounds": SettingMeta(
|
||||
"agent_max_rounds", "Agent tools", "Agent max rounds"
|
||||
),
|
||||
"read_max_chars": SettingMeta(
|
||||
"read_max_chars", "Agent tools", "Read max chars"
|
||||
),
|
||||
# --- Import & sources ---
|
||||
"import_extensions": SettingMeta(
|
||||
"import_extensions", "Import & sources", "Import extensions (CSV)"
|
||||
),
|
||||
"git_sources": SettingMeta(
|
||||
"git_sources", "Import & sources", "Git sources (CSV)"
|
||||
),
|
||||
"sources_dir": SettingMeta(
|
||||
"sources_dir", "Import & sources", "Sources dir"
|
||||
),
|
||||
"upload_dir": SettingMeta("upload_dir", "Import & sources", "Upload dir"),
|
||||
"upload_max_mb": SettingMeta(
|
||||
"upload_max_mb", "Import & sources", "Upload max MB"
|
||||
),
|
||||
# --- Images ---
|
||||
"images": SettingMeta("images", "Images", "Images"),
|
||||
"image_extensions": SettingMeta(
|
||||
"image_extensions", "Images", "Image extensions (CSV)"
|
||||
),
|
||||
"image_dir": SettingMeta("image_dir", "Images", "Image dir"),
|
||||
"chat_image_dir": SettingMeta(
|
||||
"chat_image_dir", "Images", "Chat image dir"
|
||||
),
|
||||
"chat_image_max_mb": SettingMeta(
|
||||
"chat_image_max_mb", "Images", "Chat image max MB"
|
||||
),
|
||||
"answer_images": SettingMeta("answer_images", "Images", "Answer images"),
|
||||
# --- PDF ---
|
||||
"pdf_pictorial": SettingMeta("pdf_pictorial", "PDF", "PDF pictorial"),
|
||||
"pdf_pictorial_min_chars": SettingMeta(
|
||||
"pdf_pictorial_min_chars", "PDF", "PDF pictorial min chars"
|
||||
),
|
||||
"pdf_image_skip_blank": SettingMeta(
|
||||
"pdf_image_skip_blank", "PDF", "PDF image skip blank"
|
||||
),
|
||||
"pdf_page_dir": SettingMeta("pdf_page_dir", "PDF", "PDF page dir"),
|
||||
# --- Docs push ---
|
||||
"docs_repo": SettingMeta("docs_repo", "Docs push", "Docs repo"),
|
||||
"docs_branch": SettingMeta("docs_branch", "Docs push", "Docs branch"),
|
||||
"docs_base_branch": SettingMeta(
|
||||
"docs_base_branch", "Docs push", "Docs base branch"
|
||||
),
|
||||
"docs_work_dir": SettingMeta("docs_work_dir", "Docs push", "Docs work dir"),
|
||||
# --- Admin & security ---
|
||||
"admin_password": SettingMeta(
|
||||
"admin_password", "Admin & security", "Admin password", secret=True
|
||||
),
|
||||
"session_secret": SettingMeta(
|
||||
"session_secret", "Admin & security", "Session secret",
|
||||
secret=True, restart_required=True,
|
||||
),
|
||||
"session_max_age": SettingMeta(
|
||||
"session_max_age", "Admin & security", "Session max age",
|
||||
restart_required=True,
|
||||
),
|
||||
"session_cookie": SettingMeta(
|
||||
"session_cookie", "Admin & security", "Session cookie",
|
||||
restart_required=True,
|
||||
),
|
||||
}
|
||||
|
||||
#: The masked set (LOCKED A4: write-only; ``database_url`` is read-only
|
||||
#: but its URL carries credentials, so it is masked too) — derived,
|
||||
#: never re-listed by hand.
|
||||
SECRETS: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.secret
|
||||
)
|
||||
|
||||
#: The boot-wired set (LOCKED A4: logging, the gzip middleware, the
|
||||
#: static mount, ``create_engine``, ``SessionMiddleware``) — derived.
|
||||
RESTART_REQUIRED: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.restart_required
|
||||
)
|
||||
|
||||
#: The read-only set (LOCKED A3) — derived.
|
||||
NON_OVERRIDABLE: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if not meta.overridable
|
||||
)
|
||||
|
||||
|
||||
def env_var(key: str) -> str:
|
||||
"""The env var for one bare setting key — presentation only (D1).
|
||||
|
||||
The stored key is the bare ``Settings`` field name; the ``BOR_``
|
||||
form is what the panel and the API show, never what is stored
|
||||
(a stored prefix could double up).
|
||||
"""
|
||||
return ENV_PREFIX + key.upper()
|
||||
Reference in New Issue
Block a user