phase: 168_admin_settings_panel
**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
This commit is contained in:
@@ -1,121 +0,0 @@
|
||||
# Phase 169 — Embedding generations: stamp every vector, measure comparability, re-embed in place
|
||||
|
||||
**Source:** owner request 2026-10-09 — "I'm about to upgrade the embedding model from embedding gemma to embedding gemma 2. The dimensions are the same, but the embeddings may not be comparable. What do I need to do on brain of reese's side?"
|
||||
**Story:** n/a (owner-directed infrastructure work)
|
||||
**Context:** `app/models.py:232-249` (`Chunk`, `embedding: Vector(EMBEDDING_DIM)` at L241 — the ONLY vector column in the schema), `app/config.py:102` (`llm_embed_model`), `:142` (`embedding_dim=768`), `:151` (`relevance_threshold=0.62`), `:160` (`lexical_support_floor=0.35`), `:179` (`source_usefulness_floor=0.35`), `:225` (`embed_question_max_chars`), `:548-578` (the two floor ⇄ threshold validators), `app/rag/llm.py:314-322` (`EmbeddingDimensionError` — the ONLY guard, dim-based), `:213-221` (`_TOKENS_PER_CHAR` / `_REQUEST_TOKEN_BUDGET`, written against the current endpoint's ~1024-token cap), `:346` (`embed()`), `:400` (`embed_one()`), `app/rag/importer.py:1008-1023` (content-chunk write), `:1587-1589` (the `is_summary` `position = -1` chunk), `:51-57` (the phase-118 "content-unchanged work is NOT a KB change" convention), `app/rag/retriever.py:433-455` (`_vector_list`, the brute-force cosine scan), `app/api/docs.py:488-491` (the manual-summary write), `app/api/chat.py:740` (the question embed), `app/api/sync.py:844` (`sync: done`), `app/core/auth.py:83` (`require_admin`), `app/main.py:146-172` (router includes), `app/core/settings_overlay.py` + `app/core/settings_catalog.py` (phase 168: the DB-over-env overlay — a stored override BEATS `.env`), `alembic/versions/0028_app_settings.py` (head — next free: **0029**), `scripts/regen_folder_summaries.py` (the one-shot entrypoint idiom this phase's script mirrors), `scripts/eval_retrieval.py`, `scripts/test_embed_model.py`, `scripts/llm_probe.py`, `tests/fixtures/retrieval_battery.txt` (the 43-line gated battery), `benchmarks/model_benchmarks.csv`, `tests/e2e/mock_llm.py` (`POST /v1/embeddings` → deterministic 768-dim bag-of-words), `tests/e2e/conftest.py:53,64` (the global env pins) and `:105-186` (the `app_server` env contract), `README.md:1408-1410` (the "Embedding dimension mismatch" troubleshooting entry).
|
||||
|
||||
## Objective
|
||||
|
||||
An embedding-model change that KEEPS the dimension (768 → 768) is today invisible and therefore silently destructive: nothing records which model produced a vector, `EmbeddingDimensionError` cannot fire, and cosine scores from two different geometries get compared on one list. This phase makes a vector's generation a fact in the database (`chunks.embedding_model`), makes a mismatch loud in the admin UI and the sync log, ships the two tools that answer "are the old vectors still usable?" (`--compare`, measured in cosine agreement) and "make them usable" (`--apply`, a resumable in-place re-embed over the text already stored), and writes the operator runbook the live upgrade follows.
|
||||
|
||||
## The problem, precisely (the executor reads this, not the chat)
|
||||
|
||||
- **No generation is recorded.** `chunks.embedding` is a bare `Vector(768)`; `grep -rn "embedding_model" app/ alembic/` is empty. A row does not know what made it.
|
||||
- **No guard can fire.** The only loud failure is `EmbeddingDimensionError` (`app/rag/llm.py:314-322`), which compares dimension against `BOR_EMBEDDING_DIM`. Same dims → no error, ever.
|
||||
- **A re-sync re-embeds NOTHING.** `documents.file_hash` matches → the importer returns through `_unchanged_doc` with "ZERO embed" (`app/rag/importer.py:70-84`, `:1391-1403`). Flipping `BOR_LLM_EMBED_MODEL` and pressing Sync leaves 100 % of the corpus in the old space while every QUESTION embeds in the new one (`app/api/chat.py:740`) — retrieval degrades with no error anywhere.
|
||||
- **The thresholds are model-calibrated, not absolute.** `relevance_threshold=0.62` / `lexical_support_floor=0.35` / `source_usefulness_floor=0.35` gate honesty (A8) and citation tiering, and `app/config.py:147-150` records that they were hand-re-tuned on 2026-08-21 *for the current model's* cosine distribution ("the default never discriminated"). A new model moves the whole distribution.
|
||||
- **No ANN index to rebuild** — `pg_indexes` for `chunks` is `chunks_pkey`, `ix_chunks_document_id`, `ix_chunks_tsv` only; `_vector_list` is a brute-force scan. (If a hnsw/ivfflat index is ever added, it must be built AFTER a re-embed.)
|
||||
- **Scale:** the live KB (and `data/bor_live_kb_replica.dump.sql`) is **1,000 documents / 8,866 chunks**, ~1,000 of them the `is_summary` `position = -1` row. At the benchmarked 93.8 vec/s a full re-embed is minutes — so a quiesced in-place run beats a blue/green column.
|
||||
|
||||
## Design (shared by all tasks)
|
||||
|
||||
**D1 — One nullable `chunks.embedding_model` text column (task 01, migration `0029`).** The value is the `llm_embed_model` string that was in effect when the vector was written. Nullable and **never backfilled by the migration**: for pre-phase rows the true producing model is genuinely unknown, and writing the current setting into them would be a lie that survives the very migration this phase exists to make detectable. `NULL` therefore means *unknown* and is reported as its own bucket. No index: the column is read by `GROUP BY` audits and by the re-embed's `WHERE` over ≤ 10⁴ rows, where a seq scan beats maintaining an index on a low-cardinality column on every chunk write.
|
||||
|
||||
**D2 — One write helper, so a fourth path cannot forget (task 02).** `app/core/embedding_generation.py::stamp_embedding(chunk, vector, model)` sets `embedding` and `embedding_model` together. Every path that writes a vector goes through it: the content chunks (`app/rag/importer.py:1008-1023`), the summary chunk (`:1587-1589`), and the manual-summary edit (`app/api/docs.py:488-491`). A unit pin asserts no `Chunk` vector write in `app/` bypasses it (source-scan style, the house static-pin idiom).
|
||||
|
||||
**D3 — Comparability is MEASURED, not assumed (task 04).** `scripts/reembed_embeddings.py --compare` takes N chunks (`--sample`, default 25, `ORDER BY id` — deterministic, no RNG), embeds their **stored** `content` with the endpoint/env it is given, and reports the cosine distribution (mean / median / p10 / min) against the **stored** vectors. Bands (LOCKED A4): mean ≥ **0.995** → `COMPATIBLE` (same geometry — stamp, don't re-embed); mean < **0.90** → `INCOMPATIBLE` (re-embed); between → `DRIFT` (re-embed + flag). The 0.995 bar exists because pgvector stores `float4` while the app writes Python floats: an unchanged model reproduces a vector to float4 round-trip precision (~0.9999), never to exactly 1.0. This is the tool that answers the owner's actual question instead of guessing, and it doubles as the post-migration smoke test (it must read ≈ 1.0 afterwards).
|
||||
|
||||
**D4 — Three modes, one script (task 04), mirroring `scripts/regen_folder_summaries.py`'s entrypoint idiom** (`get_settings()` + `configure_logging` + `configure_debugging()` + `ensure_admin_configured`, one event loop, one `LLMClient`, `DEBUGPY` honored and never imported by default — AGENTS rule 7):
|
||||
- `--compare` (default when no mode flag): read-only, prints the verdict, exits 0 on COMPATIBLE and non-zero otherwise so the runbook can branch on it. **It never writes.** Because it is its own process it can be run on prod with `BOR_LLM_EMBED_MODEL=<new> uv run python -m scripts.reembed_embeddings --compare` while the app keeps serving the OLD model — the measurement does not touch the running app.
|
||||
- `--stamp-only`: set `embedding_model` to the configured model on every chunk that has a vector, touching no vector. Valid **only** on a COMPATIBLE verdict — the runbook's step for "the endpoint changed name but not geometry".
|
||||
- `--apply`: re-embed in place keyed on `chunk.id`, selecting `WHERE embedding_model IS DISTINCT FROM :configured`, committing per batch, resumable by construction (re-running finishes what is left, so a mid-run outage is never a dead end). Batching is delegated to `LLMClient.embed()` — no second batching implementation. Per-batch progress plus one PLAN §9 summary line: `reembed: model=<stamp> mode=… chunks=… batches=… remaining=… duration_ms=…`.
|
||||
- Both writers **fail loud**: an `EmbeddingError` on a batch logs, stops, exits non-zero and prints the resume command. A chunk is NEVER skipped into `embedding = NULL` — a NULL embedding silently disappears from `_vector_list` (`app/rag/retriever.py:433-455`), which is the exact silent-degradation class this phase kills.
|
||||
- Both writers **never** touch `content`, `position`, `tsv`, `documents.*`, or `sources_meta.version` (the phase-118 convention, `app/rag/importer.py:51-57`: content-unchanged work is not a KB change, so saved chats must not go stale because the vectors were refreshed).
|
||||
|
||||
**D5 — A mismatch is loud in three places (tasks 03, 05, 06), and never silent:**
|
||||
- `app/core/embedding_generation.py::generation_report(session, expected_model)` → a pure, unit-tested counts object: `total`, `with_vector`, `null_vectors`, `by_model` (with a `None`/unknown bucket), `mismatched`, `unknown`, `stale`.
|
||||
- **After every sync** (`app/api/sync.py`, right after the `sync: done` line at `:844`): one INFO line `embeddings model=<configured> chunks=N mismatched=M unknown=K`. A **WARNING** only when `mismatched > 0`; `unknown > 0` alone stays INFO, because every pre-phase KB is 100 % unknown and would scream on every sync forever.
|
||||
- **Admin endpoint** `GET /api/admin/embeddings` (router-wide `Depends(require_admin)`, the `tokens`/`ui_settings` pattern) returning that report as the JSON the card and the runbook both read. Payload is a handful of numbers — it must stay under the phase-167 byte budget and appears in the `grep ' :: api '` audit.
|
||||
- **Settings view card** (phase 168's `#view-settings`, task 06): a read-only "Embedding generation" card above the settings table — `role="status"` when clean, `role="alert"` naming the offending stamp, the chunk counts, and the remediation command (`uv run python -m scripts.reembed_embeddings --apply`). Text states the state; never color alone (B5).
|
||||
|
||||
**D6 — Retrieval stays unfiltered, deliberately (LOCKED A3).** No `WHERE embedding_model = :model` in `_vector_list`: during a backfill it would silently return nothing and deflect every turn (a worse failure than the one being fixed), it costs a predicate on the hot path of every turn, and the migration window is minutes. The column is for DETECTION; the script is for REPAIR. This rejection is recorded in the code docstring so the next reader does not "fix" it.
|
||||
|
||||
**D7 — Threshold re-tuning is operator work, backed by tooling (task 07 docs).** The phase ships the measurement + the runbook; it does NOT change `relevance_threshold` / `lexical_support_floor` / `source_usefulness_floor` defaults, because the right numbers depend on the model the owner actually points at. The runbook makes the order explicit: re-embed FIRST, then re-tune (retuning against a mixed corpus tunes to noise), then verify with `tests/fixtures/retrieval_battery.txt` and the deflection rate. It also names the phase-168 trap: those keys may now be `app_settings` rows, and **a stored override beats `.env`** (`app/core/settings_overlay.py`) — so `select key, value from app_settings where key in ('relevance_threshold','lexical_support_floor','source_usefulness_floor','llm_embed_model');` is step one of any value change.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `168_admin_settings_panel` (**todo — must complete first**, this phase runs after it in the pipeline) — the overlay that makes `app_settings` beat `.env` (D7's trap), the catalog-completeness pin, and the `#view-settings` view + `/api/admin/settings` + `tests/e2e/test_admin_settings_panel.py` that this phase's card and E2E extend.
|
||||
- `97_kb_tree_catalog` + `156_table_ux_polish` (complete) — the admin catalogue/bounded-panel and `.cell-clip` contracts the card must not violate (AGENTS rule 5).
|
||||
- `167_api_payload_diet` (complete) — the 256 KB response budget + `scripts.api_payload_audit` the new endpoint is audited against.
|
||||
- `82_security_headers` + `78_static_background` + `54_asset_cache_bust_revalidation` (complete) — the CSP/`default-src 'self'` and locally-served-assets posture the card's JS lives inside (AGENTS rule 6, the No CDN Rule), and the `?v=` busting the E2E cache contract keeps.
|
||||
- `119_name_signal_read_chips` + `162_search_recall_small_sources` (complete) — own `tests/fixtures/retrieval_battery.txt`, which is this phase's measuring stick; the phase READS it and never edits it (the gated items are the retuning ruler).
|
||||
- `118_summary_seed_context` (complete) — the "content-unchanged work is not a KB change" convention (`app/rag/importer.py:51-57`) that D4 inherits for `sources_meta`.
|
||||
- `122_image_documents` + `137_pdf_image_context` (complete) — image docs store the vision description as `chunks.content`, which is what makes K3 (a re-embed never re-calls vision) true.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_embedding_model_column.md` — `Chunk.embedding_model` + reversible migration `0029_chunks_embedding_model.py` + its up/down test.
|
||||
2. `02_stamp_write_paths.md` — `stamp_embedding()` + the three vector-write call sites routed through it + the no-bypass source pin.
|
||||
3. `03_generation_report_and_logs.md` — `generation_report()` + the post-sync INFO/WARNING lines + the greppable log contract.
|
||||
4. `04_reembed_script.md` — `scripts/reembed_embeddings.py` with `--compare` / `--stamp-only` / `--apply`, the pure cosine helpers, resumability, the §9 summary line, unit + integration coverage.
|
||||
5. `05_admin_embeddings_endpoint.md` — `GET /api/admin/embeddings` (admin-only), the schema, the auth matrix, the byte-budget pin.
|
||||
6. `06_settings_status_card.md` — the read-only "Embedding generation" card in `#view-settings` (`role=status`/`role=alert`, text-first).
|
||||
7. `07_e2e_generation_and_docs.md` — `tests/e2e/test_embedding_generation.py`, the README runbook + AGENTS quick-reference updates, the gates, the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **Unit:** `tests/unit/test_embedding_generation.py` (`stamp_embedding` sets both fields; `cosine` on orthogonal / identical / zero-norm input; `generation_report` buckets — clean, mismatched, NULL-vector rows, unknown stamps, empty table), `tests/unit/test_reembed_embeddings.py` (the `scripts.reembed_embeddings` import idiom of `tests/unit/test_regen_folder_summaries.py:34`: mode parsing, COMPATIBLE/DRIFT/INCOMPATIBLE band edges at exactly 0.995 and 0.90, the deterministic `ORDER BY id` selection, `--apply` failing loud without writing a NULL, `--stamp-only` not touching `embedding`, no `sources_meta` bump), `tests/unit/test_embedding_write_paths.py` (the no-bypass source pin: no `.embedding =` / `embedding=` vector write in `app/` outside `stamp_embedding`), `tests/unit/test_settings_frontend_view.py` extension or a sibling (the card exists, no CDN, DOM built with `createElement`/`textContent`).
|
||||
- **Integration:** `tests/integration/test_migration_0029.py` (up AND down, A13), `tests/integration/test_admin_embeddings_api.py` (auth matrix anonymous / token-user / admin; counts correct against a seeded corpus with mixed stamps and one NULL-embedding chunk; the response body carries no API key), `tests/integration/test_sync_embedding_log.py` (the post-sync INFO line and the WARNING only on `mismatched > 0`), and an importer pin that a synced doc's content chunk AND its `is_summary` chunk both carry the configured stamp.
|
||||
- **Coverage:** **>90%** on `app/` — `uv run pytest --cov=app --cov-report=term-missing`.
|
||||
- **E2E (AGENTS rule 4):** `uv run pytest tests/e2e/test_embedding_generation.py -v --no-cov` green **in isolation**; regression set green in isolation: `test_admin_settings_panel.py`, `test_admin_auth.py`, `test_configurable_brand.py`, `test_global_tuning.py`, `test_llm_retry.py`, `test_sync_button.py`, `test_cache_busting.py`.
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright`.
|
||||
- **Log contract (AGENTS rule 10):** the two new greppable lines — `embeddings model=… chunks=… mismatched=… unknown=…` (post-sync) and `reembed: model=… mode=… chunks=… batches=… remaining=… duration_ms=…` — plus `GET /api/admin/embeddings` present in the byte audit and under the bar (`uv run python -m scripts.api_payload_audit`).
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it will stand after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**.
|
||||
- Task 07 rewrites `README.md`'s "Embedding dimension mismatch" troubleshooting entry (`:1408-1410`) — it currently says the fix is "drop + recreate the chunks table", which is right for a DIM change and dangerously incomplete for a same-dim swap — and adds a "Switching the embedding model" runbook section; `AGENTS.md`'s quick reference gains the compare/apply commands.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `chunks.embedding_model` exists (migration `0029`, reversible), and EVERY vector write in `app/` stamps it — proven by the no-bypass pin and by an importer integration pin covering both a content chunk and the `position = -1` summary chunk.
|
||||
- [ ] `uv run python -m scripts.reembed_embeddings --compare` prints a mean/median/p10/min cosine agreement and a `COMPATIBLE` / `DRIFT` / `INCOMPATIBLE` verdict, writes nothing, and exits non-zero unless COMPATIBLE — proven against a mock endpoint on a seeded corpus (identical vectors → COMPATIBLE; different vectors → INCOMPATIBLE).
|
||||
- [ ] `--apply` re-embeds only the chunks whose stamp differs, commits per batch, is resumable (a second run reports `remaining=0`), never leaves an embedding NULL, and does not bump `sources_meta.version` or touch `documents`/`tsv`.
|
||||
- [ ] A mixed-generation KB is LOUD: `GET /api/admin/embeddings` reports the counts, `#view-settings` shows a `role="alert"` card naming the stamp and the repair command, and the post-sync log WARNs. An all-`unknown` (pre-phase) KB WARNs nothing and shows a `role="status"` card.
|
||||
- [ ] Retrieval is UNCHANGED by the new column — a mixed-stamp corpus still returns its hits (the D6 pin: detection, not filtering).
|
||||
- [ ] `README.md` has the runbook (measure → decide → back up → migrate → re-embed → verify → retune → smoke) including the phase-168 `app_settings`-beats-`.env` warning; `AGENTS.md`'s quick reference lists both script modes.
|
||||
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean, the phase E2E + the regression set green in isolation.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] no behavior change in completed phases (retrieval order, the A8 gate, sync counts, saved-chat staleness all untouched).
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit; the phase dir moves to `.agents/phases/complete/`.
|
||||
|
||||
## Locked decisions (owner-confirmed 2026-10-09 during phase authoring)
|
||||
|
||||
- **A1 — one nullable `chunks.embedding_model` text column**, stamped with the `llm_embed_model` in effect at write time; `NULL` means *unknown* and the migration NEVER backfills a guess.
|
||||
- **A2 — every vector write goes through one helper** (`stamp_embedding`) so no path can desynchronize the pair; a source-scan test enforces it.
|
||||
- **A3 — detection only.** Retrieval never filters on the stamp — a mixed corpus still ranks. (Rejected: a `WHERE embedding_model = …` predicate in `_vector_list`, which silently deflects every turn during a backfill and taxes the hot path; the reason is in the code docstring.)
|
||||
- **A4 — comparability is measured in cosine agreement**, bands `COMPATIBLE ≥ 0.995 > DRIFT ≥ 0.90 > INCOMPATIBLE`, with the float4-round-trip rationale for the 0.995 bar; `--stamp-only` is legal only on COMPATIBLE.
|
||||
- **A5 — per-batch commits + a resumable `WHERE embedding_model IS DISTINCT FROM :configured` selection**, fail-loud on any batch error, never a NULL embedding, and NEVER a `sources_meta.version` bump (a re-embed is not a KB change — phase 118's convention).
|
||||
- **A6 — NO new `Settings` field.** Every knob is a CLI flag, so `tests/unit/test_settings_catalog.py`'s `set(CATALOG) == set(Settings.model_fields)` pin is untouched and the phase-168 panel stays display-only for this feature.
|
||||
- **A7 — one story, one phase, one Playwright file** (AGENTS rule 4): the backend is covered by unit + integration, the visible contract by `tests/e2e/test_embedding_generation.py`.
|
||||
- **A8 — the three A8/A13 tuning defaults are NOT changed by this phase** — they are operator-side, post-re-embed, battery-driven (D7). If the owner's post-migration battery says a default is now wrong, that is a separate, deliberate change.
|
||||
|
||||
## Assumptions (the request left these open; the executor does not guess)
|
||||
|
||||
- **K1:** the stamp is the model NAME. An operator who re-points the same alias server-side (aipi keeps serving `embed` but now runs gemma-2) is caught by `--compare`, not by the column — the column cannot see a name that did not change, and faking it with a vector fingerprint per row costs an endpoint call per write. This boundary is stated in the column docstring and the README runbook.
|
||||
- **K2:** the corpus is ≤ ~10⁴ chunks (live and replica: 8,866), so one quiesced in-process pass is minutes; no parallelism, no job queue, no progress table (PLAN A12: in-process state, no Valkey).
|
||||
- **K3:** every embedded text is already in `chunks.content` — image docs store the VISION DESCRIPTION as their content — so a re-embed never re-calls the vision model and never re-reads a PDF.
|
||||
- **K4:** `--compare`'s sample is `ORDER BY id LIMIT :n` (deterministic across runs; no RNG, no `RANDOM()`), and N defaults to 25 with `--sample` for more.
|
||||
- **K5:** the card is read-only and admin-only (no controls, no "fix it now" button — a re-embed is a maintenance-window operation, not a click).
|
||||
- **K6:** no new fixtures and no dump files are committed (`data/` stays gitignored); tests seed small corpora in the house way.
|
||||
- **K7:** quiescing = "do not run a sync while `--apply` runs" (single app process). The script does not take a lock; the README runbook states the order.
|
||||
- **K8:** `embedding_dim` stays 768 and stays non-overridable (phase 168 A3); a real DIM change remains the drop-and-recreate path in the README — this phase is for the same-dim swap that path does not cover.
|
||||
|
||||
## Commit
|
||||
|
||||
```bash
|
||||
git add app/ alembic/ scripts/ frontend/ tests/ README.md AGENTS.md .agents/phases/ && \
|
||||
git commit --no-gpg-sign \
|
||||
-m "feat(rag): stamp the embedding generation on every chunk and ship the in-place re-embed tooling" \
|
||||
-m "A same-dimension embedding-model swap was invisible: nothing recorded which model made a vector, EmbeddingDimensionError cannot fire at 768 -> 768, and a re-sync re-embeds nothing because the file_hash fast path returns with zero embeds. chunks.embedding_model (migration 0029, nullable, never backfilled) now records it, scripts/reembed_embeddings.py measures comparability in cosine agreement (--compare) and repairs in place (--stamp-only / --apply, per-batch commits, resumable, never a NULL embedding, never a sources_meta bump), and a mixed generation is loud in GET /api/admin/embeddings, the Settings card and the post-sync log. Retrieval is deliberately unfiltered: detection, not repair."
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# Task 01 — `chunks.embedding_model`: the nullable generation stamp (phase 169 D1, LOCKED A1)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Source:** owner request 2026-10-09 — an embedding gemma → embedding gemma 2 upgrade keeps 768 dims, so nothing detects the swap
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`chunks` gains one nullable `embedding_model` text column recording which embedding model produced each vector, with a tested downgrade — the schema fact every later task (stamping, the report, the script, the card) reads.
|
||||
|
||||
## Work
|
||||
|
||||
1. `app/models.py` — in `class Chunk` (L232), directly after `embedding: Mapped[list[float] | None] = mapped_column(Vector(EMBEDDING_DIM))` (L241), add:
|
||||
- `embedding_model: Mapped[str | None] = mapped_column(Text, nullable=True)`
|
||||
- a comment in the file's established voice stating: the value is the `llm_embed_model` in effect when the vector was written; **`NULL` means UNKNOWN (a pre-phase-169 row), never "no model"**; the dimension cannot distinguish generations because `chunks.embedding` is fixed at 768 by A6 (`EMBEDDING_DIM`, L106), so this column is the ONLY thing that distinguishes two geometries of the same width; and a name change is what is recorded — a server-side re-point that KEEPS the name is detected by `scripts/reembed_embeddings.py --compare`, not here (K1).
|
||||
2. New `alembic/versions/0029_chunks_embedding_model.py` — `revision = "0029"`, `down_revision = "0028"` (verify with `uv run alembic heads` first: `0028_app_settings` is head).
|
||||
- `upgrade()`: `op.add_column("chunks", sa.Column("embedding_model", sa.Text(), nullable=True))`
|
||||
- `downgrade()`: `op.drop_column("chunks", "embedding_model")` (A13 — every migration ships a tested downgrade).
|
||||
- NO index, NO `UPDATE`: an index on a low-cardinality column costs every chunk write for an audit `GROUP BY` over ≤ 10⁴ rows (D1), and a backfill would stamp pre-phase rows with a model name nobody verified (A1 — the migration must never guess).
|
||||
- Module docstring: the 2026-10-09 embedding-gemma-2 question, why 768 → 768 is invisible to `EmbeddingDimensionError` (`app/rag/llm.py:314-322`), why a re-sync fixes nothing (the `file_hash` fast path, `app/rag/importer.py:70-84`), and that `NULL` = unknown by design.
|
||||
3. New `tests/integration/test_migration_0029.py` — copy the structure of `tests/integration/test_migration_0028.py` (the up/down harness, the temp-database idiom it uses): upgrade `head` puts `embedding_model` on `chunks` (assert through `sa.inspect(...).get_columns("chunks")`), the column is nullable and text-typed, a chunk row can be inserted with it `NULL` and with a value, and downgrade removes it cleanly with the surviving rows intact.
|
||||
4. `README.md` — in the schema section (`grep -n "chunks" README.md` to locate the table listing), add the new column to the `chunks` description with the `NULL` = unknown note; leave the troubleshooting rewrite for task 07.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: `tests/integration/test_migration_0029.py` (up, down, nullable both ways, rows survive the downgrade). The rest of the model surface is unchanged, so the existing model/migration suites must stay green untouched.
|
||||
- Coverage: **>90%** on this task's new/modified code (the migration module is exercised by the up/down test; `app/models.py` is already covered).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run alembic upgrade head` reports `0029` and `\d chunks` (via `podman exec brain-of-reese_db_1 psql -U reese -d brain_of_reese -c '\d chunks'`) shows `embedding_model | text`.
|
||||
- [ ] `uv run alembic downgrade -1` then `upgrade head` both succeed (A13), pinned by `tests/integration/test_migration_0029.py`.
|
||||
- [ ] No `UPDATE`, no `CREATE INDEX` in the migration; no existing chunk row gains a non-NULL stamp.
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] README.md/AGENTS.md updated for anything user-visible this task changes (the phase's final pass re-checks both project-wide).
|
||||
- [ ] no behavior change in completed work (retrieval, sync counts, the A8 gate untouched).
|
||||
@@ -1,32 +0,0 @@
|
||||
# Task 02 — Stamp every vector write: `stamp_embedding()` + the three call sites (phase 169 D2, LOCKED A2)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Every code path that writes `chunks.embedding` also writes `chunks.embedding_model` from the same settings read, so the pair can never desynchronize — enforced by a source-scan test so a future fourth path fails the build rather than shipping a silent lie.
|
||||
|
||||
## Work
|
||||
|
||||
1. New `app/core/embedding_generation.py` — the phase's home for the generation logic (pure, no HTTP, no `app.db` import at module level; task 03 extends this same module):
|
||||
- `def stamp_embedding(chunk: Chunk, vector: list[float], model: str | None) -> None` — sets `chunk.embedding = list(vector)` and `chunk.embedding_model = model`. Nothing else: no flush, no commit, no validation beyond what SQLAlchemy does (the caller owns the transaction, exactly like the current code).
|
||||
- `def current_embedding_model(settings: Settings | None = None) -> str` — returns `settings.llm_embed_model` (defaults to `get_settings()`), the ONE place the stamp value is derived, so a later move to a fingerprinted stamp changes one function (K1).
|
||||
- Module docstring: why the pair exists (A6 fixes the column at 768 so the dimension cannot distinguish two geometries), the "write the pair together" rule, and the K1 boundary (a name that did not change is invisible here — `scripts/reembed_embeddings.py --compare` is the tool for that).
|
||||
2. `app/rag/importer.py` — the content-chunk write (L1008-1023): inside `for row, vec in zip(doc.chunks, vectors, strict=True):` replace `row.embedding = vec` with `stamp_embedding(row, vec, current_embedding_model(settings))`. Read `settings` the way the surrounding function already does (it takes/uses `settings` — do NOT add a second `get_settings()` call inside the loop).
|
||||
3. `app/rag/importer.py` — the summary-chunk write in `_store_summary` (L1587-1589): replace `chunk.embedding = vector` with the helper. This is the `is_summary` `position = -1` row — the one an operator would otherwise discover months later still carries the old geometry.
|
||||
4. `app/api/docs.py` — the manual-summary edit (L488-491): replace `summary_chunk.embedding = vector` with the helper (the file already constructs `LLMClient()` there; the stamp still comes from `current_embedding_model()`, NOT from the client).
|
||||
5. Greppable invariant comment at each of the three sites (`# phase 169: the vector and its generation are written together (LOCKED A2)`) — short, one line each.
|
||||
6. New `tests/unit/test_embedding_generation.py` — `stamp_embedding` sets BOTH attributes; a `Chunk` built through it carries the model; calling it twice overwrites both; `current_embedding_model()` reads `llm_embed_model` from a passed `Settings` and from `get_settings()`.
|
||||
7. New `tests/unit/test_embedding_write_paths.py` — the no-bypass pin, in the house static-source-scan style (`grep` the `app/` tree from the test, like the existing frontend/CDN pins do): every assignment matching `\.embedding\s*=` or a `Chunk(...)` construction passing `embedding=` inside `app/` lives in `app/core/embedding_generation.py`. Fail with an assertion message naming the offending file:line and the rule ("write vectors through `stamp_embedding` so the generation is stamped with them"). Also pin the reverse: the three call sites still exist (so the pin cannot be satisfied by deleting the calls).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: as above.
|
||||
- Integration (extend the existing importer/sync suites — locate them with `grep -rln "is_summary" tests/integration`): a synced markdown doc's content chunks AND its `is_summary` chunk all carry `embedding_model == "embed"` (the E2E/fixture model name); a doc edited through the summary API (`app/api/docs.py`) re-stamps its summary chunk; a doc RE-imported with changed bytes re-stamps every content chunk it re-embeds. Run the touched suite, then the whole suite.
|
||||
- Coverage: **>90%** on `app/core/embedding_generation.py` and the modified lines.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `grep -rn "\.embedding\s*=" app/ --include=*.py | grep -v __pycache__` shows assignments ONLY inside `app/core/embedding_generation.py`.
|
||||
- [ ] A sync against the mock endpoint leaves `select distinct embedding_model from chunks` = one value, the configured `BOR_LLM_EMBED_MODEL`, across content and summary chunks (integration-pinned).
|
||||
- [ ] `uv run pytest tests/unit/test_embedding_write_paths.py -v` passes, and adding a bypassing `row.embedding = [...]` anywhere in `app/` makes it fail (verified once by hand, then reverted).
|
||||
- [ ] `uv run pytest` green; coverage on the new module >90%; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work — chunk counts, sync summary numbers, `sources_meta` bumps, retrieval order and the A8 gate are all byte-identical to pre-task.
|
||||
-33
@@ -1,33 +0,0 @@
|
||||
# Task 03 — `generation_report()` + the post-sync loud lines (phase 169 D5, LOCKED A3)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
One pure, unit-tested function turns `chunks.embedding_model` into counts-per-generation plus a `stale` verdict, and the sync run reports it on every success — INFO always, WARNING only when chunks actually carry a different generation than the model the app embeds questions with.
|
||||
|
||||
## Work
|
||||
|
||||
1. `app/core/embedding_generation.py` (from task 02) — add:
|
||||
- `@dataclass(frozen=True) class GenerationReport` with `expected_model: str`, `embedding_dim: int`, `total_chunks: int`, `with_vector: int`, `null_vector: int`, `by_model: tuple[ModelCount, ...]`, `mismatched: int`, `unknown: int`, `stale: bool`. `ModelCount` is a small frozen dataclass `(model: str | None, chunks: int, embedded: int)` — `model is None` is the UNKNOWN bucket.
|
||||
- `def generation_report(db: Session, expected_model: str | None = None, embedding_dim: int | None = None) -> GenerationReport` — ONE aggregate query (`select(Chunk.embedding_model, func.count(), func.count(Chunk.embedding)).group_by(Chunk.embedding_model)`), then pure Python bucketing: `mismatched` = chunks whose stamp is non-NULL and `!= expected_model`; `unknown` = chunks whose stamp IS NULL; `stale = mismatched > 0`. NULL-vector rows stay inside their model's `chunks` count and show up in `embedded < chunks` — a chunk that lost its vector is a separate defect and must remain visible, not folded into the generation story.
|
||||
- **D6 in writing:** the docstring states that nothing here filters retrieval (`app/rag/retriever.py:433-455` keeps its unfiltered cosine scan) and why — a `WHERE embedding_model = …` predicate would silently return nothing mid-backfill and deflect every turn, which is worse than the mixed ranking it prevents, and taxes the hot path of every turn for a migration window measured in minutes (LOCKED A3, rejected alternative).
|
||||
2. `app/api/sync.py` — immediately after `logger.info("sync: done detail=%s", _status.detail)` (L844, the SUCCESS terminal only — the cancelled and failed paths log nothing new):
|
||||
- open a short-lived `SessionLocal()` (the file's own convention, L538/L805), call `generation_report(db, settings.llm_embed_model, settings.embedding_dim)`, close it;
|
||||
- always one INFO line, greppable and cron-safe (PLAN §9): `embeddings model=<expected> chunks=<total> mismatched=<n> unknown=<n> null_vector=<n>`;
|
||||
- escalate to **WARNING** only when `mismatched > 0`, appending the repair hint: `— chunks were embedded by another model, run: uv run python -m scripts.reembed_embeddings --apply`. `unknown > 0` alone NEVER warns: a pre-phase-169 KB is 100 % unknown, and a permanent warning is noise that trains the owner to ignore the one signal that matters (D5).
|
||||
- the whole block is wrapped in `except Exception` (broad, with a `# noqa: BLE001` and a comment, matching this file's background-task style): a reporting hiccup logs `embeddings report unavailable: <type>` at WARNING and **never** changes the sync's terminal state, counts, or `_status.detail`.
|
||||
3. New `tests/unit/test_embedding_generation_report.py` — clean corpus (all stamps equal → `stale=False`, `mismatched=0`), a mixed corpus (two named stamps → the counts split, `stale=True`), the UNKNOWN bucket (NULL stamps counted, `unknown=N`, NOT counted as mismatched), NULL-vector rows (`with_vector`/`null_vector` correct, `embedded < chunks`), an empty table (all zeros, `stale=False`), and the `expected_model`/`embedding_dim` defaults resolving from `get_settings()`.
|
||||
4. New `tests/integration/test_sync_embedding_log.py` — seed a tiny corpus, run the sync the way the existing sync integration suites do (`grep -rln "sync: done\|/api/sync" tests/integration` to find the harness and copy it), and assert with `caplog`: the INFO line on success with the right numbers; the WARNING + repair hint appears after `UPDATE chunks SET embedding_model='older-model'` on a subset; NO warning when the only oddity is `embedding_model IS NULL`; no new line on a cancelled run; a broken report (monkeypatch `generation_report` to raise) leaves the sync terminal `done` and its counts untouched.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit + integration as above.
|
||||
- Coverage: **>90%** on `app/core/embedding_generation.py` (every bucket branch) and the new sync block (success / warning / unknown-only / raising paths).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `generation_report()` returns correct counts for the five pinned corpus shapes, and `stale` is `True` ONLY for a non-NULL stamp mismatch.
|
||||
- [ ] A successful sync logs exactly one `embeddings model=… chunks=… mismatched=… unknown=… null_vector=…` line: `grep`-able on stdout in dev (`uv run uvicorn app.main:app --reload` + a sync) and pinned via `caplog`.
|
||||
- [ ] The WARNING fires on a mismatch and does NOT fire on an all-unknown corpus; the repair hint names `scripts.reembed_embeddings --apply`.
|
||||
- [ ] A raising `generation_report` cannot change a sync's terminal state or counts.
|
||||
- [ ] `uv run pytest` green (the existing sync suites untouched), coverage >90% on the new code, ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (the sync's `detail` payload, `sources_meta` bumping and the saved-chat staleness gate are byte-identical).
|
||||
@@ -1,42 +0,0 @@
|
||||
# Task 04 — `scripts/reembed_embeddings.py`: measure comparability, then repair in place (phase 169 D3/D4, LOCKED A4/A5)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
One script answers the owner's real question — *"are the stored vectors still usable against this endpoint?"* — by measuring cosine agreement (`--compare`), and repairs the corpus when they are not (`--apply`), with `--stamp-only` for the "endpoint changed name, not geometry" case. This is the tool a same-dimension embedding-model swap has never had.
|
||||
|
||||
## Work
|
||||
|
||||
1. `app/core/embedding_generation.py` — the pure math + selection, so coverage and unit tests stay clean and the script stays thin glue:
|
||||
- `cosine(a: Sequence[float], b: Sequence[float]) -> float` — pure stdlib (`math.fsum`), `0.0` for a zero-norm operand (never `NaN`/`ZeroDivisionError`), `ValueError` naming both lengths when `len(a) != len(b)` (a length difference is a bug, not a score).
|
||||
- Band constants, documented where they are declared: `COMPATIBLE_MEAN = 0.995` and `INCOMPATIBLE_MEAN = 0.90`, plus `VERDICT_COMPATIBLE / VERDICT_DRIFT / VERDICT_INCOMPATIBLE`. The 0.995 rationale goes in the docstring **verbatim**: pgvector stores `float4` while the app writes Python floats, so an unchanged model reproduces a vector to float4 round-trip precision (~0.9999), never to exactly 1.0 — anything meaningfully below that is a different geometry, and "expect exactly 1.0" would make every healthy corpus read as DRIFT.
|
||||
- `@dataclass(frozen=True) class AgreementStats` — `n`, `mean`, `median`, `p10`, `min`, `worst: tuple[tuple[int, float], ...]` (the ≤ 3 lowest-agreement chunk ids), `verdict: str`; `agreement(samples: Iterable[tuple[int, Sequence[float], Sequence[float]]]) -> AgreementStats` computes it (`(chunk_id, stored, fresh)` triples; empty input → `n=0`, `verdict=VERDICT_INCOMPATIBLE` — "nothing measured" is never "compatible").
|
||||
- Selection helpers (all `ORDER BY Chunk.id`, deterministic, no RNG — K4): `sample_chunks(db, limit)` → `id, content, embedding` rows `WHERE embedding IS NOT NULL`; `pending_chunks(db, model, limit=None)` → `id, content` rows `WHERE embedding IS NULL OR embedding_model IS DISTINCT FROM :model`; `count_pending(db, model) -> int`; `count_missing_vector(db) -> int`.
|
||||
- `stamp_rows(db, model, ids) -> int` — the `--stamp-only` UPDATE, **restricted to rows that actually have a vector** (`embedding IS NOT NULL`) so a vector-less row is never stamped as if it did.
|
||||
2. New `scripts/reembed_embeddings.py` — the entrypoint mirrors `scripts/regen_folder_summaries.py` EXACTLY (the house one-shot idiom): `get_settings()` → `configure_logging(settings.log_level)` → `configure_debugging()` (debugpy only under `DEBUGPY=1`, never imported otherwise — AGENTS rule 7) → `ensure_admin_configured(settings)`, a boot failure prints `reembed_embeddings: boot failed: <e>` to stderr and returns 1; one `LLMClient()`; one `asyncio.run(...)`; a short-lived `SessionLocal()` per unit of work (closed in `finally`); one cron-safe PLAN §9 summary line.
|
||||
- **Modes** (`argparse mutually exclusive group`, default `--compare`): `--compare` | `--stamp-only` | `--apply`; plus `--sample N` (default 25, compare only), `--limit N` (apply/stamp-only: cap the rows this run touches — the debug/smoke knob), `--dry-run` (apply/stamp-only: report `count_pending` and write nothing).
|
||||
- **Exit codes as module constants**, documented in the parser epilog: `EXIT_OK = 0`, `EXIT_FAILED = 1` (boot/endpoint/DB failure), `EXIT_INCOMPATIBLE = 3` (the measurement answered "no") — so the runbook branches on "the answer is no" without confusing it with "the tool broke".
|
||||
- **First log line of every run**: `reembed: starting mode=<mode> model=<llm_embed_model> dim=<embedding_dim> base=<sanitized>` with the base URL through `app.core.errors.sanitize_error` (the phase-84 SEC-13 rule: a `user:pass@` in `BOR_LLM_BASE_URL` must never reach a log). This line is also the operator's proof the run used the env the app will run with — the rule that keeps the stamp honest (K1).
|
||||
- `--compare`: `sample_chunks` → `await llm.embed([content …])` → `agreement(...)` per pair → print a human table (per-chunk id + cosine, and the aggregate), then `reembed: compare model=… n=… mean=… median=… p10=… min=… verdict=…` and, for a non-COMPATIBLE verdict, one more line stating the repair command (`--apply`) and the caveat that `--stamp-only` is only legal on COMPATIBLE. **It writes nothing, ever** — no session writes, no commit (pin it).
|
||||
- `--apply`: loop `pending_chunks(db, model, limit)` in id-ordered pages (default page 200 rows), per page `vectors = await llm.embed([content …])`, `zip(rows, vectors, strict=True)`, `stamp_embedding(row, vec, model)` (task 02's helper — the pair is written together), `session.commit()` per page. Progress INFO per page: `reembed: progress done=%d pending_total=%d`. **Fail loud:** any `EmbeddingError` logs `reembed: FAILED after %d rows — resume with: uv run python -m scripts.reembed_embeddings --apply` and returns `EXIT_FAILED`; no row is ever written with `embedding=None`, and no row is silently skipped (a NULL embedding vanishes from `_vector_list`, `app/rag/retriever.py:433-455` — exactly the silent class this phase kills).
|
||||
- `--stamp-only`: `stamp_rows(...)`, loud INFO line naming the count, and a WARNING that repeats what must be true for this to be correct ("only valid on a COMPATIBLE --compare verdict against the SAME endpoint").
|
||||
- **Never touched:** `content`, `position`, `tsv`, `documents.*`, `sources_meta.version` (the phase-118 convention, `app/rag/importer.py:51-57` — a re-embed is not a KB change, so saved chats must not go stale). No `chunk_document` call, no PDF read, no vision call (K3 — the embedded text already lives in `chunks.content`, image docs included).
|
||||
- Final summary line (all modes): `reembed: model=<stamp> mode=<mode> chunks=<processed|sampled> batches=<llm.embed_batches> remaining=<count_pending after> duration_ms=<…>`. `remaining=0` after a clean `--apply` is the resumability proof; a second run prints `pending=0` and does nothing.
|
||||
- No whitespace special-case for `content`: the chunker never yields empty chunks, and an endpoint rejection on a pathological row fails loud on the batch rule above rather than being quietly skipped.
|
||||
- Module docstring: the embedding-gemma → gemma-2 question this exists for, the three modes, the bands + float4 rationale, the resumability contract, the "run it with the app's own env" rule, and the K1 boundary (a same-name server-side re-point is invisible to the stamp and is exactly what `--compare` catches).
|
||||
3. New `tests/unit/test_reembed_embeddings.py` — the `from scripts import reembed_embeddings as reembed` idiom of `tests/unit/test_regen_folder_summaries.py:34`: mode mutual-exclusion and defaults, `--sample`/`--limit`/`--dry-run` parsing, the exit-code constants, the band edges (mean exactly 0.995 → COMPATIBLE, 0.9949 → not, exactly 0.90 → INCOMPATIBLE boundary handled per the declared rule, empty sample → INCOMPATIBLE), `cosine` (identical → 1.0, orthogonal → 0.0, zero-norm → 0.0 not NaN, length mismatch → `ValueError` naming both lengths), the deterministic sample/pending ordering, and that the base URL in the starting line is sanitized (an `https://user:pass@host` env yields no `user:pass` in the log).
|
||||
4. New `tests/integration/test_reembed_apply.py` — seed a small corpus (a handful of docs + chunks) with `embedding_model='old-model'`; with a stubbed `LLMClient` returning known vectors: `--dry-run` writes nothing (`count_pending` unchanged, no UPDATE issued); `--apply` stamps every row, sets the new vectors, leaves `content`/`position`/`tsv`/`documents` and `sources_meta.version` untouched, and a SECOND `--apply` reports `pending=0` / `remaining=0` (the resumability pin); a partial failure (the stub raises `EmbeddingError` on the second page) leaves page 1 committed, exits `EXIT_FAILED`, logs the resume line, and NO row ends up with `embedding IS NULL`; `--stamp-only` changes only `embedding_model` (vector bytes bit-identical — compare against a snapshot) and skips vector-less rows; `--compare` against a stub that echoes the stored vectors → COMPATIBLE / exit 0, against different vectors → INCOMPATIBLE / exit 3, and the chunks table byte-identical afterwards (the read-only pin).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit + integration as above; the existing `scripts` suites (`tests/unit/test_regen_folder_summaries.py`, `tests/unit/test_llm_probe.py`, `tests/integration/test_import_docs_*.py`) stay green untouched.
|
||||
- Coverage: **>90%** on `scripts/reembed_embeddings.py` and on `app/core/embedding_generation.py` (every mode, every band edge, every failure path).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run python -m scripts.reembed_embeddings --help` shows the three modes, `--sample`, `--limit`, `--dry-run` and the exit-code epilog.
|
||||
- [ ] Against the restored 1,000-doc replica locally (`restore-test-db` skill) the three modes do exactly what the docstring says — `--compare` prints a verdict and leaves the dump's fingerprint byte-identical; `--apply` on a `UPDATE chunks SET embedding_model='old-model'` corpus re-embeds all 8,866 chunks and reports `remaining=0`; a second run is a no-op.
|
||||
- [ ] No code path can write `embedding = NULL`; no path bumps `sources_meta.version`; `--compare` issues no UPDATE (pinned).
|
||||
- [ ] `--stamp-only` on a COMPATIBLE corpus leaves every `embedding` value bit-identical and stamps all vector rows.
|
||||
- [ ] `uv run pytest` green, coverage >90% on the script and module, ruff + pyright clean.
|
||||
- [ ] README.md/AGENTS.md updated for anything user-visible this task changes (the full runbook lands in task 07; the phase's final pass re-checks project-wide).
|
||||
- [ ] no behavior change in completed work (the app, the sync, and retrieval are untouched by this task — only the new script + helpers).
|
||||
-30
@@ -1,30 +0,0 @@
|
||||
# Task 05 — `GET /api/admin/embeddings`: the generation report as an admin endpoint (phase 169 D5)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
A read-only, admin-only endpoint exposing task 03's `GenerationReport` as JSON — the single source the Settings card (task 06) and the operator runbook (task 07 + the `migrate-embed-model` skill) both read to answer "is this KB on one embedding generation?".
|
||||
|
||||
## Work
|
||||
|
||||
1. `app/schemas.py` — two response models beside the other read models (the `HealthResponse` style, no `response_model_exclude` tricks):
|
||||
- `EmbeddingModelCount(BaseModel)`: `model: str | None`, `chunks: int`, `embedded: int` — `model=None` is the UNKNOWN bucket, and `embedded < chunks` names the vector-less rows inside that stamp.
|
||||
- `EmbeddingGeneration(BaseModel)`: `configured_model: str`, `embedding_dim: int`, `total_chunks: int`, `with_vector: int`, `null_vector: int`, `by_model: list[EmbeddingModelCount]`, `mismatched: int`, `unknown: int`, `stale: bool`.
|
||||
- Docstring notes: numbers only, no text from the corpus; and NO tuning values here — `relevance_threshold` / the two floors and their `db`/`env`/`default` source badges are phase 168's `GET /api/admin/settings` job, never duplicated (two sources of truth for a tuning value is how a runbook lies).
|
||||
2. New `app/api/admin_embeddings.py` — `router = APIRouter(prefix="/admin/embeddings", tags=["embeddings"], dependencies=[Depends(require_admin)])` (the `tokens.py`/`ui_settings.py` router-wide pattern: anonymous AND token-user both 403), one route `GET ""` → `EmbeddingGeneration`, `get_db` session, `generation_report(db, settings.llm_embed_model, settings.embedding_dim)` (task 03) mapped to the schema. Module docstring: what "stale" means (a chunk whose stamped model ≠ the model the app embeds questions with, i.e. two geometries on one cosine list), that detection is not filtering (LOCKED A3), and the repair pointer to `scripts.reembed_embeddings`.
|
||||
3. `app/main.py` — register it with the other admin routers (`app.include_router(admin_embeddings_router, prefix="/api")`, next to the `ui_settings`/`tokens` includes at L146-172).
|
||||
4. New `tests/integration/test_admin_embeddings_api.py` — the auth matrix copied from the existing admin-surface suite (`grep -rln "require_admin" tests/integration`): anonymous → 403, token user → 403, admin session → 200; against seeded corpora: clean (one stamp → `stale=false`, `mismatched=0`), mixed (two stamps → counts split, `stale=true`, `mismatched` exact), unknown (NULL stamps counted in `unknown` and NOT in `mismatched`), vector-less rows (`null_vector` and the per-stamp `embedded < chunks`), empty DB (all zeros, `stale=false`); the response body contains no `api_key` / `password` / `secret` substring; the payload is tiny (< 4 KB — the phase-167 byte budget is a 256 KB bar, and a handful of counters must never approach it).
|
||||
5. Byte-audit check (AGENTS rule 10): with the app running, `uv run python -m scripts.api_payload_audit` lists `/api/admin/embeddings` under the bar, and `grep ' :: api '` shows the route's line.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration as above; unit coverage comes free from task 03's report tests (the route is a mapping, so pin it here instead of duplicating the bucket tests).
|
||||
- Coverage: **>90%** on `app/api/admin_embeddings.py` and the new schemas.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/admin/embeddings` returns 200 for the admin and 403 for anonymous and token users (all pinned).
|
||||
- [ ] The JSON matches the four seeded corpus shapes exactly, `stale` is true only for a non-NULL mismatch, and `unknown` never counts toward `mismatched`.
|
||||
- [ ] No secret substring in the body; the payload is a few hundred bytes and appears under the bar in `scripts.api_payload_audit`.
|
||||
- [ ] `uv run pytest` green, coverage >90% on the new route, ruff + pyright clean.
|
||||
- [ ] README.md updated for the new endpoint in its API listing (task 07 finishes the runbook; the final pass re-checks project-wide).
|
||||
- [ ] no behavior change in completed work.
|
||||
@@ -1,34 +0,0 @@
|
||||
# Task 06 — the "Embedding generation" card in the Settings view (phase 169 D5, AGENTS rule 5)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The admin sees, unprompted and in the browser, whether the corpus and the configured embedding model agree — one read-only card at the top of phase 168's `#view-settings` that is quiet-clean on a healthy KB and unmistakably loud when chunks were embedded by another model.
|
||||
|
||||
## Work
|
||||
|
||||
1. `frontend/index.html` — inside `#view-settings` (the phase-168 view), ABOVE the settings `.table-wrap` card, a new card in the file's existing card idiom:
|
||||
- `<section class="card" id="embedding-status" aria-labelledby="embedding-status-title">` with `<h2 id="embedding-status-title">Embedding generation</h2>`, a `<p id="embedding-status-line" role="status" aria-live="polite">` for the state line, and a `<p id="embedding-status-fix" hidden>` wrapping the repair command in a `<code>` element.
|
||||
- No new view, no nav entry, no router change — this is one card inside a route that already exists (`VIEW` / `VIEW_PATH` / `VIEW_MODULES` in `frontend/assets/router.js` are untouched), which is what keeps the phase's blast radius small.
|
||||
2. `frontend/assets/settings.js` (the phase-168 module — extend it, do not fork it) — on view activation, `fetch("/api/admin/embeddings")` once and render through ONE function:
|
||||
- **clean** (`stale=false`, `unknown=0`): the line reads `All N chunks are on "<model>" (dim D).` in `role="status"`; the fix paragraph stays `hidden`.
|
||||
- **unknown only** (`stale=false`, `unknown>0`): `role="status"` and the sentence names it as pre-phase rows, not an error — `N of M chunks predate the stamp (unknown model); K have no vector.` plus the informational `--stamp-only`-or-`--apply` hint in the `<code>` line. This never paints as an error (task 03's INFO-not-WARNING rule, mirrored in the UI).
|
||||
- **stale** (`stale=true`): the SAME line element flips to `role="alert"` and names the offending stamp, the count, the configured model, and the command: `8,612 of 8,866 chunks were embedded by "embed" — the app is set to "embed2". Run: uv run python -m scripts.reembed_embeddings --apply`. Numbers via `toLocaleString()`; the model name via `textContent` (never string-built HTML).
|
||||
- **fetch failure**: `role="alert"`, `Embedding generation status unavailable.` — never a blank card and never a stale previous value left on screen (PLAN §7.4 "never stale").
|
||||
- All DOM built with `createElement` / `textContent` (the house rule — no `innerHTML` with fetched data, no CDN, AGENTS rule 6); a `null`-model bucket label is the literal text `unknown`.
|
||||
3. `frontend/assets/styles.css` — the card needs no new layout primitives: reuse `.card`, and add `overflow-wrap: anywhere;` to the command `<code>` so the repair command wraps instead of clipping (an operator must be able to SELECT and copy it — an ellipsis would break the one action the card offers). Contrast: the stale state uses the existing warning/danger text token on its background (≥ 4.5:1, the theme tokens phase 92/93 established), and the state is carried by the WORDS and the `role`, never by color alone (B5).
|
||||
4. New `tests/unit/test_settings_embedding_card.py` — the static source pins in the house style (read the two files, assert): the section + `aria-labelledby` + `role="status"` + `aria-live="polite"` exist; the fix `<code>` exists and starts `hidden`; `settings.js` references `/api/admin/embeddings`, sets `role` to `alert` on the stale path, uses `textContent` for the model name, contains no `innerHTML` assignment with fetched data, and no `https://` src/href (the No-CDN pin).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the static pins above.
|
||||
- Integration: none needed beyond task 05's endpoint suite (the card is client-side; its behavior is E2E-pinned in task 07). If phase 168's view has an integration render pin, extend it with the card's presence for an admin and its absence for a token user.
|
||||
- Coverage: **>90%** on the changed JS surface as far as the static pins reach (the runtime paths are covered by task 07's E2E).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] An admin opening `/settings.html` sees the card above the settings table, inside the 72rem container, with the correct state for the seeded data (checked in task 07's E2E).
|
||||
- [ ] A stale corpus shows `role="alert"` naming the old stamp, the counts, the configured model, and a fully visible, copyable `scripts.reembed_embeddings --apply` command; a clean corpus shows `role="status"` and no command.
|
||||
- [ ] An all-unknown corpus is a `role="status"` card, never an alert.
|
||||
- [ ] The state is legible with color removed (words + `role` carry it), focus styles are untouched, and no `innerHTML` / CDN appears in the changed JS.
|
||||
- [ ] `uv run pytest` green, ruff + pyright clean, `test_cache_busting.py` and `test_admin_theme_tab.py` still green in isolation (the asset-busting and admin-view contracts survive).
|
||||
- [ ] no behavior change in completed work (the settings table, its Save/Reset behavior, and `/api/admin/settings` are untouched).
|
||||
@@ -1,50 +0,0 @@
|
||||
# Task 07 — E2E for the generation card, the README runbook, the gates, the commit (phase 169, AGENTS rules 4/8/9)
|
||||
|
||||
**Phase:** `169_embedding_generation_migration` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove in a real browser that a mixed embedding generation is visible and that detection does not filter retrieval, write the operator runbook the live upgrade follows, run every gate, and land the phase as one atomic commit.
|
||||
|
||||
## Work
|
||||
|
||||
1. New `tests/e2e/test_embedding_generation.py` — the phase's dedicated Playwright file (AGENTS rule 4), built on the shared fixtures (`from e2e.auth_helpers import login, login_with_token`, the `page` / `app_url` / `db_ready` fixtures, `from app.db import SessionLocal`, and the `TRUNCATE chunks, documents, query_log, kb_overview, git_sources` cleanup `tests/e2e/test_sync_button.py:177` uses — TRUNCATE before AND after). Seed the corpus the way that suite seeds it, so the mock embeddings endpoint (`tests/e2e/mock_llm.py`: deterministic 768-dim bag-of-words) stamps chunks with `BOR_LLM_EMBED_MODEL`. Tests:
|
||||
- **clean corpus** — `login(page, app_url, next="/settings.html")`, open the view, `#embedding-status-line` has `role="status"`, its text contains the configured model name and the chunk count, and `#embedding-status-fix` is hidden.
|
||||
- **mixed corpus is loud** — `UPDATE chunks SET embedding_model = 'older-model'` for a known subset, reload, assert `#embedding-status-line` has `role="alert"` and that its text names `older-model`, the mismatched count, the configured model, and the literal `scripts.reembed_embeddings --apply`; the command element is visible and its full text is present in the DOM (the AGENTS rule 5 clip contract — nothing ellipsised away).
|
||||
- **detection, never filtering (LOCKED A3)** — with the SAME mixed corpus, ask a chat question whose answer is pinned by the seeded corpus and assert the turn still grounds on the same source as it did before the stamps were changed (the retrieval list is unchanged by the column — this is the pin that stops a future task from "fixing" the mixed state by hiding chunks).
|
||||
- **all-unknown is calm** — `UPDATE chunks SET embedding_model = NULL`, assert `role="status"` (NOT `alert`) and that the text says `unknown` rather than naming an error.
|
||||
- **walled surface** — `page`-less API requests (`APIRequestContext`): anonymous and `login_with_token` users get 403 from `GET /api/admin/embeddings`, the admin gets 200 (mirror phase 168's `test_anonymous_and_token_user_are_walled`).
|
||||
- **the script repairs what the card reports** — run the real script in-process against the E2E database (`BOR_LLM_BASE_URL` points at the mock endpoint, so `--apply` is deterministic and free): set all stamps to `older-model`, `reembed.main(["--apply"])` returns 0, the DB reports one stamp again, and reloading `/settings.html` returns to the clean `role="status"` card. (If `LLMClient()` construction needs the env the subprocess-style fixture provides, drive it with the same env the app_server fixture uses — do not add a new env contract.)
|
||||
- Module docstring: the embedding-gemma → gemma-2 story, the four states, and why the A3 pin matters.
|
||||
2. `README.md` — two edits, both substantive:
|
||||
- **New section "Switching the embedding model (same dimension)"** — the operator runbook, in order: (1) baseline the battery and the cosine percentiles; (2) probe the new model (`scripts.llm_probe`, `scripts.test_embed_model`); (3) measure comparability with `uv run python -m scripts.reembed_embeddings --compare` run as its OWN process with `BOR_LLM_EMBED_MODEL=<new>` — which does not disturb the running app — and read the verdict bands (COMPATIBLE ≥ 0.995 / DRIFT / INCOMPATIBLE, with the float4-round-trip reason for the bar); (4) back up (`pg_dump -Fc`, copied off-box, verified); (5) `uv run alembic upgrade head`, then `--stamp-only` on COMPATIBLE or `--apply` otherwise; (6) verify — `--compare` now ≈ 1.0, `GET /api/admin/embeddings` `stale=false`, the Settings card clean, `remaining=0`; (7) ONLY NOW re-tune `BOR_RELEVANCE_THRESHOLD` / `BOR_LEXICAL_SUPPORT_FLOOR` / `BOR_SOURCE_USEFULNESS_FLOOR` against the battery, remembering the two validators cap the floors at the threshold and that **a value stored via the Settings panel (`app_settings`) beats `.env`** (phase 168), so check `select key, value from app_settings where key in ('llm_embed_model','relevance_threshold','lexical_support_floor','source_usefulness_floor');` before and after any change; (8) smoke: the battery's gated items, an out-of-KB question still deflects, and the next N turns' deflection rate in `query_log` looks like the pre-migration one. State the maintenance-window rule (do not run a sync while `--apply` runs) and the K1 boundary (a server-side alias re-point that keeps the model NAME is invisible to the stamp — `--compare` is the detector).
|
||||
- **Rewrite the troubleshooting entry at `:1408-1410`** — "Embedding dimension mismatch" keeps its drop-and-recreate advice (a real DIM change, A6) and gains the pointer that a SAME-dim swap is the invisible one: run `--compare`, then `--apply`; and that the Settings card / `GET /api/admin/embeddings` / the post-sync `embeddings model=… mismatched=…` line are where a mismatch announces itself.
|
||||
3. `AGENTS.md` — the quick-reference block gains the two commands (with a one-line "why" in the surrounding style):
|
||||
```
|
||||
uv run python -m scripts.reembed_embeddings --compare # are the stored vectors still usable? (cosine agreement verdict)
|
||||
uv run python -m scripts.reembed_embeddings --apply # re-embed the chunks whose generation != BOR_LLM_EMBED_MODEL (resumable)
|
||||
```
|
||||
and rule 10's log list gains the two new greppable lines (`embeddings model=… chunks=… mismatched=… unknown=…`, `reembed: model=… mode=… … remaining=… duration_ms=…`).
|
||||
4. Full gates (AGENTS rule 9, all of them, in this order):
|
||||
- `uv run pytest`
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → `app/` coverage **>90%**
|
||||
- `uv run pytest tests/e2e/test_embedding_generation.py -v --no-cov` → green **in isolation**
|
||||
- regression set, each **in isolation**: `tests/e2e/test_admin_settings_panel.py`, `test_admin_auth.py`, `test_configurable_brand.py`, `test_global_tuning.py`, `test_llm_retry.py`, `test_sync_button.py`, `test_cache_busting.py`, `test_admin_theme_tab.py`
|
||||
- `uv run ruff check . && uv run pyright`
|
||||
- `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` on the restored replica (locally, via the `restore-test-db` skill) — the battery's gate items keep passing with the new column present (nothing here should move them; a movement means something in this phase changed ranking, which is a regression)
|
||||
- `uv run python -m scripts.api_payload_audit` (app running) — `/api/admin/embeddings` present and under the bar
|
||||
5. The Documentation & Consistency final pass — read `README.md` and `AGENTS.md` end to end against the post-phase project, fix every inconsistency found **including pre-existing ones this phase did not cause** (e.g. a stale command, a wrong default, a section count), and record what was fixed in the commit body.
|
||||
6. Move `.agents/phases/todo/169_embedding_generation_migration/` → `.agents/phases/complete/` and commit exactly as the overview's **Commit** block specifies (`--no-gpg-sign`, Conventional Commit, plus the `LLM-Generated: true` trailer).
|
||||
|
||||
## Testing & Quality
|
||||
- This task OWNS the phase's whole-suite gate: 100 % passing, `app/` coverage >90%, the phase E2E plus the eight-file regression set green in isolation.
|
||||
- Coverage: **>90%** on `app/` (the phase's new modules included).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `tests/e2e/test_embedding_generation.py` passes in isolation and covers all six behaviors above, including the A3 "the retrieval list is unchanged by the stamp" pin.
|
||||
- [ ] The README runbook exists and is self-sufficient (a competent operator who has never seen this chat can execute a production embedding swap from it alone, including the `app_settings`-beats-`.env` warning and the retune-after-re-embed ordering); the `:1408` troubleshooting entry no longer tells anyone to drop the table for a same-dim swap.
|
||||
- [ ] `AGENTS.md`'s quick reference and rule 10 name both script modes and both log lines.
|
||||
- [ ] Every gate in step 4 is green, with the coverage number recorded in the commit body.
|
||||
- [ ] `README.md` and `AGENTS.md` are accurate and consistent with the whole project (verified in the final pass, fixes listed in the commit body).
|
||||
- [ ] no behavior change in completed phases: the battery's ranks, the A8 gate verdicts, sync counts, `sources_meta` behavior and saved-chat staleness are unchanged.
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit; `.agents/phases/complete/169_embedding_generation_migration/` exists and the todo copy is gone.
|
||||
@@ -0,0 +1,13 @@
|
||||
**Phase 168 — Admin Settings panel — final verification: PASS**
|
||||
|
||||
1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch.
|
||||
2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override).
|
||||
3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests).
|
||||
4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.**
|
||||
5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**.
|
||||
6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326).
|
||||
7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green.
|
||||
8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**.
|
||||
9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol).
|
||||
10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's).
|
||||
11. **Next pending phase:** `170_edit_bubble_sizing`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
.................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 108 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 280 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 86 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 532 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6392 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
Task 01 complete. Report:
|
||||
|
||||
**Implemented**
|
||||
- `app/models.py`: `AppSetting` model appended after `FolderSummary` — `key` (bare field name, VARCHAR(100) PK), `value` (env-native string, TEXT NOT NULL), `updated_at` (TIMESTAMPTZ, server `now()`); module docstring table list updated. `func` was already imported — no new imports.
|
||||
- `alembic/versions/0028_app_settings.py` (revision 0028 ← 0027): `create_table` upgrade, `drop_table` downgrade with the A13 comment (0027-style docstring).
|
||||
- `tests/integration/test_migration_0028.py`: 0017-harness, 6 tests — column/PK contract, 0027 schema survives, value round-trips, duplicate-key `IntegrityError`, NULL-value rejection, A13 downgrade/round-trip.
|
||||
- `.env.example`: top comment block — DB-over-env precedence, panel never writes the file, `BOR_SETTINGS_DB=0` kill switch in container env (not committed here).
|
||||
|
||||
**Gates**
|
||||
- `uv run alembic upgrade head` → 0028 applied; `downgrade 0027` → table gone; re-upgrade clean
|
||||
- `uv run pytest tests/integration/test_migration_0028.py -v` → 6 passed
|
||||
- `uv run pytest --cov=app` (3,986 tests) → exit 0, 100% green; `app/` coverage **99%** (>90%)
|
||||
- `uv run ruff check .` → all passed; `uv run pyright` → 0 errors
|
||||
|
||||
**Docs** — `.env.example` updated (per task); no README/AGENTS.md impact (no API/CLI/user-visible change yet — the API note lands in task 04). No behavior change: nothing reads `app_settings` yet (task 02).
|
||||
|
||||
**Decisions** — migration create date 2026-10-09 (phase date); module-docstring bullet added alongside the class for house consistency.
|
||||
|
||||
**Next pending task** — `02_settings_overlay_resolver.md`
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 59%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
.......................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 280 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 73 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 505 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6107 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
All green. Task 02 complete.
|
||||
|
||||
## Report
|
||||
|
||||
**Implemented (phase 168, task 02):**
|
||||
- `app/core/settings_catalog.py` — `SettingMeta` + complete 75-field `CATALOG` (14 groups in the prescribed display order), `GROUPS`, derived `SECRETS`/`RESTART_REQUIRED`/`NON_OVERRIDABLE`, `env_var()`
|
||||
- `app/core/settings_overlay.py` — `resolve_settings()` (env base + kill switch + DB rows, never calls back into config), `enabled()` per-call `BOR_SETTINGS_DB` read, `load_overrides()` (lazy `NullPool` engine, fail-soft: DEBUG once per process for missing table, WARNING once naming the exception class), `merge()` (per-key skip: orphan/non-overridable/malformed-JSON/validator-drop with cross-field + env-collision fallback, each logged `settings override ignored key=… reason=…`), `register_invalidator`/`run_invalidators` (raising hook logged, never propagated), `reset_state_for_tests()`
|
||||
- `app/config.py` — `get_settings()` → cached lazy delegate to the overlay (`@lru_cache` + `cache_clear()` surface intact), new `invalidate_settings()`
|
||||
- `app/main.py` — post-`ensure_admin_configured` prime: `invalidate_settings()` + `get_settings()` + one boot line (`settings overrides loaded count=N` / `source=env-only disabled`)
|
||||
- Poison guards: `tests/conftest.py` (`setdefault BOR_SETTINGS_DB=0`), `tests/e2e/conftest.py` (hard pin beside `BOR_PDF_PICTORIAL`)
|
||||
- Unit pins: `test_settings_catalog.py` (completeness both directions, closed groups, exact SECRET/NON_OVERRIDABLE/RESTART sets, label/`env_var` pins) + `test_settings_overlay.py` (enabled matrix, `_coerce`, merge happy/skip/invalid/cross-field/env-collision/bool, load failure paths, invalidator registry, recursion-guard + no-`.env`-write source pins)
|
||||
|
||||
**Gates:** `uv run pytest` → 4017 passed (run twice — empty `app_settings` and with garbage rows, both green: poison guard proven) · `--cov=app` → 99% (>90%; both new modules 100%) · `ruff check .` clean · `pyright` 0 errors · live-DB checks: DB row wins on / env wins with `BOR_SETTINGS_DB=0` / `database_url`+`embedding_dim` rows skipped+logged · boot log verified both variants
|
||||
|
||||
**Docs:** README Configuration reference — added the DB-over-env precedence paragraph + `BOR_SETTINGS_DB` switch (`.env.example` block landed in task 01). AGENTS.md: no doc impact (it documents no env vars).
|
||||
|
||||
**Decisions:** standalone `BOR_RELEVANCE_THRESHOLD=0.11` raises the pre-existing cross-field validator (floor 0.35 > 0.11) — verified `Settings()` pre-phase raises identically; with consistent floors it prints 0.11. The validator-blames-env-field case (env floor above a DB threshold) resolves by dropping the colliding override row, not the env value — terminates, rest of the table stands (A5).
|
||||
|
||||
**Next pending task:** `03_live_apply_on_save.md`
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 59%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
......................................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 81 2 98%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 505 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6249 19 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
All gates green. Task complete.
|
||||
|
||||
## Report
|
||||
|
||||
**Implemented (phase 168, task 03 — live apply on Save):**
|
||||
- `app/api/chat.py`: `reset_runtime_caches()` below `get_llm()` — nulls the `_llm` / `_chat_semaphore` settings snapshots (in-flight-turn contract in the docstring; `_chat_active` untouched)
|
||||
- `app/main.py`: `create_app()` registers `register_invalidator(reset_runtime_caches)` right after the task-02 prime line, with a one-time `settings invalidator registered name=chat_runtime_caches` log line (composition root — `app/core/` stays import-clean of `app/api/`)
|
||||
- `tests/unit/test_settings_live_apply.py` (9 tests): null/reset pins, `get_llm()` singleton + rebuild, semaphore follows `BOR_CHAT_MAX_CONCURRENT`, the `create_app` wiring source pin, `app/core`→no-`app/api` grep pin, `invalidate_settings()` hook-run + cache-clear proof, raising-hook survival
|
||||
- `tests/integration/test_admin_settings_live.py` (6 tests, `db`-fixture skip idiom): `relevance_threshold` flip proven through `plan_turn` (HIGH→LOW, no restart), `LLMClient` rebuild (`llm_chat_model` + `llm_timeout` → `AsyncOpenAI` base_url + connect timeout), semaphore cap re-pointing, kill-switch→env-back, `import_extension_set` overlay reach; teardown leaves the table empty (verified: 0 rows)
|
||||
- `tests/unit/test_main.py`: added the switch-ON boot-prime test (closes the 2 previously-uncovered `app/main.py` lines — a task-02 gap)
|
||||
|
||||
**Results:** `uv run pytest` → 4033 passed, 0 failed · `uv run pytest --cov=app --cov-report=term-missing` → 99% TOTAL (>90% gate; `app/main.py` 100%, `settings_overlay.py` 100%) · `uv run ruff check .` → all passed · `uv run pyright` → 0 errors · chat pins (`test_chat_*`, `test_chat_api.py`, `test_chat_concurrency.py`) pass untouched · integration suite green (teardown leaves nothing behind)
|
||||
|
||||
**Docs:** `.env.example` — added the "restart required" sentence to the phase-168 comment block (per task step 5); `README.md` — matching one-sentence supplement to the phase-168 paragraph; no `AGENTS.md` impact (no command/convention change)
|
||||
|
||||
**Decisions:** openai normalizes `base_url` with a trailing slash — the pin compares against `rstrip("/") + "/"`; `AsyncOpenAI._client` is a wrapper (`.timeout` still delegates to httpx, `connect == 7.0` asserted); the integration fixture re-wires the reset hook after `reset_state_for_tests()` since a prior unit suite's reset would have cleared the `app.main`-import registration
|
||||
|
||||
**Next pending task:** `04_admin_settings_api.md`
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+133
@@ -0,0 +1,133 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
........................................................................ [ 99%]
|
||||
. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 280 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 84 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 505 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6255 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
All green. Final report:
|
||||
|
||||
**Implemented (task 04 — `/api/admin/settings`):**
|
||||
- `app/schemas.py`: `AdminSettingItem` / `AdminSettingsTotals` / `AdminSettingsOut` / `AdminSettingIn` / `AdminSettingsCleared` (phase-168 docstring style)
|
||||
- `app/api/admin_settings.py`: router-wide `require_admin` (anon + token user 403); GET catalogue in `GROUPS` order with `db`/`env`/`default` badges + `overlay_enabled`; PUT with 404/422 guards (A3 keys, blank secret) + full-candidate validation through the overlay's `_coerce` + real `Settings` validators (422 names the field, nothing written); `pg_insert` upsert; `DELETE /{key}` idempotent reset; `DELETE ""` clear-all; value-free log lines; `invalidate_settings()` on every write
|
||||
- `app/main.py`: router included after `ui_settings_router` with the house comment
|
||||
- Tests: `tests/integration/test_admin_settings_api.py` (11 tests: auth matrix, shape/order, write→live via `get_settings()` + `/api/config`, 404/422/invalid-JSON/cross-field, masking (raw-text assert), reset/clear-all, kill switch, <64 KB budget, caplog log pin) + `tests/unit/test_admin_settings_api.py` (8 tests: `_display`/`_field_type`, masked/default/db branches, `.env`-never-written + no-shell static pins)
|
||||
|
||||
**Results (all 100% green):**
|
||||
- `uv run pytest --cov=app` → 4052 passed, app/ coverage 99% (>90% gate); `app/api/admin_settings.py` 100%
|
||||
- `uv run pytest tests/integration/test_admin_settings_api.py -v` → 11 passed; integration suite run 3× incl. once with a deliberate poison `app_settings` row — teardown left 0 rows
|
||||
- `uv run ruff check .` clean; `uv run pyright` clean (5 initial errors fixed: Literal annotation, `RETURNING` instead of `rowcount`)
|
||||
|
||||
**Docs:** README Configuration reference — made the precedence explicit (`app_settings` row → `BOR_*` env → code default) + documented the `/api/admin/settings` routes & write-only secrets; `.env.example` already complete from tasks 01/03; no AGENTS.md impact (it documents operating rules, not endpoints)
|
||||
|
||||
**Decisions:** `source="db"` requires the overlay switch ON (kill switch off → badge falls back to env/default, `overlay_enabled=false` banner); unknown DELETE key → 404 like PUT (idempotency is row-absent only); `totals.overridden` counts in-effect rows
|
||||
|
||||
**Next pending task:** `.agents/phases/todo/168_admin_settings_panel/05_settings_view_ui.md`
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
.................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 108 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 280 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 86 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 532 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6392 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
All green. Final report:
|
||||
|
||||
**Phase 168 task 05 (Settings view) — complete** (work was largely present in the working tree from a prior partial run; I verified every item, fixed two defects, and re-verified end-to-end)
|
||||
|
||||
- `#view-settings` in `index.html` (72rem container, gate, kill-switch banner, status/alert regions, search, `.table-wrap` catalogue, Save/Clear-all/dirty-hint action row) + `#nav-settings` (ship-hidden) on all 4 Theme-bearing pages
|
||||
- `router.js` (VIEW/VIEW_PATH/VIEW_MODULES + title/description), `main.py` shell route, `caching.HTML_PAGES`, `header.js` admin-keyed reveal
|
||||
- `settings.js` (gate-first, createElement/textContent only, dirty Map, K4 stop-at-first-failure, two-step confirm, client-side filter) + `styles.css` phase-168 block (no new `:root` tokens) + `test_settings_frontend_view.py` (13 pins)
|
||||
- **Fixes:** ruff W291 trailing-whitespace and pyright `None.group()` access in `test_settings_frontend_view.py`
|
||||
- Verified live: `/settings.html` 200 + `no-cache` + `?v=` rewrite; anon API 403; admin GET → 75 items, 4 masked secrets, zero plaintext; PUT `input_placeholder` → `GET /api/config` shows the override **without restart**; value-free `settings action=…` log line; DELETE → env/default wins
|
||||
- `uv run pytest --cov=app`: **4065 passed, 99% coverage** (>90%); `uv run ruff check .`: clean; `uv run pyright`: 0 errors
|
||||
- Docs: README updated (Settings page section + DB-over-env env note); AGENTS.md — no doc impact (no page inventory; rules are generic)
|
||||
- No live-infra changes (dev-container DB only)
|
||||
|
||||
**Next pending task:** `168_admin_settings_panel/06_e2e_settings_panel.md` (the phase's Playwright suite + regression set + atomic commit)
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 108 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 280 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 86 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 532 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6392 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
All gates green and the working tree is exactly my three changes (plus prior-task artifacts the harness owns). Final report:
|
||||
|
||||
**Phase 168 · Task 06 — E2E for the Admin Settings panel: COMPLETE**
|
||||
|
||||
1. **New suite** `tests/e2e/test_admin_settings_panel.py` — **12/12 pass in isolation** (`uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov`, Postgres up).
|
||||
2. **Contract proven in a real browser**: admin saves a `.env`-only setting (75 `BOR_*` fields, `app_settings`) → **the same running app serves the new value on the next real chat turn — no restart, no `.env` write** (`.env` mtime guarded); the gate (anon 403 + no data leak, token users never see the panel), 422 refusal with the alert, the K4 failed-batch error fix (a real `settings.js` defect — the failed save's re-`GET` was clearing its own error — now fixed: re-GET first, *then* the error), Reset, Clear All, the kill-switch banner, secret rows masked, source badges, and a full keyboard walk (skip-link → … → first row control).
|
||||
3. **Live-bar proof** uses the task's sanctioned proxy question (cosine 0.25, zero FTS hits) with bars derived from the measured `top_score`; the server's per-turn log line (`threshold=0.20 deflected=False` / `threshold=0.27 deflected=True`) corroborates each flip.
|
||||
4. **Regression (each in isolation)**: `test_admin_theme_tab` 5 ✓ · `test_api_tokens` 9 ✓ · `test_configurable_brand` 5 ✓ · `test_global_tuning` 6 ✓ · `test_admin_auth` 6 ✓ · `test_cache_busting` 4 ✓ · `test_wide_desktop_column` 3 ✓.
|
||||
5. **Full suite**: 4065 passed · **coverage 99%** (>90% gate) · ruff clean · pyright 0 errors.
|
||||
6. **Byte audit** (`scripts/api_payload_audit.py`, now includes the catalogue): `/api/admin/settings` = 23.6 KB identity / 3.3 KB gzip — **ok** (256 KB budget); ` :: api ` log line observed live for the PUT/DELETE routes; the write-side log line carries key + status only (A4 — never the value).
|
||||
7. **DB clean**: `app_settings` 0 rows, `ui_settings` 0, no `e2e-*` tokens (module autouse fixture TRUNCATEs before + after every test).
|
||||
8. Working tree only — no git operations; README/AGENTS already document the panel + audit (no changes needed).
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 8%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 108 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 280 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 234 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 285 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 86 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 841 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 156 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 532 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6392 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -2,6 +2,22 @@
|
||||
# Copy to `.env` and adjust: cp .env.example .env
|
||||
# (`.env` is gitignored; never commit secrets.)
|
||||
|
||||
# --- DB setting overrides (admin Settings panel, phase 168) ---
|
||||
# Precedence: the app reads every BOR_* from the environment / this
|
||||
# file first; an override the admin saves in the Settings panel
|
||||
# (stored in Postgres, `app_settings`) WINS over that value for the
|
||||
# key — with no restart. The panel's per-key Reset (or "Clear all
|
||||
# overrides") deletes the DB row and this value wins again. The
|
||||
# panel NEVER writes this file (LOCKED A1).
|
||||
# Kill switch: BOR_SETTINGS_DB=0 disables ALL DB overrides (the app
|
||||
# reads exactly this file / the environment). Set it in the CONTAINER
|
||||
# ENV, not here — a 0 committed in this file is a silent "my change
|
||||
# did nothing" that the panel cannot see or fix.
|
||||
# "No restart" is not universal: the values the panel marks "restart
|
||||
# required" (DB pool sizes, session cookie wiring, log level,
|
||||
# compression, the static dir) apply on the next
|
||||
# `podman compose restart app`; everything else applies immediately.
|
||||
|
||||
# --- App ---
|
||||
BOR_ENVIRONMENT=development
|
||||
# BOR_APP_NAME=Brain of Reese # display name on all pages — titles, header brand, status labels, aria text (phase 39)
|
||||
|
||||
@@ -67,6 +67,7 @@ client-side router. Direct URLs deep-link to the matching view.
|
||||
| **Saved chats** | `/history.html` | Admin only |
|
||||
| **Access tokens** | `/tokens.html` | Admin only |
|
||||
| **Theme** | `/theme.html` | Admin only |
|
||||
| **Settings** | `/settings.html` | Admin only |
|
||||
| **Document viewer** | `/document.html?source=X&path=Y` | Token users & admins |
|
||||
| **Login** | `/login.html` | Everyone |
|
||||
| **Shared chat** | `/shared/<token>` | Everyone (anonymous) |
|
||||
@@ -207,6 +208,22 @@ chat. Admin-only view.
|
||||
Generate tokens and hand them out — a token opens chat, the answers, and
|
||||
the documents they cite. Shared chats stay open to everyone.
|
||||
|
||||
### Settings (`/settings.html`)
|
||||
|
||||
Edit any setting that today lives in `.env` — every `BOR_*` setting, grouped
|
||||
and type-correct — from the browser. A value you **Save** is stored in the
|
||||
database and **wins over the environment variable from that instant on, with
|
||||
no restart** (boot-wired settings — the pool, the session middleware, the log
|
||||
level — carry a `restart required` badge and apply on the next start); **Reset**
|
||||
hands control back to the environment, and **Clear all overrides** empties the
|
||||
override table. `.env` is never written: the environment stays the base, the
|
||||
database row wins. Secrets are write-only (you set a value, never read one
|
||||
back); a value that fails validation is rejected before anything is stored.
|
||||
Admin-only — anonymous visitors see a sign-in gate.
|
||||
|
||||
The overlay can be switched off for the whole process with
|
||||
`BOR_SETTINGS_DB=0` (env-only; the panel stays visible and says so).
|
||||
|
||||
### Shared chat (`/shared/<token>`)
|
||||
|
||||
A read-only snapshot of any conversation, shareable via a link. Open to
|
||||
@@ -1075,6 +1092,29 @@ Everything is an environment variable (the `BOR_` prefix, plus
|
||||
[`.env.example`](.env.example) is documented below** — copy that file
|
||||
to `.env` and adjust.
|
||||
|
||||
**DB-over-env overrides (phase 168):** the precedence for every setting
|
||||
is **`app_settings` row → `BOR_*` environment → code default** — an
|
||||
override the admin saves in the Settings panel is stored in Postgres
|
||||
(`app_settings`, one row per key) and **wins over the env var for
|
||||
that key — with no restart**. The panel never writes `.env` (the env
|
||||
value stays the base; per-key Reset deletes the row and the env value
|
||||
wins again). `BOR_SETTINGS_DB=0` (set in the **container env**, not
|
||||
`.env` — a committed `0` is a silent "my change did nothing")
|
||||
disables all DB overrides — the pre-phase behavior. Two keys are NOT
|
||||
overridable: `BOR_DATABASE_URL` (it builds the very engine that would
|
||||
read the override) and `BOR_EMBEDDING_DIM` (fixed at `chunks` table
|
||||
creation). A stored value that fails the setting's validators is
|
||||
skipped at read with a logged reason — a DB row never bricks boot.
|
||||
"No restart" is not universal: the settings the panel marks
|
||||
**restart required** (DB pool sizes, session cookie wiring, log
|
||||
level, compression, the static dir) apply on the next
|
||||
`podman compose restart app`; everything else applies immediately.
|
||||
Admins read/write the overrides through the admin-only
|
||||
`/api/admin/settings` routes (GET the catalogue, PUT `/{key}` set one
|
||||
value, DELETE `/{key}` reset one, DELETE all clear everything);
|
||||
secrets are write-only there (the response reports `has_value`,
|
||||
never the plaintext).
|
||||
|
||||
### App & branding
|
||||
|
||||
| Env | Default | Meaning |
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""app_settings: admin DB-side overrides for any BOR_* setting (phase 168, task 01)
|
||||
|
||||
Revision ID: 0028
|
||||
Revises: 0027
|
||||
Create Date: 2026-10-09
|
||||
|
||||
``TODO.md`` L3: "I want an admin panel that can edit anything in .env
|
||||
and save it to the database. The database value should override the
|
||||
env var. The change should take effect on clicking 'save' on that
|
||||
page." This revision carries the storage half of that (phase 168 D1)
|
||||
— the resolver (D2) and the live-apply hook (D3) are task 02/03.
|
||||
|
||||
* ``app_settings`` — one row per admin setting override, the whole
|
||||
table a narrow key/value pair:
|
||||
|
||||
* ``key`` — VARCHAR(100) NOT NULL, PRIMARY KEY. The BARE
|
||||
``Settings`` field name (``relevance_threshold``) — never the
|
||||
``BOR_…`` form: the prefix is a presentation concern
|
||||
(``"BOR_" + key.upper()``), so a stored prefix could double up.
|
||||
The PK is the uniqueness guard: two rows for one setting would
|
||||
make "which one wins" undefined, and Postgres enforces it for
|
||||
free (the ``ui_settings`` single-row shape is not an option here
|
||||
— there are 75 settings, and a wide row would need per-field DDL
|
||||
for every new one, defeating "edit ANYTHING in .env").
|
||||
* ``value`` — TEXT NOT NULL. The value in the SAME textual form
|
||||
the env var would carry (``"0.55"``, ``"true"``, ``"md,txt"``,
|
||||
a JSON array for the one ``list[str]`` field). Storing the env-
|
||||
native string — not a typed column — keeps coercion and
|
||||
validation in exactly one place (the pydantic validators the
|
||||
overlay runs the full candidate set through, phase 168 D1/A5),
|
||||
and makes "anything in .env" work for all 75 settings with zero
|
||||
per-field DDL. NOT NULL on purpose: "clear the override" is a
|
||||
DELETE of the row (the panel's per-row Reset / Clear all), never
|
||||
an empty value, so the resolver never has to distinguish
|
||||
"no override" from "override to empty".
|
||||
* ``updated_at`` — TIMESTAMPTZ NOT NULL, ``now()`` server default
|
||||
(the ``folder_summaries`` / ``kb_overview.updated_at`` house
|
||||
style): when the admin last saved the override.
|
||||
|
||||
No index beyond the PK: the whole table is read by primary key order
|
||||
in ONE ``SELECT`` per settings resolution (the ``ui_settings``
|
||||
"display fact, never a query surface" note, 0027).
|
||||
|
||||
One additive, fully reversible migration (A13).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "0028"
|
||||
down_revision = "0027"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# No index beyond the PK (a display fact, never a query surface —
|
||||
# the whole table is one SELECT per resolution) and no server
|
||||
# default on ``value`` (NOT NULL — the reset is a DELETE).
|
||||
op.create_table(
|
||||
"app_settings",
|
||||
sa.Column("key", sa.String(length=100), primary_key=True),
|
||||
sa.Column("value", sa.Text(), nullable=False),
|
||||
sa.Column(
|
||||
"updated_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.func.now(),
|
||||
nullable=False,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# The table is the only 0028 artefact — dropping it leaves 0027's
|
||||
# schema byte-identical (A13, fully reversible). No data is lost
|
||||
# by the drop in any meaningful sense: the values are re-enterable
|
||||
# from the panel (they are an override of the env, not a source of
|
||||
# truth — the env file stays the base, LOCKED A1).
|
||||
op.drop_table("app_settings")
|
||||
@@ -0,0 +1,347 @@
|
||||
"""The admin Settings panel's read/write surface (phase 168, task 04).
|
||||
|
||||
``TODO.md`` L3: "…can edit anything in .env and save it to the
|
||||
database. The database value should override the env var. The change
|
||||
should take effect on clicking 'save'."
|
||||
|
||||
The whole router sits behind :func:`app.core.auth.require_admin`
|
||||
(router-wide ``dependencies`` — the :mod:`app.api.ui_settings` pattern):
|
||||
anonymous callers AND token users get 403 on every route (only the
|
||||
admin retunes the deployment).
|
||||
|
||||
Routes (under ``/api`` via the ``main`` registration):
|
||||
|
||||
* ``GET /api/admin/settings`` — the whole catalogue: one item per
|
||||
``Settings`` field, in ``GROUPS`` order (the stable order the E2E
|
||||
pins against), each with its effective value in env-native string
|
||||
form, a ``db``/``env``/``default`` source badge, and the three
|
||||
capability flags. Reads only; creates nothing.
|
||||
* ``PUT /api/admin/settings/{key}`` — set one value (the body's string
|
||||
is stored as-is in ``app_settings``). Fail loud BEFORE the write
|
||||
(LOCKED A5): the FULL candidate set (every overridable stored row +
|
||||
this key/value, layered over the env base) is run through the
|
||||
real ``Settings`` validators — a bad shape OR a cross-field rule
|
||||
(e.g. ``lexical_support_floor`` above a DB-overridden
|
||||
``relevance_threshold``) is a 422 that names the field and stores
|
||||
NOTHING. Unknown key → 404; a non-overridable key (LOCKED A3) → 422
|
||||
naming the env var; a blank value for a secret → 422 (an empty API
|
||||
key / session secret is a self-lockout; ``DELETE`` is the explicit
|
||||
clear). A successful write commits, calls ``invalidate_settings()``
|
||||
(task 03 makes it live — no restart) and returns the refreshed item.
|
||||
* ``DELETE /api/admin/settings/{key}`` — reset to env: deletes the row
|
||||
(idempotent — deleting an absent key is a success, not a 404),
|
||||
invalidates, returns the item with ``source="env"``/``"default"``.
|
||||
* ``DELETE /api/admin/settings`` — "Clear all overrides": deletes every
|
||||
row, invalidates, ``{"cleared": rowcount}``.
|
||||
|
||||
Three invariants (the unit tests pin them):
|
||||
|
||||
1. **Validate-before-store** — the write path never persists a value
|
||||
the ``Settings`` validators reject (A5);
|
||||
2. **Masked secrets** (LOCKED A4) — a ``secret`` field reports
|
||||
``masked``/``has_value`` only, never the plaintext, in no response
|
||||
body; every write logs one line carrying the key and the action,
|
||||
NEVER the value (AGENTS rule 10);
|
||||
3. **Never write ``.env``** (LOCKED A1) — this module contains no file
|
||||
writes, never shells out and never ``eval``s a value; the env file
|
||||
stays the base, the DB row wins, the reset deletes the row.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import Any, Literal, get_origin
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import delete, func, text
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings, get_settings, invalidate_settings
|
||||
from app.core.auth import require_admin
|
||||
from app.core.settings_catalog import CATALOG, GROUPS, env_var
|
||||
from app.core.settings_overlay import LIST_FIELDS, _coerce, enabled
|
||||
from app.db import get_db
|
||||
from app.models import AppSetting
|
||||
from app.schemas import (
|
||||
AdminSettingIn,
|
||||
AdminSettingItem,
|
||||
AdminSettingsCleared,
|
||||
AdminSettingsOut,
|
||||
AdminSettingsTotals,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("app.api.admin_settings")
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/admin/settings",
|
||||
tags=["admin-settings"],
|
||||
dependencies=[Depends(require_admin)], # phase 168: the settings panel is admin-only
|
||||
)
|
||||
|
||||
|
||||
def _field_type(key: str) -> Literal["int", "float", "bool", "str", "json"]:
|
||||
"""The panel control kind for one field, from its annotation
|
||||
(``bool`` before ``int`` — ``isinstance(True, int)`` is ``True``;
|
||||
the one ``list[str]`` field is ``json``, K2)."""
|
||||
annotation = Settings.model_fields[key].annotation
|
||||
if get_origin(annotation) in (list, tuple):
|
||||
return "json"
|
||||
if annotation is bool:
|
||||
return "bool"
|
||||
if annotation is int:
|
||||
return "int"
|
||||
if annotation is float:
|
||||
return "float"
|
||||
return "str"
|
||||
|
||||
|
||||
def _display(field_name: str, value: Any) -> str:
|
||||
"""Stringify one effective value in the env-native string form
|
||||
(D1: stored strings, one coercion place — the router's display is
|
||||
the mirror of ``settings_overlay._coerce``): bool →
|
||||
``"true"``/``"false"``, the one ``list[str]`` field → a JSON array
|
||||
(what the env spelling would carry, K2), everything else
|
||||
``str()``."""
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
if field_name in LIST_FIELDS:
|
||||
return json.dumps(value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def _rows(db: Session) -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings`` — the whole table in one
|
||||
round trip (the ``ui_settings`` "display fact, never a query
|
||||
surface" note)."""
|
||||
return {key: value for key, value in db.execute(text("SELECT key, value FROM app_settings"))}
|
||||
|
||||
|
||||
def _item(
|
||||
key: str,
|
||||
rows: dict[str, str],
|
||||
effective: Settings,
|
||||
) -> AdminSettingItem:
|
||||
"""One catalogue entry for one field (the stable builder for all
|
||||
three read paths — GET, the PUT/DELETE returns, and the error
|
||||
paths' neighbours)."""
|
||||
meta = CATALOG[key]
|
||||
value = getattr(effective, key)
|
||||
#: Only an OVERRIDABLE row counts as an override: a hand-inserted
|
||||
#: ``database_url`` / ``embedding_dim`` row is never in effect
|
||||
#: (the overlay drops it, LOCKED A3) — it is not an override.
|
||||
has_override = meta.overridable and key in rows
|
||||
#: ``source="db"`` means "in effect": a row the overlay actually
|
||||
#: applies. With the kill switch off, no row is in effect, so the
|
||||
#: badge falls back to env/default (the panel's ``overlay_enabled``
|
||||
#: banner explains the rest — D6).
|
||||
in_effect = has_override and enabled()
|
||||
if in_effect:
|
||||
source: str = "db"
|
||||
elif os.environ.get(env_var(key)) is not None:
|
||||
source = "env"
|
||||
else:
|
||||
source = "default"
|
||||
if meta.secret:
|
||||
# LOCKED A4 — write-only: the plaintext appears in NO response;
|
||||
# the read side reports has_value (is a non-empty value in
|
||||
# effect at all) and nothing else about it.
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=True,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=None,
|
||||
masked=True,
|
||||
has_value=bool(value),
|
||||
has_override=has_override,
|
||||
db_value=None,
|
||||
default="",
|
||||
)
|
||||
return AdminSettingItem(
|
||||
key=key,
|
||||
env_var=env_var(key),
|
||||
group=meta.group,
|
||||
label=meta.label,
|
||||
type=_field_type(key),
|
||||
secret=False,
|
||||
restart_required=meta.restart_required,
|
||||
overridable=meta.overridable,
|
||||
source=source,
|
||||
value=_display(key, value),
|
||||
masked=False,
|
||||
has_value=bool(_display(key, value)),
|
||||
has_override=has_override,
|
||||
db_value=rows.get(key) if has_override else None,
|
||||
default=_display(key, Settings.model_fields[key].default),
|
||||
)
|
||||
|
||||
|
||||
def _validate_candidate(candidate: dict[str, str]) -> None:
|
||||
"""Validate the FULL candidate set before anything is stored
|
||||
(LOCKED A5, D5): every overridable stored row + the new
|
||||
key/value, layered over the env base, through the SAME coercion
|
||||
(``settings_overlay._coerce`` — one place for the string →
|
||||
field-value decision) and the REAL ``Settings`` validators. A bad
|
||||
shape OR a cross-field rule is a 422 that names the field — and
|
||||
nothing is written (the caller writes only on return)."""
|
||||
kwargs: dict[str, Any] = {}
|
||||
for key, raw in candidate.items():
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {key}: {exc}"
|
||||
) from exc
|
||||
try:
|
||||
Settings(**kwargs)
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
field = str(err["loc"][0])
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"invalid value for {field}: {err['msg']}"
|
||||
) from exc
|
||||
|
||||
|
||||
@router.get("", response_model=AdminSettingsOut)
|
||||
def get_catalogue(
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingsOut:
|
||||
"""The whole catalogue (75 items, ``GROUPS`` order — the stable
|
||||
order the E2E pins against).
|
||||
|
||||
Reads the table once, resolves the effective ``Settings`` through
|
||||
``get_settings()`` (the overlay — DB-over-env, task 02), and
|
||||
emits one item per field. GETs log nothing here: the phase-167
|
||||
``ResponseBudgetMiddleware`` already writes the
|
||||
``api method=… bytes=…`` line, and the whole catalogue must stay
|
||||
under ``BOR_LOG_LARGE_RESPONSE_BYTES``' INFO bar (75 items ≈ 20 KB
|
||||
— verified in task 06 with the audit script).
|
||||
"""
|
||||
rows = _rows(db)
|
||||
effective = get_settings()
|
||||
items = [
|
||||
_item(key, rows, effective)
|
||||
for group in GROUPS
|
||||
for key, meta in CATALOG.items()
|
||||
if meta.group == group
|
||||
]
|
||||
return AdminSettingsOut(
|
||||
items=items,
|
||||
totals=AdminSettingsTotals(
|
||||
count=len(items),
|
||||
overridden=sum(1 for item in items if item.source == "db"),
|
||||
),
|
||||
overlay_enabled=enabled(),
|
||||
)
|
||||
|
||||
|
||||
@router.put("/{key}", response_model=AdminSettingItem)
|
||||
def set_value(
|
||||
key: str,
|
||||
payload: AdminSettingIn,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Set one value (store it in ``app_settings`` — env-native string
|
||||
form, D1), make it live (``invalidate_settings()`` — task 03) and
|
||||
return the refreshed item.
|
||||
|
||||
The write path (LOCKED A5, A3, A4): unknown key → 404; a
|
||||
non-overridable key → 422 naming the env var (A3: the engine is
|
||||
built from ``database_url`` — an override is a chicken-and-egg
|
||||
brick; ``chunks.embedding`` is fixed at ``embedding_dim`` by A6,
|
||||
so a dim override would corrupt retrieval); a blank value for a
|
||||
secret → 422 (a self-lockout; ``DELETE`` is the explicit clear);
|
||||
then the FULL candidate set is validated before anything is
|
||||
stored (a 422 names the field and writes nothing).
|
||||
"""
|
||||
meta = CATALOG.get(key)
|
||||
if meta is None:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
if not meta.overridable:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=f"{env_var(key)} is not settable from the database",
|
||||
)
|
||||
if meta.secret and not payload.value.strip():
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"{env_var(key)} must not be blank — use Reset to fall "
|
||||
"back to the environment"
|
||||
),
|
||||
)
|
||||
# The candidate set: the overridable stored rows + this key/value
|
||||
# (the same drop rules ``settings_overlay.merge`` applies on the
|
||||
# read side — an orphan / non-overridable row can never affect
|
||||
# validation or the effective set).
|
||||
rows = _rows(db)
|
||||
candidate = {
|
||||
k: v for k, v in rows.items() if k in CATALOG and CATALOG[k].overridable
|
||||
}
|
||||
candidate[key] = payload.value
|
||||
_validate_candidate(candidate)
|
||||
# The write: upsert on the PK (INSERT … ON CONFLICT (key) DO
|
||||
# UPDATE — the row's updated_at moves with the save).
|
||||
stmt = pg_insert(AppSetting).values(key=key, value=payload.value)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["key"],
|
||||
set_={"value": stmt.excluded.value, "updated_at": func.now()},
|
||||
)
|
||||
db.execute(stmt)
|
||||
db.commit()
|
||||
# One line per write, never the value (AGENTS rule 10, A4).
|
||||
logger.info("settings action=set key=%s status=200", key)
|
||||
# Make it live (task 03): clear the cache + run the runtime reset
|
||||
# hooks, then report the refreshed item from the fresh settings.
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("/{key}", response_model=AdminSettingItem)
|
||||
def reset_value(
|
||||
key: str,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> AdminSettingItem:
|
||||
"""Reset one key to env (LOCKED A1: the reset is a row DELETE —
|
||||
never an empty value, so the resolver never distinguishes "no
|
||||
override" from "override to empty").
|
||||
|
||||
Idempotent: deleting a key with NO ROW is a success, not a 404
|
||||
(the panel's per-row Reset is safe to double-click — the env
|
||||
value is already in effect; an unknown *setting* key is a 404,
|
||||
like the PUT — a typo in the key is a client bug, and the item
|
||||
shape needs the catalog metadata, so "nothing to reset" can only
|
||||
mean "the row is already gone"). A successful delete logs one
|
||||
value-free line; a no-op double Reset logs nothing.
|
||||
"""
|
||||
if key not in CATALOG:
|
||||
raise HTTPException(status_code=404, detail=f"Unknown setting {key}")
|
||||
result = db.execute(
|
||||
delete(AppSetting).where(AppSetting.key == key).returning(AppSetting.key)
|
||||
)
|
||||
db.commit()
|
||||
if len(result.all()):
|
||||
logger.info("settings action=clear key=%s", key)
|
||||
invalidate_settings()
|
||||
return _item(key, _rows(db), get_settings())
|
||||
|
||||
|
||||
@router.delete("", response_model=AdminSettingsCleared)
|
||||
def reset_all(db: Session = Depends(get_db)) -> AdminSettingsCleared: # noqa: B008
|
||||
"""The panel's "Clear all overrides": delete every row (the env
|
||||
values win again for all 75 settings), invalidate (task 03) and
|
||||
report how many rows went."""
|
||||
result = db.execute(delete(AppSetting).returning(AppSetting.key))
|
||||
db.commit()
|
||||
count = len(result.all())
|
||||
logger.info("settings action=clear_all count=%d", count)
|
||||
invalidate_settings()
|
||||
return AdminSettingsCleared(cleared=count)
|
||||
@@ -326,6 +326,22 @@ def get_llm() -> LLMClient:
|
||||
return _llm
|
||||
|
||||
|
||||
def reset_runtime_caches() -> None:
|
||||
"""Drop the settings-derived singletons (phase 168, task 03).
|
||||
|
||||
Called by ``invalidate_settings()`` after the admin saves. The next
|
||||
turn rebuilds the client from the CURRENT ``get_settings()`` — an
|
||||
in-flight turn keeps the client it started with (its ``AsyncOpenAI``
|
||||
transport is mid-request; swapping it would kill the stream), which
|
||||
is the same accepted-semantics note as A12's in-process state.
|
||||
``_chat_active`` is deliberately untouched: it counts live turns, and
|
||||
the new ``chat_max_concurrent`` applies as permits are taken.
|
||||
"""
|
||||
global _llm, _chat_semaphore
|
||||
_llm = None
|
||||
_chat_semaphore = None
|
||||
|
||||
|
||||
def _whole_line_number(value: Any) -> bool:
|
||||
"""A ``read`` range argument's whole-line-number guard (phase 131,
|
||||
task 04).
|
||||
|
||||
+34
-1
@@ -905,4 +905,37 @@ class Settings(BaseSettings):
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
return Settings()
|
||||
"""Effective settings: env/.env with the admin's DB overrides layered
|
||||
on top (phase 168, TODO L3: "The database value should override
|
||||
the env var").
|
||||
|
||||
The DB half lives in :mod:`app.core.settings_overlay` — imported
|
||||
LAZILY, on purpose: ``app.core.*`` imports ``app.config`` at module
|
||||
scope (and ``app.db`` → ``app.config`` runs at import time), so a
|
||||
module-scope import here would deadlock the import graph. The
|
||||
``@lru_cache`` is unchanged from pre-phase — every existing
|
||||
``get_settings.cache_clear()`` call site keeps working (they only
|
||||
retune the environment, so skipping the reset hooks there is
|
||||
correct), and :func:`invalidate_settings` is the save-side wrapper
|
||||
that also runs the runtime reset hooks (phase 168, task 03).
|
||||
"""
|
||||
from app.core.settings_overlay import resolve_settings
|
||||
|
||||
return resolve_settings()
|
||||
|
||||
|
||||
def invalidate_settings() -> None:
|
||||
"""Save-side settings refresh (phase 168, task 02/03).
|
||||
|
||||
Clears the ``get_settings`` cache (the next resolution re-reads the
|
||||
DB overrides — no restart) and runs the registered runtime reset
|
||||
hooks (task 03: the in-process singletons — the chat LLM client
|
||||
and semaphore — rebuild from the fresh settings). The six existing
|
||||
``get_settings.cache_clear()`` test call sites stay the env-only
|
||||
retune path: they re-resolve without running the hooks, which is
|
||||
exactly what a test that only re-tunes the environment wants.
|
||||
"""
|
||||
get_settings.cache_clear()
|
||||
from app.core.settings_overlay import run_invalidators
|
||||
|
||||
run_invalidators()
|
||||
|
||||
@@ -177,6 +177,7 @@ HTML_PAGES: tuple[str, ...] = (
|
||||
"/history.html", # phase 50: the admin saved-chats page
|
||||
"/tokens.html", # phase 79 task 06: the admin tokens page (shell route)
|
||||
"/theme.html", # phase 91 task 04: the admin theme page (shell route)
|
||||
"/settings.html", # phase 168: the admin settings view (shell route)
|
||||
# phase 51: the shared page's STATIC path (the static mount serves
|
||||
# shared.html at /shared.html as well as the real route serves the
|
||||
# dynamic /shared/<token> — both must carry the no-cache + ?v=
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
"""The admin Settings panel's catalogue (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "I want an admin panel that can edit anything in .env
|
||||
and save it to the database."
|
||||
|
||||
This module is the single source of truth for what the panel shows and
|
||||
what the DB-over-env overlay (``app.core.settings_overlay``) is allowed
|
||||
to do with each of the ``Settings`` fields:
|
||||
|
||||
* :data:`CATALOG` maps every ``Settings`` field name to its
|
||||
presentation metadata (``group`` + ``label``) and its capability
|
||||
flags:
|
||||
|
||||
- ``secret`` — write-only (LOCKED A4): the API reports ``is_set``,
|
||||
never the plaintext; a saved value is masked on read.
|
||||
- ``restart_required`` — boot-wired (LOCKED A4): the middleware /
|
||||
engine / logging / static mount read it once at boot, so the
|
||||
override applies on the NEXT boot (the panel shows the badge).
|
||||
- ``overridable`` — ``False`` for LOCKED A3's ``database_url``
|
||||
(it builds the very engine that would read the override — a
|
||||
chicken-and-egg brick) and ``embedding_dim`` (A6 fixes
|
||||
``chunks.embedding`` at table-creation; a dim override would
|
||||
corrupt retrieval).
|
||||
|
||||
The panel (task 05) is GENERATED from this catalogue — never
|
||||
hand-written — so a new ``Settings`` field appears there only once
|
||||
it has a :class:`SettingMeta`; the completeness pin
|
||||
(``tests/unit/test_settings_catalog.py``: ``set(CATALOG) ==
|
||||
set(Settings.model_fields)``) fails the build the moment a field is
|
||||
added without metadata, so it can never silently disappear from the
|
||||
panel.
|
||||
* :data:`GROUPS` is the display order of the groups — it mirrors
|
||||
``app/config.py``'s own section comments (App, HTTP response
|
||||
compression / byte budget, UI customization, Database, LLM, Chat
|
||||
history, RAG tuning, …).
|
||||
* :data:`SECRETS`, :data:`RESTART_REQUIRED` and
|
||||
:data:`NON_OVERRIDABLE` are DERIVED from :data:`CATALOG` — never
|
||||
re-listed by hand, so they cannot drift.
|
||||
|
||||
:func:`env_var` keeps the ``BOR_`` prefix a pure presentation concern
|
||||
(D1): the stored key is the bare ``Settings`` field name
|
||||
(``relevance_threshold``), ``"BOR_" + key.upper()`` is what the panel
|
||||
and the API show, and a stored prefix could double up — so it is
|
||||
derived, never stored.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SettingMeta:
|
||||
"""One settings field's panel metadata + capability flags."""
|
||||
|
||||
key: str
|
||||
group: str
|
||||
label: str
|
||||
#: Write-only (LOCKED A4) — masked in responses, ``is_set`` only.
|
||||
secret: bool = False
|
||||
#: Boot-wired (LOCKED A4) — applies on the next boot.
|
||||
restart_required: bool = False
|
||||
#: ``False`` = read-only row (LOCKED A3).
|
||||
overridable: bool = True
|
||||
|
||||
|
||||
#: The ``BOR_`` prefix is presentation only (D1) — see :func:`env_var`.
|
||||
ENV_PREFIX = "BOR_"
|
||||
|
||||
#: Display order of the groups (mirrors ``app/config.py``'s section
|
||||
#: comments); :data:`CATALOG` is keyed in the same order.
|
||||
GROUPS: tuple[str, ...] = (
|
||||
"App & branding",
|
||||
"HTTP & logging",
|
||||
"Database",
|
||||
"LLM (aipi)",
|
||||
"Chat history",
|
||||
"Retrieval & honesty",
|
||||
"Chunking & embedding",
|
||||
"Prompt budgets",
|
||||
"Agent tools",
|
||||
"Import & sources",
|
||||
"Images",
|
||||
"PDF",
|
||||
"Docs push",
|
||||
"Admin & security",
|
||||
)
|
||||
|
||||
#: Every ``Settings`` field, in :data:`GROUPS` display order. Completeness
|
||||
#: is pinned by ``tests/unit/test_settings_catalog.py`` (the diff is in
|
||||
#: the failure message — a new setting without metadata fails the build).
|
||||
CATALOG: dict[str, SettingMeta] = {
|
||||
# --- App & branding ---
|
||||
"app_name": SettingMeta("app_name", "App & branding", "App name"),
|
||||
"app_version": SettingMeta("app_version", "App & branding", "App version"),
|
||||
"environment": SettingMeta("environment", "App & branding", "Environment"),
|
||||
"input_placeholder": SettingMeta(
|
||||
"input_placeholder", "App & branding", "Input placeholder"
|
||||
),
|
||||
"footer_text": SettingMeta("footer_text", "App & branding", "Footer text"),
|
||||
"suggestions": SettingMeta(
|
||||
"suggestions", "App & branding", "Suggestions (JSON array)"
|
||||
),
|
||||
# --- HTTP & logging ---
|
||||
"log_level": SettingMeta(
|
||||
"log_level", "HTTP & logging", "Log level", restart_required=True
|
||||
),
|
||||
"log_large_response_bytes": SettingMeta(
|
||||
"log_large_response_bytes",
|
||||
"HTTP & logging",
|
||||
"Log large response bytes",
|
||||
),
|
||||
"compress_min_bytes": SettingMeta(
|
||||
"compress_min_bytes", "HTTP & logging", "Compress min bytes",
|
||||
restart_required=True,
|
||||
),
|
||||
"static_dir": SettingMeta(
|
||||
"static_dir", "HTTP & logging", "Static dir", restart_required=True
|
||||
),
|
||||
# --- Database ---
|
||||
"database_url": SettingMeta(
|
||||
"database_url", "Database", "Database URL",
|
||||
secret=True, overridable=False,
|
||||
),
|
||||
"db_pool_size": SettingMeta(
|
||||
"db_pool_size", "Database", "DB pool size", restart_required=True
|
||||
),
|
||||
"db_pool_max_overflow": SettingMeta(
|
||||
"db_pool_max_overflow", "Database", "DB pool max overflow",
|
||||
restart_required=True,
|
||||
),
|
||||
"db_pool_recycle": SettingMeta(
|
||||
"db_pool_recycle", "Database", "DB pool recycle", restart_required=True
|
||||
),
|
||||
"chat_max_concurrent": SettingMeta(
|
||||
"chat_max_concurrent", "Database", "Chat max concurrent"
|
||||
),
|
||||
# --- LLM (aipi) ---
|
||||
"llm_base_url": SettingMeta(
|
||||
"llm_base_url", "LLM (aipi)", "LLM base URL"
|
||||
),
|
||||
"llm_api_key": SettingMeta(
|
||||
"llm_api_key", "LLM (aipi)", "LLM API key", secret=True
|
||||
),
|
||||
"llm_chat_model": SettingMeta(
|
||||
"llm_chat_model", "LLM (aipi)", "LLM chat model"
|
||||
),
|
||||
"llm_embed_model": SettingMeta(
|
||||
"llm_embed_model", "LLM (aipi)", "LLM embed model"
|
||||
),
|
||||
"llm_summary_model": SettingMeta(
|
||||
"llm_summary_model", "LLM (aipi)", "LLM summary model"
|
||||
),
|
||||
"stream_thinking": SettingMeta(
|
||||
"stream_thinking", "LLM (aipi)", "Stream thinking"
|
||||
),
|
||||
"llm_retries": SettingMeta("llm_retries", "LLM (aipi)", "LLM retries"),
|
||||
"llm_retry_delay": SettingMeta(
|
||||
"llm_retry_delay", "LLM (aipi)", "LLM retry delay"
|
||||
),
|
||||
"llm_timeout": SettingMeta("llm_timeout", "LLM (aipi)", "LLM timeout (s)"),
|
||||
# --- Chat history ---
|
||||
"history_max_turns": SettingMeta(
|
||||
"history_max_turns", "Chat history", "History max turns"
|
||||
),
|
||||
"history_max_chars": SettingMeta(
|
||||
"history_max_chars", "Chat history", "History max chars"
|
||||
),
|
||||
# --- Retrieval & honesty ---
|
||||
"embedding_dim": SettingMeta(
|
||||
"embedding_dim", "Retrieval & honesty", "Embedding dim",
|
||||
overridable=False,
|
||||
),
|
||||
"top_n_docs": SettingMeta("top_n_docs", "Retrieval & honesty", "Top N docs"),
|
||||
"relevance_threshold": SettingMeta(
|
||||
"relevance_threshold", "Retrieval & honesty", "Relevance threshold"
|
||||
),
|
||||
"lexical_support_floor": SettingMeta(
|
||||
"lexical_support_floor", "Retrieval & honesty", "Lexical support floor"
|
||||
),
|
||||
"source_usefulness_floor": SettingMeta(
|
||||
"source_usefulness_floor",
|
||||
"Retrieval & honesty",
|
||||
"Source usefulness floor",
|
||||
),
|
||||
"related_max_docs": SettingMeta(
|
||||
"related_max_docs", "Retrieval & honesty", "Related max docs"
|
||||
),
|
||||
"suggested_docs": SettingMeta(
|
||||
"suggested_docs", "Retrieval & honesty", "Suggested docs"
|
||||
),
|
||||
"suggestion_preview_chars": SettingMeta(
|
||||
"suggestion_preview_chars",
|
||||
"Retrieval & honesty",
|
||||
"Suggestion preview chars",
|
||||
),
|
||||
"hybrid_vector_candidates": SettingMeta(
|
||||
"hybrid_vector_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid vector candidates",
|
||||
),
|
||||
"hybrid_lexical_candidates": SettingMeta(
|
||||
"hybrid_lexical_candidates",
|
||||
"Retrieval & honesty",
|
||||
"Hybrid lexical candidates",
|
||||
),
|
||||
"rrf_k": SettingMeta("rrf_k", "Retrieval & honesty", "RRF k"),
|
||||
"recency_boost": SettingMeta(
|
||||
"recency_boost", "Retrieval & honesty", "Recency boost"
|
||||
),
|
||||
"recency_half_life_days": SettingMeta(
|
||||
"recency_half_life_days",
|
||||
"Retrieval & honesty",
|
||||
"Recency half-life days",
|
||||
),
|
||||
"name_hit_bonus": SettingMeta(
|
||||
"name_hit_bonus", "Retrieval & honesty", "Name hit bonus"
|
||||
),
|
||||
# --- Chunking & embedding ---
|
||||
"chunk_target_chars": SettingMeta(
|
||||
"chunk_target_chars", "Chunking & embedding", "Chunk target chars"
|
||||
),
|
||||
"chunk_overlap_chars": SettingMeta(
|
||||
"chunk_overlap_chars", "Chunking & embedding", "Chunk overlap chars"
|
||||
),
|
||||
"embed_batch_size": SettingMeta(
|
||||
"embed_batch_size", "Chunking & embedding", "Embed batch size"
|
||||
),
|
||||
"embed_question_max_chars": SettingMeta(
|
||||
"embed_question_max_chars",
|
||||
"Chunking & embedding",
|
||||
"Embed question max chars",
|
||||
),
|
||||
# --- Prompt budgets ---
|
||||
"max_output_tokens": SettingMeta(
|
||||
"max_output_tokens", "Prompt budgets", "Max output tokens"
|
||||
),
|
||||
"steering_max_chars": SettingMeta(
|
||||
"steering_max_chars", "Prompt budgets", "Steering max chars"
|
||||
),
|
||||
"summary_max_chars": SettingMeta(
|
||||
"summary_max_chars", "Prompt budgets", "Summary max chars"
|
||||
),
|
||||
"kb_overview_max_chars": SettingMeta(
|
||||
"kb_overview_max_chars", "Prompt budgets", "KB overview max chars"
|
||||
),
|
||||
"overview_input_max_chars": SettingMeta(
|
||||
"overview_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Overview input max chars",
|
||||
),
|
||||
"folder_summary_input_max_chars": SettingMeta(
|
||||
"folder_summary_input_max_chars",
|
||||
"Prompt budgets",
|
||||
"Folder summary input max chars",
|
||||
),
|
||||
# --- Agent tools ---
|
||||
"agent_max_rounds": SettingMeta(
|
||||
"agent_max_rounds", "Agent tools", "Agent max rounds"
|
||||
),
|
||||
"read_max_chars": SettingMeta(
|
||||
"read_max_chars", "Agent tools", "Read max chars"
|
||||
),
|
||||
# --- Import & sources ---
|
||||
"import_extensions": SettingMeta(
|
||||
"import_extensions", "Import & sources", "Import extensions (CSV)"
|
||||
),
|
||||
"git_sources": SettingMeta(
|
||||
"git_sources", "Import & sources", "Git sources (CSV)"
|
||||
),
|
||||
"sources_dir": SettingMeta(
|
||||
"sources_dir", "Import & sources", "Sources dir"
|
||||
),
|
||||
"upload_dir": SettingMeta("upload_dir", "Import & sources", "Upload dir"),
|
||||
"upload_max_mb": SettingMeta(
|
||||
"upload_max_mb", "Import & sources", "Upload max MB"
|
||||
),
|
||||
# --- Images ---
|
||||
"images": SettingMeta("images", "Images", "Images"),
|
||||
"image_extensions": SettingMeta(
|
||||
"image_extensions", "Images", "Image extensions (CSV)"
|
||||
),
|
||||
"image_dir": SettingMeta("image_dir", "Images", "Image dir"),
|
||||
"chat_image_dir": SettingMeta(
|
||||
"chat_image_dir", "Images", "Chat image dir"
|
||||
),
|
||||
"chat_image_max_mb": SettingMeta(
|
||||
"chat_image_max_mb", "Images", "Chat image max MB"
|
||||
),
|
||||
"answer_images": SettingMeta("answer_images", "Images", "Answer images"),
|
||||
# --- PDF ---
|
||||
"pdf_pictorial": SettingMeta("pdf_pictorial", "PDF", "PDF pictorial"),
|
||||
"pdf_pictorial_min_chars": SettingMeta(
|
||||
"pdf_pictorial_min_chars", "PDF", "PDF pictorial min chars"
|
||||
),
|
||||
"pdf_image_skip_blank": SettingMeta(
|
||||
"pdf_image_skip_blank", "PDF", "PDF image skip blank"
|
||||
),
|
||||
"pdf_page_dir": SettingMeta("pdf_page_dir", "PDF", "PDF page dir"),
|
||||
# --- Docs push ---
|
||||
"docs_repo": SettingMeta("docs_repo", "Docs push", "Docs repo"),
|
||||
"docs_branch": SettingMeta("docs_branch", "Docs push", "Docs branch"),
|
||||
"docs_base_branch": SettingMeta(
|
||||
"docs_base_branch", "Docs push", "Docs base branch"
|
||||
),
|
||||
"docs_work_dir": SettingMeta("docs_work_dir", "Docs push", "Docs work dir"),
|
||||
# --- Admin & security ---
|
||||
"admin_password": SettingMeta(
|
||||
"admin_password", "Admin & security", "Admin password", secret=True
|
||||
),
|
||||
"session_secret": SettingMeta(
|
||||
"session_secret", "Admin & security", "Session secret",
|
||||
secret=True, restart_required=True,
|
||||
),
|
||||
"session_max_age": SettingMeta(
|
||||
"session_max_age", "Admin & security", "Session max age",
|
||||
restart_required=True,
|
||||
),
|
||||
"session_cookie": SettingMeta(
|
||||
"session_cookie", "Admin & security", "Session cookie",
|
||||
restart_required=True,
|
||||
),
|
||||
}
|
||||
|
||||
#: The masked set (LOCKED A4: write-only; ``database_url`` is read-only
|
||||
#: but its URL carries credentials, so it is masked too) — derived,
|
||||
#: never re-listed by hand.
|
||||
SECRETS: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.secret
|
||||
)
|
||||
|
||||
#: The boot-wired set (LOCKED A4: logging, the gzip middleware, the
|
||||
#: static mount, ``create_engine``, ``SessionMiddleware``) — derived.
|
||||
RESTART_REQUIRED: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if meta.restart_required
|
||||
)
|
||||
|
||||
#: The read-only set (LOCKED A3) — derived.
|
||||
NON_OVERRIDABLE: frozenset[str] = frozenset(
|
||||
key for key, meta in CATALOG.items() if not meta.overridable
|
||||
)
|
||||
|
||||
|
||||
def env_var(key: str) -> str:
|
||||
"""The env var for one bare setting key — presentation only (D1).
|
||||
|
||||
The stored key is the bare ``Settings`` field name; the ``BOR_``
|
||||
form is what the panel and the API show, never what is stored
|
||||
(a stored prefix could double up).
|
||||
"""
|
||||
return ENV_PREFIX + key.upper()
|
||||
@@ -0,0 +1,281 @@
|
||||
"""The DB-over-env settings overlay (phase 168, task 02).
|
||||
|
||||
``TODO.md`` L3: "…save it to the database. The database value should
|
||||
override the env var."
|
||||
|
||||
:func:`resolve_settings` layers the ``app_settings`` rows (phase 168
|
||||
task 01: bare ``Settings`` field name → env-native string) over the
|
||||
environment and hands every existing consumer one valid ``Settings`` —
|
||||
through :func:`app.config.get_settings` (which delegates here, lazily,
|
||||
under its unchanged ``lru_cache``). No consumer changes, no restart
|
||||
(D2). The kill switch is env-only: ``BOR_SETTINGS_DB=0`` (read
|
||||
straight from ``os.environ`` per call — it is not a ``Settings`` field,
|
||||
so the escape hatch cannot disable itself).
|
||||
|
||||
Fail loud on write, fail soft on read (LOCKED A5, D5): a write
|
||||
validates the FULL candidate set (task 04); a read that finds an
|
||||
unusable row (hand-edited, or a validator changed in a later release)
|
||||
skips THAT key, logs ``settings override ignored key=… reason=…`` and
|
||||
applies the rest — a DB value must never brick boot, because a
|
||||
bricked app cannot be un-bricked from the panel.
|
||||
|
||||
The overlay's engine is a lazily-created ``NullPool`` engine built from
|
||||
``Settings().database_url`` (never the app's real pool): importing
|
||||
``app.db`` from here would be a cycle (``app.db`` → ``app.config`` at
|
||||
import time), and the pool whose size this table can change must not
|
||||
be the one that reads the table. ``NullPool`` = one connection per
|
||||
resolution, none held.
|
||||
|
||||
A missing ``app_settings`` table (fresh deployment, migrations not
|
||||
applied) means "no overrides" + one DEBUG line per process (K3 — a
|
||||
pre-migration deployment is a normal state); any other DB failure is
|
||||
one WARNING per process naming the exception class ("settings
|
||||
overrides unavailable — env values in effect"). Both "once per process"
|
||||
guards are module-level flags, reset by :func:`reset_state_for_tests`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any, get_origin
|
||||
|
||||
from pydantic import ValidationError
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.engine import Engine
|
||||
from sqlalchemy.exc import ProgrammingError, SQLAlchemyError
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
from app.config import Settings
|
||||
from app.core.settings_catalog import CATALOG
|
||||
|
||||
logger = logging.getLogger("app.settings")
|
||||
|
||||
#: ``BOR_SETTINGS_DB`` — env-only, read per call from ``os.environ`` so
|
||||
#: tests can flip it; unset/``1`` = on, the house bool off-spellings
|
||||
#: are off. It is NOT a ``Settings`` field, so the escape hatch cannot
|
||||
#: disable itself from the database (D6).
|
||||
_DISABLED_SPELLOS = frozenset({"0", "false", "no", "off"})
|
||||
|
||||
_engine: Engine | None = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators: list[Callable[[], None]] = []
|
||||
|
||||
#: The one ``list[str]`` field (derived — a new list field would join
|
||||
#: automatically); every other field is a scalar or CSV string (K2).
|
||||
LIST_FIELDS: frozenset[str] = frozenset(
|
||||
name
|
||||
for name, field in Settings.model_fields.items()
|
||||
if get_origin(field.annotation) in (list, tuple)
|
||||
)
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
"""The ``BOR_SETTINGS_DB`` kill switch — read per call (D6)."""
|
||||
value = os.environ.get("BOR_SETTINGS_DB", "1")
|
||||
return value.strip().lower() not in _DISABLED_SPELLOS
|
||||
|
||||
|
||||
def _get_engine() -> Engine:
|
||||
"""The overlay's lazily-created ``NullPool`` engine (see module
|
||||
docstring — never the app's real pool: import cycle + the pool
|
||||
whose size this table can change must not read the table)."""
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(
|
||||
Settings().database_url, poolclass=NullPool, pool_pre_ping=True
|
||||
)
|
||||
return _engine
|
||||
|
||||
|
||||
def load_overrides() -> dict[str, str]:
|
||||
"""``SELECT key, value FROM app_settings ORDER BY key`` — fail-soft.
|
||||
|
||||
A missing table (pre-migration deployment) is one DEBUG line per
|
||||
process (K3); any other DB failure is one WARNING per process
|
||||
naming the exception class. Both return ``{}`` — the env values
|
||||
stay in effect, and the NEXT resolution retries (a boot-time miss
|
||||
is never a permanently cached env-only view — the ``create_app``
|
||||
prime, task 02 step 4).
|
||||
"""
|
||||
global _warned_db_failure, _debugged_missing_table
|
||||
try:
|
||||
with _get_engine().connect() as conn:
|
||||
rows = conn.execute(text("SELECT key, value FROM app_settings ORDER BY key"))
|
||||
return {key: value for key, value in rows}
|
||||
except ProgrammingError:
|
||||
# Undefined table (psycopg ``UndefinedTable`` → SQLAlchemy
|
||||
# ProgrammingError): the pre-migration deployment — DEBUG (K3).
|
||||
if not _debugged_missing_table:
|
||||
_debugged_missing_table = True
|
||||
logger.debug(
|
||||
"app_settings table missing — no DB overrides in effect "
|
||||
"(pre-migration deployment, phase 168 K3)"
|
||||
)
|
||||
return {}
|
||||
except (SQLAlchemyError, OSError) as exc:
|
||||
if not _warned_db_failure:
|
||||
_warned_db_failure = True
|
||||
logger.warning(
|
||||
"settings overrides unavailable — env values in effect (%s)",
|
||||
type(exc).__name__,
|
||||
)
|
||||
return {}
|
||||
|
||||
|
||||
def _coerce(field_name: str, raw: str) -> Any:
|
||||
"""Coerce one env-native string row value for the candidate set.
|
||||
|
||||
The one ``list[str]`` field (``suggestions``) is ``json.loads``-ed
|
||||
— it recovers exactly what the env spelling (a JSON array, K2)
|
||||
wrote. Every other field is the raw string: pydantic-settings
|
||||
lax-coerces ``str`` → ``int`` / ``float`` / ``bool``, and init
|
||||
kwargs rank ABOVE env in pydantic-settings — precisely "the
|
||||
database value overrides the env var" (D2). A malformed
|
||||
``suggestions`` JSON raises :class:`ValueError`, which
|
||||
:func:`merge` turns into a per-key skip (LOCKED A5: one bad row
|
||||
costs one setting, never the whole table). The return type is
|
||||
``Any`` on purpose — the value is whatever pydantic will
|
||||
lax-coerce, decided in exactly one place (the ``Settings``
|
||||
validators).
|
||||
"""
|
||||
if field_name in LIST_FIELDS:
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"not a JSON array ({exc.msg})") from exc
|
||||
return raw
|
||||
|
||||
|
||||
def merge(
|
||||
rows: Mapping[str, str],
|
||||
) -> tuple[Settings, list[tuple[str, str]]]:
|
||||
"""Layer ``rows`` over the env base (``Settings()``) and validate.
|
||||
|
||||
Returns ``(settings, skipped)`` — ``skipped`` is the
|
||||
``(key, reason)`` pairs the read-side fail-soft (LOCKED A5)
|
||||
dropped so one unusable row costs one setting, not the whole
|
||||
table. Dropped keys: an orphan row (not a ``Settings`` field —
|
||||
the setting was removed in a later release), a LOCKED A3 key
|
||||
(a hand-inserted ``database_url`` / ``embedding_dim`` row must
|
||||
never hijack the engine / the fixed 768-dim table), a malformed
|
||||
``suggestions`` JSON, and any value the real ``Settings``
|
||||
validators reject (retried per field — ``error["loc"][0]``). Each
|
||||
dropped key is logged once at ERROR
|
||||
(``settings override ignored key=… reason=…``).
|
||||
"""
|
||||
kwargs: dict[str, Any] = {}
|
||||
skipped: list[tuple[str, str]] = []
|
||||
for key, raw in rows.items():
|
||||
if key not in Settings.model_fields:
|
||||
skipped.append((key, "unknown setting (orphan row from a removed field)"))
|
||||
continue
|
||||
if not CATALOG[key].overridable:
|
||||
skipped.append((key, "not DB-overridable (LOCKED A3 — read-only)"))
|
||||
continue
|
||||
try:
|
||||
kwargs[key] = _coerce(key, raw)
|
||||
except ValueError as exc:
|
||||
skipped.append((key, str(exc)))
|
||||
while True:
|
||||
try:
|
||||
settings = Settings(**kwargs)
|
||||
break
|
||||
except ValidationError as exc:
|
||||
err = exc.errors()[0]
|
||||
bad = str(err["loc"][0])
|
||||
if bad in kwargs:
|
||||
# The offending row is an override — drop it and retry
|
||||
# (LOCKED A5: one bad row costs one setting).
|
||||
kwargs.pop(bad)
|
||||
skipped.append((bad, err["msg"]))
|
||||
continue
|
||||
# The validator blames a field with NO override — its value
|
||||
# comes from env/.env and collides with a cross-field
|
||||
# override (e.g. the env lexical_support_floor above a
|
||||
# DB-overridden relevance_threshold). The env base alone is
|
||||
# always valid (the app booted with it pre-phase), so try
|
||||
# dropping the override(s) until the candidate set is valid
|
||||
# again — one unresolvable collision costs the colliding
|
||||
# row, the rest of the table stands.
|
||||
resolved = False
|
||||
for candidate in list(kwargs):
|
||||
trial = dict(kwargs)
|
||||
trial.pop(candidate)
|
||||
try:
|
||||
settings = Settings(**trial)
|
||||
except ValidationError:
|
||||
continue
|
||||
skipped.append(
|
||||
(
|
||||
candidate,
|
||||
f"cross-field collision with env/base {bad}: {err['msg']} "
|
||||
"— override skipped, env value for the other field stands",
|
||||
)
|
||||
)
|
||||
kwargs = trial
|
||||
resolved = True
|
||||
break
|
||||
if not resolved:
|
||||
# Nothing to drop (no overrides at all) or no single drop
|
||||
# heals the set: the env base alone would boot (pre-phase
|
||||
# proof), so return it — overrides skipped, boot intact.
|
||||
skipped.append(
|
||||
(
|
||||
bad,
|
||||
f"unresolvable cross-field collision: {err['msg']} — "
|
||||
"all overrides skipped, env values in effect",
|
||||
)
|
||||
)
|
||||
settings = Settings()
|
||||
break
|
||||
for key, reason in skipped:
|
||||
logger.error("settings override ignored key=%s reason=%s", key, reason)
|
||||
return settings, skipped
|
||||
|
||||
|
||||
def resolve_settings() -> Settings:
|
||||
"""The whole overlay: env base + kill switch + DB rows.
|
||||
|
||||
Never calls :func:`app.config.get_settings` (recursion guard — that
|
||||
resolver delegates HERE): ``Settings()`` is the env-only entry
|
||||
point, and the kill switch short-circuits before any DB access.
|
||||
"""
|
||||
if not enabled():
|
||||
return Settings()
|
||||
settings, _skipped = merge(load_overrides())
|
||||
return settings
|
||||
|
||||
|
||||
def register_invalidator(fn: Callable[[], None]) -> None:
|
||||
"""Register a runtime reset hook (task 03: the chat LLM client +
|
||||
semaphore). Hooks run on every :func:`app.config.invalidate_settings`
|
||||
— after the cache clear — so the next resolution rebuilds the
|
||||
singletons from the fresh settings."""
|
||||
_invalidators.append(fn)
|
||||
|
||||
|
||||
def run_invalidators() -> None:
|
||||
"""Run the registered reset hooks, in order. A hook that raises is
|
||||
logged, never propagated (a failed reset must not undo a
|
||||
successful save — LOCKED A5's "the save stands")."""
|
||||
for fn in list(_invalidators):
|
||||
try:
|
||||
fn()
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"settings invalidator failed — the save stands, this "
|
||||
"runtime reset did not run"
|
||||
)
|
||||
|
||||
|
||||
def reset_state_for_tests() -> None:
|
||||
"""Clear the module engine + the once-per-process flags + the hook
|
||||
list (the unit suites flip the kill switch and stub the engine)."""
|
||||
global _engine, _warned_db_failure, _debugged_missing_table
|
||||
_engine = None
|
||||
_warned_db_failure = False
|
||||
_debugged_missing_table = False
|
||||
_invalidators.clear()
|
||||
+37
-1
@@ -20,7 +20,9 @@ from fastapi.staticfiles import StaticFiles
|
||||
from starlette.middleware.sessions import SessionMiddleware
|
||||
from starlette.responses import FileResponse
|
||||
|
||||
from app.api.admin_settings import router as admin_settings_router
|
||||
from app.api.auth import router as auth_router
|
||||
from app.api.chat import reset_runtime_caches
|
||||
from app.api.chat import router as chat_router
|
||||
from app.api.chat_images import router as chat_images_router
|
||||
from app.api.chats import (
|
||||
@@ -42,7 +44,7 @@ from app.api.suggestions import router as suggestions_router
|
||||
from app.api.sync import router as sync_router
|
||||
from app.api.tokens import router as tokens_router
|
||||
from app.api.ui_settings import router as ui_settings_router
|
||||
from app.config import get_settings
|
||||
from app.config import get_settings, invalidate_settings
|
||||
from app.core.auth import ensure_admin_configured
|
||||
from app.core.caching import configure_caching
|
||||
from app.core.compression import SelectiveGZipMiddleware
|
||||
@@ -50,6 +52,8 @@ from app.core.debugging import configure_debugging
|
||||
from app.core.logging import configure_logging
|
||||
from app.core.response_budget import ResponseBudgetMiddleware
|
||||
from app.core.security_headers import SecurityHeadersMiddleware
|
||||
from app.core.settings_overlay import enabled as settings_db_enabled
|
||||
from app.core.settings_overlay import load_overrides, register_invalidator
|
||||
|
||||
configure_logging()
|
||||
configure_debugging()
|
||||
@@ -97,6 +101,33 @@ def create_app() -> FastAPI:
|
||||
# variable(s) — the app never starts in a half-authenticated state.
|
||||
ensure_admin_configured(settings)
|
||||
|
||||
# Phase 168 (task 02, D2/D3): prime the DB-over-env resolver now
|
||||
# that auth is verified. The module-level ``settings`` above stays
|
||||
# the pre-boot value the middleware wiring below uses (LOCKED A4:
|
||||
# those boot-wired settings apply on the NEXT boot); every consumer
|
||||
# that calls get_settings() from here on resolves DB-over-env — no
|
||||
# restart. invalidate_settings() runs the runtime reset hooks
|
||||
# (task 03) and clears the cache; a boot-time DB miss is fail-soft
|
||||
# (one WARNING, env values in effect) and the next resolution
|
||||
# retries, so the prime can never cache an env-only view forever.
|
||||
invalidate_settings()
|
||||
if settings_db_enabled():
|
||||
get_settings()
|
||||
logger.info("settings overrides loaded count=%d", len(load_overrides()))
|
||||
else:
|
||||
get_settings()
|
||||
logger.info("settings overrides loaded source=env-only disabled")
|
||||
|
||||
# Phase 168 (task 03): the settings panel's Save must retune the
|
||||
# running process, and app/core must never import app/api (PLAN §3's
|
||||
# direction) — so create_app(), the composition root, wires the
|
||||
# reset hook. It runs on every invalidate_settings() after the cache
|
||||
# clear: the next chat turn rebuilds the shared LLMClient and the
|
||||
# concurrency semaphore from the CURRENT settings (no restart), while
|
||||
# in-flight turns keep the client they started with (A12 semantics).
|
||||
register_invalidator(reset_runtime_caches)
|
||||
logger.info("settings invalidator registered name=chat_runtime_caches")
|
||||
|
||||
app = FastAPI(title=settings.app_name, version=settings.app_version)
|
||||
|
||||
# Signed single-admin session cookie (Starlette middleware, itsdangerous
|
||||
@@ -134,6 +165,10 @@ def create_app() -> FastAPI:
|
||||
# single ui_settings row — the Theme tab's persistence) — admin-only
|
||||
# (router-wide require_admin; anonymous AND token users stay 403).
|
||||
app.include_router(ui_settings_router, prefix="/api")
|
||||
# Phase 168 (task 04): the admin settings panel surface (read/write
|
||||
# the app_settings overrides — DB-over-env, live on Save) — admin-only
|
||||
# (router-wide require_admin; a token user stays 403).
|
||||
app.include_router(admin_settings_router, prefix="/api")
|
||||
# Phase 51: the anonymous shared-chat read — NO admin dependency.
|
||||
# /api/shared/<token> is the JSON snapshot; /shared/<token> (the
|
||||
# page route below, registered without a prefix) is the page.
|
||||
@@ -197,6 +232,7 @@ def create_app() -> FastAPI:
|
||||
"/history.html",
|
||||
"/tokens.html", # phase 79 task 06: the Tokens view
|
||||
"/theme.html", # phase 91 task 04: the Theme view (shell route)
|
||||
"/settings.html", # phase 168 task 05: the Settings view (shell route)
|
||||
),
|
||||
)
|
||||
app.mount("/", StaticFiles(directory=static_dir, html=True), name="static")
|
||||
|
||||
@@ -70,6 +70,13 @@ Data model — see ``.agents/PLAN.md`` §Data Model:
|
||||
``lite`` summary the drill-down
|
||||
``ls`` shows next to each folder (phase 94;
|
||||
``folder_path = ""`` = the source root).
|
||||
* ``app_settings`` — one row per admin setting override (phase
|
||||
168, TODO.md L3): the Settings panel persists
|
||||
what used to be ``.env``-only ``BOR_*`` knobs
|
||||
here, keyed by the BARE ``Settings`` field
|
||||
name, the value in its env-native string form;
|
||||
the row WINS over the environment (resolved
|
||||
inside ``app.config.get_settings``).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -649,3 +656,44 @@ class FolderSummary(Base):
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), onupdate=func.now()
|
||||
)
|
||||
|
||||
|
||||
class AppSetting(Base):
|
||||
"""One admin setting override (phase 168, task 01).
|
||||
|
||||
``TODO.md`` L3: the admin Settings panel persists what used to be
|
||||
``.env``-only knobs here, and the row WINS over the environment
|
||||
(resolved by ``app.core.settings_overlay``, consumed by
|
||||
``app.config.get_settings``). The narrow key/value shape is the
|
||||
whole design (phase 168 D1): one row per setting, the BARE field
|
||||
name as the key, the value in the SAME textual form the env var
|
||||
would carry — so "anything in .env" is editable for all 75
|
||||
settings with ZERO per-field DDL (no typed columns, no wide row),
|
||||
and coercion/validation stays in exactly one place (the pydantic
|
||||
validators, phase 168 A5). A DELETE of the row is the reset
|
||||
("use the env value again") — never a NULL/empty value, so the
|
||||
resolver never has to distinguish "no override" from "override to
|
||||
empty".
|
||||
|
||||
No index beyond the PK: the whole table is read by primary key
|
||||
order in ONE ``SELECT`` per settings resolution (the
|
||||
``ui_settings`` "display fact, never a query surface" note, 0027).
|
||||
"""
|
||||
|
||||
__tablename__ = "app_settings"
|
||||
|
||||
#: The BARE ``Settings`` field name (``relevance_threshold``) — never
|
||||
#: the ``BOR_…`` form: the prefix is presentation
|
||||
#: (``"BOR_" + key.upper()``), so a stored prefix could double up.
|
||||
key: Mapped[str] = mapped_column(String(100), primary_key=True)
|
||||
#: The value in the SAME textual form the env var would carry
|
||||
#: (``"0.55"``, ``"true"``, ``"md,txt"``, a JSON array for the one
|
||||
#: ``list[str]`` field) — coercion and validation belong to
|
||||
#: pydantic, in exactly one place (phase 168 D1/A5). NOT NULL:
|
||||
#: "clear the override" is a DELETE, never an empty value.
|
||||
value: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
#: When the admin last saved it (the ``folder_summaries`` /
|
||||
#: ``kb_overview.updated_at`` house style: server-side now()).
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
@@ -1990,3 +1990,93 @@ class UiSettingsOut(BaseModel):
|
||||
accent_bg: str
|
||||
accent_ink: str
|
||||
accent_line: str
|
||||
|
||||
|
||||
class AdminSettingItem(BaseModel):
|
||||
"""One row of the admin settings catalogue
|
||||
(``GET``/``PUT``/``DELETE /api/admin/settings[/{key}]``, phase 168,
|
||||
task 04).
|
||||
|
||||
One entry per ``Settings`` field (75 today — the catalogue is
|
||||
completeness-pinned, ``tests/unit/test_settings_catalog.py``), in
|
||||
``GROUPS`` order: the field's metadata (``group`` + ``label`` + the
|
||||
three capability flags), where its EFFECTIVE value comes from
|
||||
(``source``: ``db`` = an overridable row is in effect, ``env`` = the
|
||||
``BOR_*`` var is in the process environment, ``default`` = the code
|
||||
default), and the value itself — ``value`` in the SAME env-native
|
||||
string form the env var would carry (bool → ``"true"``/``"false"``,
|
||||
the one ``list[str]`` field → a JSON array, everything else
|
||||
``str()``; the stringification lives in the router, D1), ``None``
|
||||
while masked. LOCKED A4: a ``secret`` field is write-only —
|
||||
``masked=True``, ``value=None``, ``db_value=None`` and
|
||||
``has_value`` (is a non-empty value in effect at all) are all the
|
||||
read side ever reports; the plaintext appears in no response.
|
||||
"""
|
||||
|
||||
key: str
|
||||
env_var: str
|
||||
group: str
|
||||
label: str
|
||||
type: Literal["int", "float", "bool", "str", "json"]
|
||||
secret: bool
|
||||
restart_required: bool
|
||||
overridable: bool
|
||||
source: Literal["db", "env", "default"]
|
||||
#: The effective value in env-native string form (``None`` masked).
|
||||
value: str | None
|
||||
#: Locked A4: ``True`` for a write-only field — ``value`` is ``None``
|
||||
#: while masked and the plaintext is in no response.
|
||||
masked: bool
|
||||
#: For a masked field: is a non-empty value in effect at all.
|
||||
has_value: bool
|
||||
#: An (allowed) row exists in ``app_settings`` for this key.
|
||||
has_override: bool
|
||||
#: The stored string while it is a stored non-secret override; ``None``
|
||||
#: otherwise (a masked or non-overridable row never exposes its text).
|
||||
db_value: str | None
|
||||
#: The code default, stringified (``""`` for a masked field).
|
||||
default: str
|
||||
|
||||
|
||||
class AdminSettingsTotals(BaseModel):
|
||||
"""The catalogue's footer counts (phase 168, task 04): how many
|
||||
settings there are and how many an ``app_settings`` row currently
|
||||
overrides (in effect — ``source == "db"``)."""
|
||||
|
||||
count: int
|
||||
overridden: int
|
||||
|
||||
|
||||
class AdminSettingsOut(BaseModel):
|
||||
"""``GET /api/admin/settings`` response (phase 168, task 04): the
|
||||
whole catalogue — one :class:`AdminSettingItem` per ``Settings``
|
||||
field, in ``GROUPS`` order (the stable order the E2E pins
|
||||
against) — plus the footer counts and the ``BOR_SETTINGS_DB``
|
||||
state: while ``overlay_enabled`` is ``false`` the panel shows a
|
||||
banner instead of pretending a save took effect (the kill switch
|
||||
disables the mechanism, not the panel — D6)."""
|
||||
|
||||
items: list[AdminSettingItem]
|
||||
totals: AdminSettingsTotals
|
||||
overlay_enabled: bool
|
||||
|
||||
|
||||
class AdminSettingIn(BaseModel):
|
||||
"""``PUT /api/admin/settings/{key}`` body (phase 168, task 04):
|
||||
always a string — the value in the env-native string form (D1
|
||||
stores strings; coercion and validation belong to pydantic's
|
||||
``Settings`` validators, in exactly one place, A5). An empty
|
||||
string for a non-secret field is a legitimate value (e.g. clearing
|
||||
``git_sources``); an empty string for a SECRET field is a 422 from
|
||||
the router (an empty API key / session secret is a self-lockout —
|
||||
``DELETE`` is the explicit clear)."""
|
||||
|
||||
value: str
|
||||
|
||||
|
||||
class AdminSettingsCleared(BaseModel):
|
||||
"""``DELETE /api/admin/settings`` response (phase 168, task 04):
|
||||
how many ``app_settings`` rows the "Clear all overrides" action
|
||||
deleted."""
|
||||
|
||||
cleared: int
|
||||
|
||||
@@ -226,6 +226,13 @@ export async function initSharedHeader() {
|
||||
// no-op. A token user (role "user") never sees it.
|
||||
const navTheme = document.querySelector("#nav-theme");
|
||||
if (navTheme) navTheme.hidden = !admin;
|
||||
// Phase 168 (task 05): the Settings nav link (the shell's eighth view
|
||||
// — the phase-34 one-bar contract ships it on every page's nav) —
|
||||
// admin-only, the same ship-hidden / reveal-for-admin contract as
|
||||
// the Theme link above. Null-safe: a page without the link is a
|
||||
// no-op. A token user (role "user") never sees it.
|
||||
const navSettings = document.querySelector("#nav-settings");
|
||||
if (navSettings) navSettings.hidden = !admin;
|
||||
// Phase 34: the steering panel (phase 15) is module-owned. The
|
||||
// navbar #steering-toggle was removed at owner request (2026-08-28)
|
||||
// — the panel ships hidden and is only kept fresh. Admin: refresh
|
||||
|
||||
@@ -82,6 +82,7 @@ const VIEW = {
|
||||
"/history.html": "history", // phase 76 task 03: the History view (saved chats)
|
||||
"/tokens.html": "tokens", // phase 79 task 06: the Tokens view (access tokens)
|
||||
"/theme.html": "theme", // phase 91 task 04: the Theme view (admin palette + branding)
|
||||
"/settings.html": "settings", // phase 168 task 05: the Settings view (admin settings panel)
|
||||
};
|
||||
|
||||
/* The nav-link href the router stamps active for each view (the
|
||||
@@ -94,6 +95,7 @@ const VIEW_PATH = {
|
||||
history: "/history.html",
|
||||
tokens: "/tokens.html",
|
||||
theme: "/theme.html", // phase 91 task 04: the Theme view (admin-only)
|
||||
settings: "/settings.html", // phase 168 task 05: the Settings view (admin-only)
|
||||
};
|
||||
|
||||
/* The lazy view modules — ONLY the non-chat views (chat needs no
|
||||
@@ -108,6 +110,7 @@ const VIEW_MODULES = {
|
||||
history: () => import("./history.js"), // phase 76 task 03
|
||||
tokens: () => import("./tokens.js"), // phase 79 task 06
|
||||
theme: () => import("./theme.js"), // phase 91 task 04 (wiring) + task 05 (editor)
|
||||
settings: () => import("./settings.js"), // phase 168 task 05 (the settings panel)
|
||||
};
|
||||
|
||||
/* Per-view document.head values, carried over from the old pages'
|
||||
@@ -122,6 +125,7 @@ const TITLES = {
|
||||
history: "Saved chats · Brain of Reese", // old history.html <title>
|
||||
tokens: "Access tokens · Brain of Reese",
|
||||
theme: "Theme · Brain of Reese", // phase 91 task 04: no old page — the view is new
|
||||
settings: "Settings · Brain of Reese", // phase 168 task 05: no old page — the view is new
|
||||
};
|
||||
const DESCRIPTIONS = {
|
||||
chat:
|
||||
@@ -135,6 +139,7 @@ const DESCRIPTIONS = {
|
||||
"Saved chats — every conversation is saved automatically, one click back.", // old history.html meta
|
||||
tokens: "Generate and revoke the API tokens that let people use the app.",
|
||||
theme: "Set the palette and branding — the theme is baked into every served page, live on the first paint.",
|
||||
settings: "Edit any application setting — a saved value wins over the environment, live on save; reset hands control back.",
|
||||
};
|
||||
|
||||
/* The brand-resolved display name (phase 39 — brand.js is the single
|
||||
|
||||
@@ -0,0 +1,642 @@
|
||||
/* Brain of Reese — Settings view module (phase 168, task 05): the
|
||||
* admin settings panel — "edit anything in .env and save it to the
|
||||
* database. The database value should override the env var. The
|
||||
* change should take effect on clicking 'save'" (TODO.md L3).
|
||||
*
|
||||
* The phase-76 shell-view-module contract (the theme.js / tokens.js
|
||||
* shape): the router (assets/router.js) lazy-imports this module on
|
||||
* FIRST show of #view-settings and calls mount(root) ONCE (mount-once,
|
||||
* hide-forever — root is the view's <section id="view-settings">,
|
||||
* every DOM lookup scoped to it). The initSharedHeader() call never
|
||||
* happens here: in the shell the shared header boots exactly once,
|
||||
* via the chat module (app.js) at shell boot.
|
||||
*
|
||||
* What the panel does:
|
||||
*
|
||||
* • gate — fetchIsAdmin() (the SAME cached /api/whoami promise
|
||||
* header.js exports, zero extra requests): admin hides
|
||||
* #settings-gate and reveals #settings-content; anonymous /
|
||||
* token-user keeps the gate (the #nav-settings link is already
|
||||
* hidden by header.js — the gate is the DIRECT-URL case, the
|
||||
* #tokens-gate pattern). No /api/admin/settings request is ever
|
||||
* made outside the admin branch (the router 403s them — the
|
||||
* phase-79 contract).
|
||||
* • load — GET /api/admin/settings → render the 75-row catalogue
|
||||
* in the SERVER's order (GROUPS order — the stable order the
|
||||
* E2E pins against): one <tr> per setting (the group band rows
|
||||
* are created per group) + the ship-rebuilt no-match row, all
|
||||
* through createElement + textContent (this file never builds
|
||||
* HTML — the XSS-safe-by-construction house rule). A failed
|
||||
* fetch keeps the last rendered catalogue (never a blanked panel)
|
||||
* and shows the house error line.
|
||||
* • rows — the Setting cell (the label in a .cell-clip ellipsis
|
||||
* — the phase-156 contract with the full text in the control's
|
||||
* aria-label — + the BOR_* name in <code class="settings-env">
|
||||
* + the literal-text `restart required` .settings-flag for the
|
||||
* boot-wired fields (text, never color alone, B5, with the
|
||||
* title/aria-describedby copy explaining when it applies)),
|
||||
* the Value cell (the control by the server's `type`: int/float
|
||||
* → <input type=number> (step="any" for float), bool → a
|
||||
* true/false <select>, json → a spellcheck=false <textarea>
|
||||
* (the one list[str] field, K2), str → <input type=text> (a
|
||||
* <textarea rows=2> for the CSV/path fields whose value exceeds
|
||||
* 40 chars), secret → <input type=password
|
||||
* autocomplete=new-password placeholder="•••••• (stored)"> with
|
||||
* an aria-label naming the env var — an EMPTY secret input
|
||||
* means "leave unchanged" (never an accidental wipe); every
|
||||
* control is id=set-<key> with a <label for>), the Source cell
|
||||
* (the literal-text DB / env / default badge — the color is
|
||||
* decoration, B5), and the Actions cell (the per-row Reset —
|
||||
* disabled with a title when there is nothing to reset or the
|
||||
* field is read-only, LOCKED A3).
|
||||
* • dirty tracking — module-state Map<key, value>: an input/
|
||||
* change marks the row (.is-dirty + the "N unsaved changes"
|
||||
* hint) and enables #settings-save; a row that returns to its
|
||||
* served value un-marks.
|
||||
* • Save — the §7.4 never-stale lifecycle (disabled + aria-busy +
|
||||
* "Saving…", re-enabled on success AND failure — a click can
|
||||
* never leave the button stuck): for each dirty row in DOM
|
||||
* order PUT /api/admin/settings/<key> with { value }; STOP AT
|
||||
* THE FIRST FAILURE (K4) — its server detail lands in
|
||||
* #settings-error (role=alert), the catalogue re-GETs so the
|
||||
* badges show the truth, the remaining rows' edits are
|
||||
* re-applied to the re-rendered controls (they stay in the form
|
||||
* for correction), and focus goes to the offending input. All
|
||||
* good → one #settings-status line `Saved N settings.` (then
|
||||
* role=status) — a saved restart-required row appends "—
|
||||
* applies on the next restart" (the owner is never told
|
||||
* "saved" and then sees nothing change) — + a re-GET so every
|
||||
* badge/value is the settled server state.
|
||||
* • Reset — per row: DELETE /api/admin/settings/<key> → the
|
||||
* status line `BOR_X is back to the environment value.` + a
|
||||
* re-GET.
|
||||
* • Clear all overrides — the house two-step inline confirm
|
||||
* (the history-confirm-* classes, focus moves to the confirm
|
||||
* button, Escape / No cancels) → DELETE /api/admin/settings →
|
||||
* the status line `Cleared N overrides.` (N from the
|
||||
* response) + a re-GET.
|
||||
* • search — #settings-filter: a case-insensitive substring over
|
||||
* the label + BOR_* name cells, applied on input with NO fetch
|
||||
* (the phase-101 tokens D4 contract) and RE-APPLIED after every
|
||||
* load (the query lives in the input — it survives re-shows,
|
||||
* the phase-77 contract). A group with no visible rows hides
|
||||
* its group row; zero matches with a non-empty query → one
|
||||
* no-match row `No settings match "<query>"` (textContent).
|
||||
* • the kill-switch banner — while overlay_enabled is false
|
||||
* (BOR_SETTINGS_DB=0, D6) the #settings-disabled alert explains
|
||||
* that saving stores the value but it will not take effect
|
||||
* (the mechanism is off, not the panel).
|
||||
*
|
||||
* Every value is rendered with textContent — this file never builds
|
||||
* HTML (the XSS-safe-by-construction house rule; a full-file source
|
||||
* pin enforces it).
|
||||
*/
|
||||
|
||||
import { fetchIsAdmin } from "./header.js";
|
||||
|
||||
export async function mount(root) {
|
||||
/* ---------- view elements (the view's section, scoped to root) ---------- */
|
||||
const gateEl = root.querySelector("#settings-gate");
|
||||
const contentEl = root.querySelector("#settings-content");
|
||||
const disabledBanner = root.querySelector("#settings-disabled");
|
||||
const statusEl = root.querySelector("#settings-status");
|
||||
const errorEl = root.querySelector("#settings-error");
|
||||
const filter = root.querySelector("#settings-filter");
|
||||
const tbody = root.querySelector("#settings-tbody");
|
||||
const saveBtn = root.querySelector("#settings-save");
|
||||
const clearAllBtn = root.querySelector("#settings-clear-all");
|
||||
const dirtyHint = root.querySelector("#settings-dirty");
|
||||
|
||||
const SAVE_LABEL = "Save settings";
|
||||
|
||||
/* Module state (the phase-76 mount-once contract — it survives
|
||||
every switch): the last settled catalogue (the row meta for the
|
||||
save's restart check), the pending edits (key → the value being
|
||||
edited), and the in-flight save flag (the §7.4 guard). */
|
||||
let lastItems = [];
|
||||
const dirty = new Map();
|
||||
let saveInFlight = false;
|
||||
let clearAllEsc = null; // the armed Escape handler (null = disarmed)
|
||||
|
||||
/* The no-match row (rebuilt into the tbody on every render — the
|
||||
<td> text is JS-filled; it is never a data row). */
|
||||
const noMatchRow = document.createElement("tr");
|
||||
noMatchRow.className = "settings-no-match";
|
||||
noMatchRow.hidden = true;
|
||||
const noMatchTd = document.createElement("td");
|
||||
noMatchTd.colSpan = 4;
|
||||
noMatchRow.appendChild(noMatchTd);
|
||||
|
||||
/* ---------- feedback lines (never stale, §7.4) ---------- */
|
||||
|
||||
function announce(message) {
|
||||
if (statusEl) statusEl.textContent = message;
|
||||
}
|
||||
|
||||
function showError(message) {
|
||||
if (!errorEl) return;
|
||||
errorEl.textContent = message; // textContent — the server detail is data
|
||||
errorEl.hidden = false;
|
||||
}
|
||||
|
||||
function clearError() {
|
||||
if (!errorEl) return;
|
||||
errorEl.textContent = "";
|
||||
errorEl.hidden = true;
|
||||
}
|
||||
|
||||
/* FastAPI error bodies: a string detail (the house 422s — the
|
||||
detail names the offending field) or the validation-error array
|
||||
(the first entry's msg). Same extraction as theme.js. */
|
||||
async function apiDetail(r, fallback) {
|
||||
try {
|
||||
const data = await r.json();
|
||||
if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {
|
||||
return String(data.detail[0].msg);
|
||||
}
|
||||
if (typeof data.detail === "string" && data.detail) return data.detail;
|
||||
} catch {
|
||||
/* non-JSON error body */
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
/* ---------- catalogue render (createElement + textContent only) ---------- */
|
||||
|
||||
const dataRows = () =>
|
||||
tbody ? [...tbody.querySelectorAll("tr.settings-row")] : [];
|
||||
|
||||
function makeControl(item) {
|
||||
let control;
|
||||
if (item.secret) {
|
||||
/* LOCKED A4: the secret is write-only — the input starts
|
||||
EMPTY (the plaintext never leaves the server); an empty
|
||||
value means "leave unchanged" (never an accidental wipe). */
|
||||
control = document.createElement("input");
|
||||
control.type = "password";
|
||||
control.setAttribute("autocomplete", "new-password");
|
||||
control.placeholder = "•••••• (stored)";
|
||||
} else if (item.type === "int") {
|
||||
control = document.createElement("input");
|
||||
control.type = "number";
|
||||
control.value = item.value;
|
||||
} else if (item.type === "float") {
|
||||
control = document.createElement("input");
|
||||
control.type = "number";
|
||||
control.step = "any";
|
||||
control.value = item.value;
|
||||
} else if (item.type === "bool") {
|
||||
control = document.createElement("select");
|
||||
for (const opt of ["true", "false"]) {
|
||||
const option = document.createElement("option");
|
||||
option.value = opt;
|
||||
option.textContent = opt;
|
||||
control.appendChild(option);
|
||||
}
|
||||
control.value = item.value;
|
||||
} else if (item.type === "json") {
|
||||
/* The one list[str] field (K2) — edited as a JSON array in a
|
||||
textarea (the server's env-native string form, verbatim). */
|
||||
control = document.createElement("textarea");
|
||||
control.spellcheck = false;
|
||||
control.rows = 2;
|
||||
control.value = item.value;
|
||||
} else if (item.type === "str" && (item.value || "").length > 40) {
|
||||
/* A CSV / path field with a long value — the two-row
|
||||
textarea (no single-line clip for 200-char lists). */
|
||||
control = document.createElement("textarea");
|
||||
control.rows = 2;
|
||||
control.value = item.value;
|
||||
} else {
|
||||
control = document.createElement("input");
|
||||
control.type = "text";
|
||||
control.setAttribute("autocomplete", "off");
|
||||
control.value = item.value;
|
||||
}
|
||||
control.id = "set-" + item.key;
|
||||
control.classList.add("settings-control");
|
||||
/* The full (unclipped) label stays reachable: the .cell-clip
|
||||
ellipsis is visual only (the phase-156 contract), and the
|
||||
secret's aria-label names the env var (the house pattern for
|
||||
table-hosted inputs — the Setting cell's label is the
|
||||
visible <label for>). */
|
||||
control.setAttribute(
|
||||
"aria-label",
|
||||
item.secret ? item.env_var : `${item.label} — ${item.env_var}`,
|
||||
);
|
||||
return control;
|
||||
}
|
||||
|
||||
function makeRow(item) {
|
||||
const tr = document.createElement("tr");
|
||||
tr.className = "settings-row";
|
||||
tr.dataset.key = item.key;
|
||||
|
||||
/* The Setting cell: the label (the .cell-clip ellipsis — the
|
||||
full text is the label's own textContent + the control's
|
||||
aria-label), the BOR_* name (presentation only, D1), and —
|
||||
when boot-wired (LOCKED A4) — the literal-text flag with
|
||||
the when-it-applies copy (title + the shared
|
||||
aria-describedby target). */
|
||||
const nameTd = document.createElement("td");
|
||||
nameTd.className = "settings-name-cell";
|
||||
const label = document.createElement("label");
|
||||
label.htmlFor = "set-" + item.key;
|
||||
const clip = document.createElement("span");
|
||||
clip.className = "cell-clip";
|
||||
clip.textContent = item.label;
|
||||
label.appendChild(clip);
|
||||
const env = document.createElement("code");
|
||||
env.className = "settings-env";
|
||||
env.textContent = item.env_var;
|
||||
nameTd.append(label, env);
|
||||
if (item.restart_required) {
|
||||
const flag = document.createElement("span");
|
||||
flag.className = "settings-flag";
|
||||
flag.textContent = "restart required"; // text, never color alone (B5)
|
||||
flag.title =
|
||||
"Applies on the next restart — the app reads this setting once at boot";
|
||||
flag.setAttribute("aria-describedby", "settings-restart-desc");
|
||||
nameTd.appendChild(flag);
|
||||
}
|
||||
tr.appendChild(nameTd);
|
||||
|
||||
/* The Value cell: the control by type (see makeControl). */
|
||||
const valueTd = document.createElement("td");
|
||||
valueTd.className = "settings-value-cell";
|
||||
const control = makeControl(item);
|
||||
valueTd.appendChild(control);
|
||||
tr.appendChild(valueTd);
|
||||
|
||||
/* The Source cell: the literal-text badge (db → "DB", env,
|
||||
default — the color is decoration, B5). */
|
||||
const srcTd = document.createElement("td");
|
||||
const src = document.createElement("span");
|
||||
src.className =
|
||||
"settings-src " +
|
||||
({ db: "is-db", env: "is-env", default: "is-default" })[item.source];
|
||||
src.textContent = item.source === "db" ? "DB" : item.source;
|
||||
srcTd.appendChild(src);
|
||||
tr.appendChild(srcTd);
|
||||
|
||||
/* The Actions cell: the per-row Reset — disabled with a title
|
||||
when there is NOTHING to reset (no saved override) or the
|
||||
field is READ-ONLY (LOCKED A3: the engine is built from it /
|
||||
the dim is fixed at table creation). */
|
||||
const actTd = document.createElement("td");
|
||||
actTd.className = "settings-actions-cell";
|
||||
const reset = document.createElement("button");
|
||||
reset.type = "button";
|
||||
reset.className = "settings-reset";
|
||||
reset.textContent = "Reset";
|
||||
reset.setAttribute("aria-label", `Reset ${item.env_var}`);
|
||||
if (!item.overridable) {
|
||||
reset.disabled = true;
|
||||
reset.title = `read-only — ${item.env_var} is not settable from the database`;
|
||||
} else if (!item.has_override) {
|
||||
reset.disabled = true;
|
||||
reset.title = "no saved override — the environment value is in effect";
|
||||
}
|
||||
reset.addEventListener("click", () => void resetRow(item.key, item.env_var));
|
||||
actTd.appendChild(reset);
|
||||
tr.appendChild(actTd);
|
||||
|
||||
/* Dirty tracking (input + change — the select/change pair). */
|
||||
control.addEventListener("input", () => markDirty(item, control));
|
||||
control.addEventListener("change", () => markDirty(item, control));
|
||||
return tr;
|
||||
}
|
||||
|
||||
/* GET /api/admin/settings → render the groups in the server's
|
||||
order (re-entrant: the tbody is REPLACED — never a duplicate
|
||||
set — and the no-match row is re-appended). A FAILED load
|
||||
returns before the render (the last catalogue stays) and shows
|
||||
the house error line. */
|
||||
function render(items) {
|
||||
if (!tbody) return;
|
||||
const rows = [];
|
||||
let currentGroup = null;
|
||||
for (const item of items) {
|
||||
if (item.group !== currentGroup) {
|
||||
currentGroup = item.group;
|
||||
const groupRow = document.createElement("tr");
|
||||
groupRow.className = "settings-group-row";
|
||||
const th = document.createElement("th");
|
||||
th.colSpan = 4;
|
||||
th.scope = "colgroup";
|
||||
th.textContent = item.group;
|
||||
groupRow.appendChild(th);
|
||||
rows.push(groupRow);
|
||||
}
|
||||
rows.push(makeRow(item));
|
||||
}
|
||||
rows.push(noMatchRow);
|
||||
tbody.replaceChildren(...rows);
|
||||
}
|
||||
|
||||
async function loadSettings() {
|
||||
let r;
|
||||
try {
|
||||
r = await fetch("/api/admin/settings");
|
||||
} catch {
|
||||
showError("Couldn't load the settings — is the app reachable?");
|
||||
return false;
|
||||
}
|
||||
if (!r.ok) {
|
||||
showError("Couldn't load the settings — try again.");
|
||||
return false;
|
||||
}
|
||||
let data;
|
||||
try {
|
||||
data = await r.json();
|
||||
} catch {
|
||||
showError("Couldn't load the settings — try again.");
|
||||
return false;
|
||||
}
|
||||
clearError();
|
||||
lastItems = data.items;
|
||||
/* The kill-switch banner (D6): the mechanism is off, not the
|
||||
panel — a save stores the value but it will not take effect. */
|
||||
if (disabledBanner) disabledBanner.hidden = data.overlay_enabled;
|
||||
render(data.items);
|
||||
/* A failed batch keeps the remaining edits in the form (K4):
|
||||
re-apply every pending value to the re-rendered control and
|
||||
re-mark the row. */
|
||||
for (const [key, value] of dirty) {
|
||||
const control = root.querySelector("#set-" + key);
|
||||
if (!control) continue;
|
||||
control.value = value;
|
||||
const row = control.closest("tr");
|
||||
if (row) row.classList.add("is-dirty");
|
||||
}
|
||||
applyFilter();
|
||||
updateDirtyUI();
|
||||
return true;
|
||||
}
|
||||
|
||||
/* ---------- dirty tracking (module state) ---------- */
|
||||
|
||||
function markDirty(item, control) {
|
||||
/* The served value: the effective value in env-native string
|
||||
form — for a MASKED field it is "" (an empty input means
|
||||
"leave unchanged", never an accidental wipe). */
|
||||
const served = item.secret ? "" : item.value ?? "";
|
||||
const value = control.value;
|
||||
if (value === served) {
|
||||
dirty.delete(item.key);
|
||||
} else {
|
||||
dirty.set(item.key, value);
|
||||
}
|
||||
updateDirtyUI();
|
||||
}
|
||||
|
||||
function updateDirtyUI() {
|
||||
for (const row of dataRows()) {
|
||||
row.classList.toggle("is-dirty", dirty.has(row.dataset.key));
|
||||
}
|
||||
if (dirtyHint) {
|
||||
dirtyHint.textContent = dirty.size
|
||||
? `${dirty.size} unsaved changes`
|
||||
: "";
|
||||
}
|
||||
if (saveBtn && !saveInFlight) saveBtn.disabled = dirty.size === 0;
|
||||
}
|
||||
|
||||
/* ---------- the client-side search (phase-101 D4 contract) ----------
|
||||
* Case-insensitive substring over the label + BOR_* name cells,
|
||||
* applied on `input` with NO fetch and re-applied after every
|
||||
* load (the query lives in the input — it survives re-shows).
|
||||
* A group with no visible rows hides its group row; zero matches
|
||||
* with a non-empty query → the one no-match row (textContent). */
|
||||
function applyFilter() {
|
||||
const query = (filter ? filter.value : "").trim().toLowerCase();
|
||||
const children = tbody ? [...tbody.children] : [];
|
||||
let visible = 0;
|
||||
let currentGroupRow = null;
|
||||
const groupVisible = new Map();
|
||||
for (const tr of children) {
|
||||
if (tr === noMatchRow) continue; // the state row is never a data row
|
||||
if (tr.classList.contains("settings-group-row")) {
|
||||
currentGroupRow = tr;
|
||||
groupVisible.set(tr, false);
|
||||
continue;
|
||||
}
|
||||
const nameCell = tr.querySelector(".settings-name-cell");
|
||||
const labelEl = nameCell ? nameCell.querySelector("label") : null;
|
||||
const envEl = nameCell
|
||||
? nameCell.querySelector(".settings-env")
|
||||
: null;
|
||||
const haystack =
|
||||
((labelEl ? labelEl.textContent : "") +
|
||||
" " +
|
||||
(envEl ? envEl.textContent : "")).toLowerCase();
|
||||
const show = query === "" || haystack.includes(query);
|
||||
tr.hidden = !show;
|
||||
if (show) visible += 1;
|
||||
if (currentGroupRow) groupVisible.set(currentGroupRow, true);
|
||||
}
|
||||
for (const [groupRow, hasVisible] of groupVisible) {
|
||||
groupRow.hidden = !hasVisible;
|
||||
}
|
||||
const noMatch = query !== "" && visible === 0;
|
||||
noMatchRow.hidden = !noMatch;
|
||||
if (noMatch) {
|
||||
noMatchTd.textContent = `No settings match "${
|
||||
filter ? filter.value.trim() : ""
|
||||
}".`;
|
||||
}
|
||||
}
|
||||
|
||||
/* ---------- Save (dirty rows only — stop at the first failure,
|
||||
K4; the §7.4 never-stale lifecycle) ---------- */
|
||||
async function saveSettings() {
|
||||
if (!dirty.size || saveInFlight) return;
|
||||
saveInFlight = true;
|
||||
if (saveBtn) {
|
||||
saveBtn.disabled = true;
|
||||
saveBtn.setAttribute("aria-busy", "true");
|
||||
saveBtn.textContent = "Saving…";
|
||||
}
|
||||
clearError();
|
||||
const saved = [];
|
||||
let failure = null;
|
||||
for (const row of dataRows()) {
|
||||
const key = row.dataset.key;
|
||||
if (!key || !dirty.has(key)) continue;
|
||||
const value = dirty.get(key);
|
||||
try {
|
||||
const r = await fetch(`/api/admin/settings/${key}`, {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ value }),
|
||||
});
|
||||
if (!r.ok) {
|
||||
failure = { key, r };
|
||||
break; // stop at the first failure (K4)
|
||||
}
|
||||
saved.push(key);
|
||||
dirty.delete(key);
|
||||
} catch {
|
||||
failure = { key, r: null };
|
||||
break; // the app is unreachable — stop
|
||||
}
|
||||
}
|
||||
if (failure) {
|
||||
/* The 422 detail (it names the field) or the house network
|
||||
line; the catalogue re-GETs first (the badges show the
|
||||
truth, the remaining edits are re-applied — K4), and the
|
||||
error is shown AFTER the re-GET (the re-GET's clearError
|
||||
must not wipe it — the previous success line is cleared
|
||||
with it, so the stale "Saved N settings." never lingers,
|
||||
the §7.4 never-stale contract). Focus goes to the
|
||||
offending input. */
|
||||
await loadSettings();
|
||||
if (statusEl) statusEl.textContent = "";
|
||||
showError(
|
||||
failure.r
|
||||
? await apiDetail(failure.r, "Couldn't save the settings — try again.")
|
||||
: "Couldn't save the settings — is the app reachable?",
|
||||
);
|
||||
const input = root.querySelector("#set-" + failure.key);
|
||||
if (input) input.focus();
|
||||
} else {
|
||||
/* One status line (role=status) — a saved restart-required
|
||||
row says so (the owner is never told "saved" and then sees
|
||||
nothing change). */
|
||||
const anyRestart = saved.some(
|
||||
(key) => (lastItems.find((i) => i.key === key) || {}).restart_required,
|
||||
);
|
||||
announce(
|
||||
`Saved ${saved.length} settings.` +
|
||||
(anyRestart ? " — applies on the next restart" : ""),
|
||||
);
|
||||
await loadSettings(); // every badge/value is the settled state
|
||||
}
|
||||
saveInFlight = false;
|
||||
if (saveBtn) {
|
||||
saveBtn.removeAttribute("aria-busy");
|
||||
saveBtn.textContent = SAVE_LABEL;
|
||||
saveBtn.disabled = dirty.size === 0;
|
||||
}
|
||||
}
|
||||
|
||||
/* ---------- the per-row Reset (DELETE the row → env wins) ---------- */
|
||||
async function resetRow(key, envVar) {
|
||||
let r;
|
||||
try {
|
||||
r = await fetch(`/api/admin/settings/${key}`, { method: "DELETE" });
|
||||
} catch {
|
||||
announce(`Couldn't reset ${envVar} — is the app reachable?`);
|
||||
return;
|
||||
}
|
||||
if (!r.ok) {
|
||||
announce(`Couldn't reset ${envVar} — try again.`);
|
||||
return;
|
||||
}
|
||||
announce(`${envVar} is back to the environment value.`);
|
||||
await loadSettings(); // the badge falls back to env/default
|
||||
}
|
||||
|
||||
/* ---------- Clear all overrides (the house two-step inline
|
||||
confirm — the history-confirm-* classes; focus to Yes; Escape
|
||||
/ No cancels) ---------- */
|
||||
function restoreClearAll() {
|
||||
if (clearAllEsc) {
|
||||
document.removeEventListener("keydown", clearAllEsc, true);
|
||||
clearAllEsc = null;
|
||||
}
|
||||
const wrap = root.querySelector(".settings-confirm");
|
||||
if (wrap) wrap.replaceWith(clearAllBtn);
|
||||
if (clearAllBtn) clearAllBtn.focus();
|
||||
}
|
||||
|
||||
async function confirmClearAll(yesBtn, restore) {
|
||||
yesBtn.disabled = true; // no double-fire while the request is in flight
|
||||
let cleared = null;
|
||||
try {
|
||||
const r = await fetch("/api/admin/settings", { method: "DELETE" });
|
||||
if (!r.ok) {
|
||||
announce("Couldn't clear the overrides — try again.");
|
||||
restore();
|
||||
return;
|
||||
}
|
||||
const data = await r.json();
|
||||
cleared = data.cleared;
|
||||
} catch {
|
||||
announce("Couldn't clear the overrides — is the app reachable?");
|
||||
restore();
|
||||
return;
|
||||
}
|
||||
restore();
|
||||
announce(`Cleared ${cleared} overrides.`);
|
||||
await loadSettings(); // every badge falls back to env/default
|
||||
}
|
||||
|
||||
function showClearAllConfirm() {
|
||||
if (!clearAllBtn) return;
|
||||
const wrap = document.createElement("span");
|
||||
wrap.className = "settings-confirm";
|
||||
const label = document.createElement("span");
|
||||
label.className = "history-confirm-text";
|
||||
label.textContent = "Clear all overrides?";
|
||||
const yes = document.createElement("button");
|
||||
yes.type = "button";
|
||||
yes.className = "history-confirm-yes";
|
||||
yes.textContent = "Yes";
|
||||
const no = document.createElement("button");
|
||||
no.type = "button";
|
||||
no.className = "history-confirm-no";
|
||||
no.textContent = "No";
|
||||
yes.addEventListener("click", () => void confirmClearAll(yes, restoreClearAll));
|
||||
no.addEventListener("click", restoreClearAll);
|
||||
wrap.append(label, yes, no);
|
||||
clearAllBtn.replaceWith(wrap);
|
||||
yes.focus(); // the confirm pair takes over the focus
|
||||
/* Escape cancels (the task contract — the capture phase so the
|
||||
page's own Esc handlers never race it). */
|
||||
clearAllEsc = (e) => {
|
||||
if (e.key === "Escape") {
|
||||
e.stopPropagation();
|
||||
restoreClearAll();
|
||||
}
|
||||
};
|
||||
document.addEventListener("keydown", clearAllEsc, true);
|
||||
}
|
||||
|
||||
/* ---------- view boot (phase 168 task 05) ----------
|
||||
* The shared header is NOT booted here — in the shell it runs
|
||||
* exactly once, via the chat module (app.js) at shell boot. The
|
||||
* gate reads fetchIsAdmin() — the SAME cached whoami promise the
|
||||
* header uses (zero extra requests). Anonymous / token-user: the
|
||||
* gate in, the content out — and NO /api/admin/settings request
|
||||
* at all (the router 403s them — the phase-79 contract). */
|
||||
if (!(await fetchIsAdmin())) {
|
||||
if (gateEl) gateEl.hidden = false;
|
||||
if (contentEl) contentEl.hidden = true;
|
||||
return;
|
||||
}
|
||||
if (gateEl) gateEl.hidden = true;
|
||||
if (contentEl) contentEl.hidden = false;
|
||||
|
||||
/* Bindings — armed in the ADMIN branch only (like the refresh
|
||||
listener): Save (the §7.4 lifecycle), Clear all (the two-step
|
||||
confirm), the client-side search (NO fetch). */
|
||||
if (saveBtn) saveBtn.addEventListener("click", () => void saveSettings());
|
||||
if (clearAllBtn) clearAllBtn.addEventListener("click", showClearAllConfirm);
|
||||
if (filter) filter.addEventListener("input", () => applyFilter());
|
||||
|
||||
/* Phase 77 (the re-show refresh contract): a user-initiated
|
||||
re-show of this already-mounted view makes the router dispatch
|
||||
bor:view-refresh on the section — re-run the load then (the
|
||||
tab always shows the settled server state when re-shown; the
|
||||
search query survives — it lives in the input). Armed ONLY
|
||||
here, after the whoami gate passed. `started` flips true once
|
||||
the first loadSettings() is made, so the listener can only
|
||||
ever re-run a load the mount already did. */
|
||||
let started = false;
|
||||
root.addEventListener("bor:view-refresh", () => {
|
||||
if (started) void loadSettings();
|
||||
});
|
||||
started = true;
|
||||
void loadSettings(); // the first load (the catalogue)
|
||||
}
|
||||
@@ -4646,6 +4646,232 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
|
||||
padding: 0.45rem 0.8rem;
|
||||
}
|
||||
|
||||
/* ---------- Settings view (phase 168) ----------
|
||||
The admin settings panel: ONE .table-wrap card (the phase-156
|
||||
bounded scroll region — max-height: min(70vh, 64rem) — and the
|
||||
sticky thead via the .table-wrap rule) holding the 75-row
|
||||
catalogue (grouped, type-correct controls, literal-text source
|
||||
badges — B5: text + color, never color alone), the client-side
|
||||
search row, and the Save / Clear-all action row. Every color is
|
||||
an EXISTING AA token (B3: no new palette variable, no new :root
|
||||
token — the phase-156 rule): the badges are TEXT and the color
|
||||
is decoration. Nothing animates, so prefers-reduced-motion needs
|
||||
no new rule. */
|
||||
.settings-shell {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 1.25rem;
|
||||
flex: 1;
|
||||
}
|
||||
/* The catalogue: FULL-WIDTH (AGENTS.md rule 5 — the .git-sources-
|
||||
table language: no skinny list), one cell per column, the
|
||||
.table-wrap card + its bounded scroll + the sticky thead carry
|
||||
the phase-156 contract. */
|
||||
.settings-table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
min-width: 560px;
|
||||
font-size: 0.93rem;
|
||||
}
|
||||
.settings-table th, .settings-table td {
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
padding: 0.7rem 1rem;
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.settings-table td { white-space: nowrap; }
|
||||
/* The group band: the --surface row, ink text (15.3:1, AA). */
|
||||
.settings-group-row th {
|
||||
background: var(--surface);
|
||||
color: var(--ink);
|
||||
font-size: 0.82rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
.settings-table tbody tr.settings-row:hover td { background: var(--bg); }
|
||||
/* The Setting cell: the label (bold) over the BOR_* name (mono,
|
||||
ink-soft 8.2:1 on --surface, AA). */
|
||||
.settings-name-cell label { margin: 0; font-weight: 600; min-width: 0; }
|
||||
.settings-env {
|
||||
font-family: var(--mono);
|
||||
font-size: 0.78rem;
|
||||
color: var(--ink-soft);
|
||||
}
|
||||
/* The restart-required flag (LOCKED A4, boot-wired): the accent
|
||||
STATE family (--accent-ink on --accent-bg 9.5:1, AA) — text +
|
||||
color, never color alone (B5); the title / aria-describedby
|
||||
carry the when-it-applies copy. */
|
||||
.settings-flag {
|
||||
display: inline-block;
|
||||
margin-top: 0.25rem;
|
||||
padding: 0.06rem 0.5rem;
|
||||
border: 1px solid var(--accent-line);
|
||||
border-radius: 999px;
|
||||
font-size: 0.72rem;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.04em;
|
||||
text-transform: uppercase;
|
||||
background: var(--accent-bg);
|
||||
color: var(--accent-ink);
|
||||
}
|
||||
/* The Value cell: the type-correct control (number / select /
|
||||
textarea / text / password) — the .token-label input language
|
||||
(--line hairline, --surface fill, ink text 15.3:1, AA; the
|
||||
values are data — mono) + the ≥44px target + the global
|
||||
:focus-visible ring. */
|
||||
.settings-control {
|
||||
width: 100%;
|
||||
min-width: 0;
|
||||
min-height: 44px;
|
||||
padding: 0.35rem 0.7rem;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--surface);
|
||||
color: var(--ink);
|
||||
font-family: var(--mono);
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
.settings-control:disabled { opacity: 0.6; }
|
||||
/* The Source cell: the literal-text badge — the three is-*
|
||||
variants are SOLID existing tokens (B3): DB is brand-ink on
|
||||
brand-soft (9.0:1, AA), env is ink-soft on surface (8.2:1, AA;
|
||||
the --line border keeps it visible on the card), default is
|
||||
ink-soft on line (7.1:1, AA). The badge is TEXT — the color
|
||||
is decoration (B5). */
|
||||
.settings-src {
|
||||
display: inline-block;
|
||||
padding: 0.12rem 0.55rem;
|
||||
border-radius: 999px;
|
||||
font-family: var(--mono);
|
||||
font-size: 0.72rem;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.04em;
|
||||
text-transform: uppercase;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.settings-src.is-db { background: var(--brand-soft); color: var(--brand-ink); }
|
||||
.settings-src.is-env { background: var(--surface); color: var(--ink-soft); border: 1px solid var(--line); }
|
||||
.settings-src.is-default { background: var(--line); color: var(--ink-soft); }
|
||||
/* A dirty row: the --brand left marker on the row's first cell
|
||||
(an inset box-shadow — no layout shift) so "unsaved" survives
|
||||
grayscale + the N-unsaved-changes hint (text, B5). */
|
||||
.settings-row.is-dirty > td:first-child { box-shadow: inset 3px 0 0 var(--brand); }
|
||||
/* The per-row Reset: the .token-revoke language (≥44px, --line
|
||||
hairline, transparent fill, ink-soft, the err hover pair — it
|
||||
hands control back to the environment) + the dimmed :disabled
|
||||
(nothing to reset / read-only). */
|
||||
.settings-reset {
|
||||
min-height: 44px;
|
||||
padding: 0.35rem 0.7rem;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
background: transparent;
|
||||
color: var(--ink-soft);
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
font-size: 0.82rem;
|
||||
white-space: nowrap;
|
||||
cursor: pointer;
|
||||
}
|
||||
.settings-reset:hover:not(:disabled) { background: var(--err-bg); color: var(--err-ink); border-color: var(--err-line); }
|
||||
.settings-reset:disabled { opacity: 0.5; cursor: default; }
|
||||
/* The search row: the .token-create flex (wraps below 640px via
|
||||
the media query below) + the #token-label input language. */
|
||||
.settings-search {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
}
|
||||
.settings-search label { white-space: nowrap; }
|
||||
#settings-filter {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
min-height: 44px;
|
||||
padding: 0.35rem 0.7rem;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--surface);
|
||||
color: var(--ink);
|
||||
font: inherit;
|
||||
font-size: 0.93rem;
|
||||
}
|
||||
#settings-filter:focus-visible { outline: 3px solid var(--brand); outline-offset: 2px; }
|
||||
/* The action row: Save (the .token-generate brand pill — the
|
||||
§7.4 in-flight state is the dimmed :disabled + aria-busy) and
|
||||
Clear all (the .token-revoke ghost — the destructive err hover
|
||||
pair; the two-step confirm reuses the .history-confirm-* rules
|
||||
unchanged). */
|
||||
.settings-actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
#settings-save {
|
||||
min-height: 44px;
|
||||
padding: 0.4rem 1.1rem;
|
||||
border: 0;
|
||||
border-radius: 999px;
|
||||
background: var(--brand);
|
||||
color: var(--bg);
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
font-size: 0.9rem;
|
||||
white-space: nowrap;
|
||||
cursor: pointer;
|
||||
}
|
||||
#settings-save:hover:not(:disabled) { background: var(--brand-hover); color: var(--bg); }
|
||||
#settings-save:disabled { opacity: 0.6; cursor: wait; }
|
||||
#settings-clear-all {
|
||||
min-height: 44px;
|
||||
padding: 0.35rem 0.7rem;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
background: transparent;
|
||||
color: var(--ink-soft);
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
font-size: 0.82rem;
|
||||
white-space: nowrap;
|
||||
cursor: pointer;
|
||||
}
|
||||
#settings-clear-all:hover { background: var(--err-bg); color: var(--err-ink); border-color: var(--err-line); }
|
||||
/* The "N unsaved changes" hint (textContent only, aria-hidden):
|
||||
the .tokens-status voice (mono, ink-soft). */
|
||||
.settings-dirty {
|
||||
font-family: var(--mono);
|
||||
font-size: 0.8rem;
|
||||
color: var(--brand-ink);
|
||||
}
|
||||
/* The ONE status region (the .tokens-status mirror — the
|
||||
min-height holds the layout so a line never reflows the table).
|
||||
The error line (the .theme-error mirror). */
|
||||
.settings-status {
|
||||
display: block;
|
||||
min-height: 1.2em;
|
||||
color: var(--ink-soft);
|
||||
font-family: var(--mono);
|
||||
font-size: 0.8rem;
|
||||
padding-block: 0.25rem;
|
||||
}
|
||||
.settings-error {
|
||||
display: block;
|
||||
margin: 0;
|
||||
min-height: 1.2em;
|
||||
font-family: var(--mono);
|
||||
font-size: 0.8rem;
|
||||
padding-block: 0.25rem;
|
||||
color: var(--err-ink);
|
||||
}
|
||||
/* The no-match row (the .tokens-empty-row language). */
|
||||
.settings-no-match td {
|
||||
padding: 0.9rem 1rem;
|
||||
color: var(--ink-soft);
|
||||
font-family: var(--mono);
|
||||
font-size: 0.85rem;
|
||||
white-space: normal;
|
||||
}
|
||||
|
||||
/* ---------- Shared page (phase 51, task 03) ----------
|
||||
/shared/<token>: the anonymous read-only conversation (owner-locked
|
||||
2026-08-29, TODO.md L6). The shared page reads exactly like the
|
||||
@@ -6160,6 +6386,18 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
|
||||
.theme-actions { flex-direction: column; align-items: stretch; }
|
||||
#theme-save { width: 100%; }
|
||||
.theme-reset { width: 100%; }
|
||||
/* Phase 168 (task 05): the Settings panel squeezes — the search
|
||||
row stacks (the label over the full-width input) and the action
|
||||
row stacks (Save full width, Clear all under it; the ≥44px
|
||||
targets are the global rules above — the stack keeps them on
|
||||
phones). The actions cell wraps so the Reset control fits the
|
||||
phone width (the table's own horizontal scroll covers the
|
||||
columns). */
|
||||
.settings-search { flex-direction: column; align-items: stretch; }
|
||||
.settings-actions { flex-direction: column; align-items: stretch; }
|
||||
#settings-save { width: 100%; }
|
||||
#settings-clear-all { width: 100%; }
|
||||
.settings-actions-cell { white-space: normal; }
|
||||
/* Phase 51: the shared page squeezes like the chat column — the
|
||||
title and the note step down (the empty-state-title family); the
|
||||
shell is full-width here (the 72rem cap never binds on a phone —
|
||||
|
||||
@@ -72,6 +72,13 @@
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/theme.html" class="nav-link" id="nav-theme" hidden>Theme</a>
|
||||
<!-- Phase 168 (task 05): the Settings link is admin-only —
|
||||
hidden by default, header.js reveals it once whoami says
|
||||
admin, exactly like the Theme link above (the phase-34
|
||||
one-bar contract: the SAME nav ships on every page —
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/settings.html" class="nav-link" id="nav-settings" hidden>Settings</a>
|
||||
<!-- Phase 46 (mobile dropdown copy: sign-in — desktop bar copy is
|
||||
outside the nav; see styles.css .sign-in-mobile rules). -->
|
||||
<a href="/login.html?next=/" class="auth-link sign-in-link sign-in-mobile" id="sign-in-link-mobile" hidden>
|
||||
|
||||
@@ -60,6 +60,13 @@
|
||||
view). No mobile dropdown copy is needed: the link lives
|
||||
in the SAME #app-nav element the hamburger opens. -->
|
||||
<a href="/theme.html" class="nav-link" id="nav-theme" hidden>Theme</a>
|
||||
<!-- Phase 168 (task 05): the Settings link is admin-only — hidden
|
||||
by default, header.js reveals it once whoami says admin,
|
||||
exactly like the Theme link above (the phase-34 one-bar
|
||||
contract; the shell's eighth view). No mobile dropdown copy
|
||||
is needed: the link lives in the SAME #app-nav element the
|
||||
hamburger opens. A token user never sees it. -->
|
||||
<a href="/settings.html" class="nav-link" id="nav-settings" hidden>Settings</a>
|
||||
<!-- Phase 46 (mobile dropdown copy: sign-in — desktop bar copy is
|
||||
outside the nav; see styles.css .sign-in-mobile rules). -->
|
||||
<a href="/login.html?next=/" class="auth-link sign-in-link sign-in-mobile" id="sign-in-link-mobile" hidden>
|
||||
@@ -1478,6 +1485,133 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- Phase 168 (task 05): the Settings view — the shell's eighth
|
||||
view (the #view-tokens skeleton, verbatim): the 72rem
|
||||
.container, the anonymous sign-in gate (the #tokens-gate
|
||||
shape), and the ship-hidden #settings-content (settings.js
|
||||
reveals it for admin only). The catalogue's <tbody> elements
|
||||
are created by settings.js (one per group, each opening with
|
||||
a <tr class="settings-group-row">); the .table-wrap +
|
||||
sticky-thead rules in styles.css carry the bounded-scroll +
|
||||
sticky-header contract (the phase-156 catalogue language).
|
||||
No <script> tag: the router lazy-imports settings.js
|
||||
(rule 6 — everything local). -->
|
||||
<section class="view" id="view-settings" hidden inert aria-label="Settings" tabindex="-1">
|
||||
<div class="container settings-shell">
|
||||
<div class="page-head">
|
||||
<h1>Settings</h1>
|
||||
<p class="page-sub">
|
||||
Values saved here are stored in the database and win over
|
||||
the server's environment variables; clearing one hands
|
||||
control back to the environment. <code>.env</code> is never
|
||||
written — the environment stays the base, the database row
|
||||
wins.
|
||||
</p>
|
||||
</div>
|
||||
<!-- Phase 168 (task 05): anonymous sign-in gate — the EXACT
|
||||
#tokens-gate shape (phase 79) and the same .sources-gate
|
||||
visual language: every setting is what the login locks.
|
||||
Visible for anonymous, hidden for the admin (settings.js).
|
||||
The gate's Sign in returns to the Settings view (the
|
||||
header's ?next= convention; the static href is the no-JS
|
||||
fallback). -->
|
||||
<section class="sources-gate" id="settings-gate" aria-labelledby="settings-gate-title" hidden>
|
||||
<div class="sources-gate-glyph" aria-hidden="true">
|
||||
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"><rect x="4" y="10" width="16" height="10" rx="2"/><path d="M8 10V7a4 4 0 0 1 8 0v3"/><circle cx="12" cy="14.5" r="1.4" fill="currentColor" stroke="none"/><path d="M12 16v2"/></svg>
|
||||
</div>
|
||||
<h2 id="settings-gate-title">Sign in to change settings</h2>
|
||||
<p class="sources-gate-sub">
|
||||
Editing application settings is admin-only. Chat — and any
|
||||
document an answer cites — stays open to everyone.
|
||||
</p>
|
||||
<a class="sources-gate-link" href="/login.html?next=/settings.html">Sign in</a>
|
||||
</section>
|
||||
|
||||
<!-- Phase 168 (task 05): the panel — SHIPS hidden
|
||||
(anonymous-safe; the gate is what anonymous sees).
|
||||
settings.js reveals it once the cached whoami says admin
|
||||
(the #theme-content pattern) and renders the catalogue's
|
||||
groups/rows from GET /api/admin/settings. -->
|
||||
<div id="settings-content" hidden>
|
||||
<!-- Shown when the server's overlay_enabled is false
|
||||
(the BOR_SETTINGS_DB=0 kill switch — D6): the panel
|
||||
stays editable, but a save cannot take effect. -->
|
||||
<div class="kb-banner is-error" id="settings-disabled" role="alert" hidden>
|
||||
DB overrides are switched off on the server
|
||||
(<code>BOR_SETTINGS_DB=0</code>) — saving stores the
|
||||
value but it will not take effect.
|
||||
</div>
|
||||
|
||||
<!-- The ONE status region (the #tokens-status precedent):
|
||||
settings.js sets textContent here — a save's outcome,
|
||||
a reset's outcome, a clear-all's outcome, nothing else
|
||||
(the error line has its own role=alert below). -->
|
||||
<span class="settings-status" id="settings-status" role="status" aria-live="polite"></span>
|
||||
|
||||
<!-- The error line (the #theme-error precedent): the
|
||||
server's 422 detail (it names the offending field)
|
||||
and the house load-failure lines; the rows keep their
|
||||
edits on failure. -->
|
||||
<p class="settings-error" id="settings-error" role="alert" hidden></p>
|
||||
|
||||
<!-- The client-side filter (the phase-101 tokens D4
|
||||
contract): a case-insensitive substring over the label
|
||||
+ BOR_* name cells, applied on input with NO fetch,
|
||||
re-applied after every re-load (the query survives
|
||||
re-shows — the phase-77 contract). -->
|
||||
<div class="settings-search">
|
||||
<label for="settings-filter">Search settings</label>
|
||||
<input
|
||||
id="settings-filter"
|
||||
type="search"
|
||||
autocomplete="off"
|
||||
>
|
||||
</div>
|
||||
|
||||
<!-- The scroll card: the .table-wrap + sticky-th rules in
|
||||
styles.css are already the bounded-scroll + sticky-
|
||||
header contract (the phase-156 catalogue language).
|
||||
The <tbody> elements are created by settings.js (one
|
||||
per group, each opening with a settings-group-row). -->
|
||||
<div class="table-wrap">
|
||||
<table class="settings-table">
|
||||
<caption class="visually-hidden">Application settings — the value in effect and where it comes from</caption>
|
||||
<thead>
|
||||
<tr>
|
||||
<th scope="col">Setting</th>
|
||||
<th scope="col">Value</th>
|
||||
<th scope="col">Source</th>
|
||||
<th scope="col"><span class="visually-hidden">Actions</span></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="settings-tbody"></tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- The action row: Save (dirty rows only — ships disabled
|
||||
until a row is edited; settings.js runs the §7.4
|
||||
never-stale lifecycle), Clear all overrides (the house
|
||||
two-step inline confirm, settings.js), and the
|
||||
"N unsaved changes" hint (textContent only, aria-hidden
|
||||
— the rows' .is-dirty state + the status line carry
|
||||
the information). -->
|
||||
<div class="settings-actions">
|
||||
<button type="button" class="btn primary" id="settings-save" disabled>Save settings</button>
|
||||
<button type="button" class="btn ghost" id="settings-clear-all">Clear all overrides</button>
|
||||
<span class="settings-dirty" id="settings-dirty" aria-hidden="true"></span>
|
||||
</div>
|
||||
|
||||
<!-- The shared description target for every
|
||||
restart-required row's flag (aria-describedby). -->
|
||||
<p class="visually-hidden" id="settings-restart-desc">
|
||||
Applies on the next restart — the app reads this setting
|
||||
once at boot, so the saved value takes effect when the
|
||||
app next starts.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
</main>
|
||||
|
||||
<!-- Phase 79 (task 05): the in-app token gate — a body-level
|
||||
|
||||
@@ -65,6 +65,13 @@
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/theme.html" class="nav-link" id="nav-theme" hidden>Theme</a>
|
||||
<!-- Phase 168 (task 05): the Settings link is admin-only —
|
||||
hidden by default, header.js reveals it once whoami says
|
||||
admin, exactly like the Theme link above (the phase-34
|
||||
one-bar contract: the SAME nav ships on every page —
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/settings.html" class="nav-link" id="nav-settings" hidden>Settings</a>
|
||||
<!-- Phase 46 (mobile dropdown copy: sign-in — desktop bar copy is
|
||||
outside the nav; see styles.css .sign-in-mobile rules). -->
|
||||
<a href="/login.html?next=/" class="auth-link sign-in-link sign-in-mobile" id="sign-in-link-mobile" hidden>
|
||||
|
||||
@@ -66,6 +66,13 @@
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/theme.html" class="nav-link" id="nav-theme" hidden>Theme</a>
|
||||
<!-- Phase 168 (task 05): the Settings link is admin-only —
|
||||
hidden by default, header.js reveals it once whoami says
|
||||
admin, exactly like the Theme link above (the phase-34
|
||||
one-bar contract: the SAME nav ships on every page —
|
||||
test_nav_consistency pins the inventory parity).
|
||||
Null-safe: header.js is a no-op on a page without it. -->
|
||||
<a href="/settings.html" class="nav-link" id="nav-settings" hidden>Settings</a>
|
||||
<!-- Phase 46 (mobile dropdown copy: sign-in — desktop bar copy is
|
||||
outside the nav; see styles.css .sign-in-mobile rules). -->
|
||||
<a href="/login.html?next=/" class="auth-link sign-in-link sign-in-mobile" id="sign-in-link-mobile" hidden>
|
||||
|
||||
@@ -315,6 +315,7 @@ def run(args: argparse.Namespace) -> tuple[int, list[Row], list[str]]:
|
||||
rows.append(measure(client, "tokens", "/api/tokens"))
|
||||
rows.append(measure(client, "config", "/api/config"))
|
||||
rows.append(measure(client, "whoami", "/api/whoami"))
|
||||
rows.append(measure(client, "admin settings (catalogue)", "/api/admin/settings"))
|
||||
|
||||
corpus.append(f"sources={len(depth1.get('sources', []))}")
|
||||
totals = depth1.get("totals") or {}
|
||||
|
||||
@@ -35,6 +35,15 @@ SESSION_SECRET = "test-session-secret-0123456789abcdef0123456789abcdef"
|
||||
os.environ.setdefault("BOR_ADMIN_PASSWORD", ADMIN_PASSWORD)
|
||||
os.environ.setdefault("BOR_SESSION_SECRET", SESSION_SECRET)
|
||||
|
||||
# Phase 168 (task 02): the DB-over-env overlay
|
||||
# (app/core/settings_overlay) is pinned OFF for every in-process suite
|
||||
# — the shared test DB must never retune the app under test (the same
|
||||
# poison-guard reasoning as the phase-140 BOR_PDF_PICTORIAL pin in
|
||||
# tests/e2e/conftest.py): a row left behind by one suite would
|
||||
# silently change every later suite's settings. Tests that WANT the
|
||||
# overlay set BOR_SETTINGS_DB=1 + get_settings.cache_clear() themselves.
|
||||
os.environ.setdefault("BOR_SETTINGS_DB", "0")
|
||||
|
||||
# Phase 61 (defect fix): the app under test must see the code DEFAULTS,
|
||||
# not an operator's local (gitignored) ``.env`` — ``Settings`` loads
|
||||
# ``env_file=".env"`` from the repo root, and a machine-specific corpus
|
||||
|
||||
@@ -52,6 +52,17 @@ SESSION_SECRET = "e2e-session-secret-0123456789abcdef0123456789abcdef"
|
||||
# (``env["BOR_PDF_PICTORIAL"] = "1"`` in its module app).
|
||||
os.environ["BOR_PDF_PICTORIAL"] = "0"
|
||||
|
||||
# Phase 168 (task 02): the DB-over-env overlay
|
||||
# (app/core/settings_overlay) is pinned OFF for every E2E app — the
|
||||
# cross-suite poison guard (the phase-140 BOR_PDF_PICTORIAL pin, same
|
||||
# reasoning): a row saved by one suite (tests/e2e/
|
||||
# test_admin_settings_panel.py opts in with BOR_SETTINGS_DB=1 in its
|
||||
# own module app fixture — the test_pictorial_pdf.py pattern) would
|
||||
# otherwise silently retune every later suite's app subprocess. The
|
||||
# line rides into every app's env through the house ``env =
|
||||
# dict(os.environ)`` pattern.
|
||||
os.environ["BOR_SETTINGS_DB"] = "0"
|
||||
|
||||
|
||||
def _wait_http(url: str, timeout: float = 40.0) -> None:
|
||||
deadline = time.monotonic() + timeout
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,518 @@
|
||||
"""Integration: the admin settings panel's read/write surface (phase 168,
|
||||
task 04) — ``/api/admin/settings`` GET / PUT ``/{key}`` / DELETE
|
||||
``/{key}`` / DELETE all, driven through the real app (TestClient keeps
|
||||
the cookie jar — the house ``test_ui_settings_api`` /
|
||||
``test_tokens_api`` pattern).
|
||||
|
||||
The pins (00_phase, Testing & Quality):
|
||||
|
||||
* the auth matrix — anonymous AND a signed-in token USER get 403 on ALL
|
||||
four routes (the router-wide ``require_admin`` closes the surface on
|
||||
every method), the admin 200s on all four;
|
||||
* the shape — ``items`` length == ``len(Settings.model_fields)`` (75),
|
||||
groups in ``GROUPS`` order (the stable order the E2E pins against),
|
||||
``totals.count`` / ``overridden`` correct, ``overlay_enabled`` true;
|
||||
on an empty table a field whose ``BOR_*`` var the test env sets
|
||||
reports ``source="env"``, an untouched one ``source="default"`` and
|
||||
none reports ``"db"``;
|
||||
* write → live — a saved ``relevance_threshold`` is in
|
||||
``get_settings()`` with NO restart (the cache clear + task-03 hooks
|
||||
are the whole job), the row is in the table with a non-null
|
||||
``updated_at``, and the DB value reaches the ANONYMOUS
|
||||
``/api/config`` endpoint through the same overlay;
|
||||
* errors — unknown key 404; the two LOCKED A3 keys 422 naming the env
|
||||
var; a bad shape 422 naming the field with NOTHING stored; a
|
||||
cross-field pair that only breaks in combination (the second PUT 422s
|
||||
and the first row is unchanged);
|
||||
* masking (LOCKED A4) — a stored secret reports ``masked``/``has_value``
|
||||
only, the plaintext appears NOWHERE in the raw response text, a blank
|
||||
secret PUT 422s, and a DELETE clears it;
|
||||
* reset — a DELETE ``/{key}`` restores the env value (idempotent);
|
||||
DELETE ``""`` clears everything (``{"cleared": 3}``);
|
||||
* the kill switch (D6) — ``BOR_SETTINGS_DB=0``: ``overlay_enabled``
|
||||
false, a PUT still stores the row, ``get_settings()`` still reports
|
||||
the env value — the switch disables the mechanism, not the panel;
|
||||
* the byte budget — the whole catalogue stays under the phase-167 INFO
|
||||
bar (64 KB), so it never floods the log;
|
||||
* fail-soft boot — a malformed row inserted DIRECTLY into the table
|
||||
(bypassing the API's 422 gate) never bricks: a fresh ``create_app()``
|
||||
runs with it in the table, the row is skipped with a logged reason,
|
||||
the rest of the table still applies, and ``/api/health`` is 200.
|
||||
|
||||
The module's teardown leaves NO row behind (the same discipline
|
||||
``test_admin_theme_tab.py`` applies to ``ui_settings`` — a surviving
|
||||
row would retune every later suite through the overlay).
|
||||
|
||||
Real Postgres (``podman compose up -d db``); no LLM involved.
|
||||
|
||||
Requires: podman compose up -d db
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api import chat as chat_api
|
||||
from app.config import Settings, get_settings, invalidate_settings
|
||||
from app.core import settings_overlay
|
||||
from app.core.settings_catalog import CATALOG, GROUPS
|
||||
from app.core.settings_overlay import register_invalidator
|
||||
from app.main import app as fastapi_app
|
||||
from app.main import create_app
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
API = "/api/admin/settings"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_state(
|
||||
db: Session, monkeypatch: pytest.MonkeyPatch
|
||||
) -> Iterator[None]:
|
||||
"""A clean ``app_settings`` table + the overlay switch ON + a fresh
|
||||
overlay/hook slate (the ``test_admin_settings_live`` idiom: reset
|
||||
the process-wide hook list and re-wire what ``create_app()``
|
||||
wires, so a save in this module runs the chat runtime reset).
|
||||
Teardown deletes the rows + invalidates + clears the cache — NO row
|
||||
may survive this module (a surviving row would retune every later
|
||||
suite through the overlay).
|
||||
"""
|
||||
settings_overlay.reset_state_for_tests()
|
||||
register_invalidator(chat_api.reset_runtime_caches)
|
||||
db.execute(text("DELETE FROM app_settings"))
|
||||
db.commit()
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
invalidate_settings()
|
||||
yield
|
||||
db.execute(text("DELETE FROM app_settings"))
|
||||
db.commit()
|
||||
invalidate_settings()
|
||||
get_settings.cache_clear()
|
||||
|
||||
|
||||
def _admin_client() -> TestClient:
|
||||
"""A fresh client signed in as the admin (the ``_admin_client``
|
||||
pattern from ``test_ui_settings_api``)."""
|
||||
c = TestClient(fastapi_app)
|
||||
r = c.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
|
||||
return c
|
||||
|
||||
|
||||
def _token_user_client(admin: TestClient, anon: TestClient) -> TestClient:
|
||||
"""A signed-in token USER (role ``"user"`` — issue the token the way
|
||||
``test_tokens_api`` does): the only non-admin identity, 403 on this
|
||||
surface on every method (the phase-79 token matrix contract)."""
|
||||
r = admin.post("/api/tokens", json={"label": "alice"})
|
||||
assert r.status_code == 201, r.text
|
||||
token = r.json()["token"]
|
||||
assert anon.post("/api/token-auth", json={"token": token}).status_code == 204
|
||||
return anon
|
||||
|
||||
|
||||
def _item(body: dict, key: str) -> dict:
|
||||
"""One catalogue item by key (the GET body's ``items`` list)."""
|
||||
for item in body["items"]:
|
||||
if item["key"] == key:
|
||||
return item
|
||||
raise AssertionError(f"catalogue item missing: {key}")
|
||||
|
||||
|
||||
def test_auth_matrix_all_four_routes(client: TestClient, db: Session) -> None:
|
||||
"""Anonymous AND token user: 403 ``admin only`` on ALL four routes
|
||||
(the router-wide dependency closes the surface on every method);
|
||||
the admin passes on all four."""
|
||||
admin = _admin_client()
|
||||
token_user = _token_user_client(admin, client)
|
||||
|
||||
for caller in (client, token_user):
|
||||
r = caller.get(API)
|
||||
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
|
||||
r = caller.put(f"{API}/relevance_threshold", json={"value": "0.5"})
|
||||
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
|
||||
r = caller.delete(f"{API}/relevance_threshold")
|
||||
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
|
||||
r = caller.delete(API)
|
||||
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
|
||||
|
||||
r = admin.get(API)
|
||||
assert r.status_code == 200
|
||||
r = admin.put(f"{API}/top_n_docs", json={"value": "3"})
|
||||
assert r.status_code == 200, r.text
|
||||
r = admin.delete(f"{API}/top_n_docs")
|
||||
assert r.status_code == 200, r.text
|
||||
r = admin.delete(API)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {"cleared": 0}
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
||||
|
||||
|
||||
def test_catalogue_shape_and_sources_on_empty_table(admin_client) -> None:
|
||||
"""The whole catalogue, grouped and stable: 75 items (exactly
|
||||
``Settings.model_fields``), groups in ``GROUPS`` order (each group a
|
||||
contiguous block, the block order = ``GROUPS``), ``totals`` correct,
|
||||
``overlay_enabled`` true. On an empty table: the env-var-set field
|
||||
reports ``"env"``, the untouched one ``"default"`` and NOTHING
|
||||
reports ``"db"``."""
|
||||
r = admin_client.get(API)
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
items = body["items"]
|
||||
assert len(items) == len(Settings.model_fields)
|
||||
assert body["totals"] == {
|
||||
"count": len(Settings.model_fields),
|
||||
"overridden": 0,
|
||||
}
|
||||
assert body["overlay_enabled"] is True
|
||||
|
||||
# The stable order: each group a contiguous block, block order GROUPS.
|
||||
groups_seen: list[str] = []
|
||||
for item in items:
|
||||
assert item["group"] in GROUPS
|
||||
if not groups_seen or groups_seen[-1] != item["group"]:
|
||||
groups_seen.append(item["group"])
|
||||
assert groups_seen == [g for g in GROUPS if g in groups_seen]
|
||||
assert len(groups_seen) == len(set(groups_seen))
|
||||
# Every item carries the full contract (the panel generates from it).
|
||||
for item in items:
|
||||
assert set(item) == {
|
||||
"key", "env_var", "group", "label", "type", "secret",
|
||||
"restart_required", "overridable", "source", "value",
|
||||
"masked", "has_value", "has_override", "db_value", "default",
|
||||
}
|
||||
assert item["env_var"] == "BOR_" + item["key"].upper()
|
||||
assert item["source"] in ("db", "env", "default")
|
||||
|
||||
# Empty table: env / default / never db.
|
||||
sources = {item["key"]: item["source"] for item in items}
|
||||
assert "db" not in sources.values()
|
||||
assert sources["input_placeholder"] == "env" # BOR_… set in tests/conftest.py
|
||||
assert sources["docs_work_dir"] == "default" # no BOR_… in the test env
|
||||
assert sources["relevance_threshold"] == "env" # pinned 0.30 in conftest
|
||||
|
||||
|
||||
def test_write_is_live_without_restart(admin_client, db: Session) -> None:
|
||||
"""Write → live (no restart): the saved value is in ``get_settings()``
|
||||
in the SAME process, the row is in the table with a non-null
|
||||
``updated_at``, the item reports ``source="db"`` and
|
||||
``totals.overridden == 1`` — and the DB value reaches the ANONYMOUS
|
||||
``/api/config`` endpoint through the same overlay (the DB-over-env
|
||||
rule is deployment-wide, not panel-local)."""
|
||||
assert get_settings().relevance_threshold == 0.30 # the conftest env value
|
||||
|
||||
r = admin_client.put(f"{API}/relevance_threshold", json={"value": "0.5"})
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["key"] == "relevance_threshold"
|
||||
assert body["source"] == "db"
|
||||
assert body["has_override"] is True
|
||||
assert body["db_value"] == "0.5"
|
||||
assert get_settings().relevance_threshold == 0.5 # NO restart (task 03)
|
||||
|
||||
row = db.execute(
|
||||
text("SELECT value, updated_at FROM app_settings WHERE key = 'relevance_threshold'")
|
||||
).first()
|
||||
assert row is not None
|
||||
assert row[0] == "0.5"
|
||||
assert row[1] is not None # the server-side now()
|
||||
|
||||
r = admin_client.get(API)
|
||||
assert r.json()["totals"]["overridden"] == 1
|
||||
|
||||
# Visible through the public config endpoint (no auth) — the same
|
||||
# overlay retunes every consumer at once (D2).
|
||||
anon = TestClient(fastapi_app)
|
||||
r = anon.get("/api/config")
|
||||
assert r.status_code == 200
|
||||
assert r.json()["input_placeholder"] == "Ask me anything…"
|
||||
r = admin_client.put(f"{API}/input_placeholder", json={"value": "Type your question…"})
|
||||
assert r.status_code == 200, r.text
|
||||
r = anon.get("/api/config")
|
||||
assert r.status_code == 200
|
||||
assert r.json()["input_placeholder"] == "Type your question…"
|
||||
|
||||
|
||||
def test_put_errors_404_and_422(admin_client, db: Session) -> None:
|
||||
"""The write-path guards (LOCKED A5, A3, A4): unknown key 404; the
|
||||
two A3 keys 422 naming the env var; a bad shape 422 naming the FIELD
|
||||
with NOTHING stored; a cross-field pair that only breaks in
|
||||
combination — the second PUT 422s and the first row is unchanged."""
|
||||
r = admin_client.put(f"{API}/bogus_key", json={"value": "1"})
|
||||
assert r.status_code == 404
|
||||
assert r.json() == {"detail": "Unknown setting bogus_key"}
|
||||
|
||||
r = admin_client.put(f"{API}/database_url", json={"value": "postgresql://x"})
|
||||
assert r.status_code == 422
|
||||
assert "BOR_DATABASE_URL is not settable from the database" in r.json()["detail"]
|
||||
r = admin_client.put(f"{API}/embedding_dim", json={"value": "384"})
|
||||
assert r.status_code == 422
|
||||
assert "BOR_EMBEDDING_DIM is not settable from the database" in r.json()["detail"]
|
||||
|
||||
# A bad shape: names the field, stores nothing.
|
||||
r = admin_client.put(f"{API}/relevance_threshold", json={"value": "abc"})
|
||||
assert r.status_code == 422
|
||||
assert "relevance_threshold" in r.json()["detail"]
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
||||
|
||||
# The one list[str] field (K2): a malformed JSON array is a 422
|
||||
# naming the field (the _coerce ValueError branch), nothing stored.
|
||||
r = admin_client.put(f"{API}/suggestions", json={"value": "[not json"})
|
||||
assert r.status_code == 422
|
||||
assert "suggestions" in r.json()["detail"]
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
||||
# A well-formed JSON array stores fine (the JSON round-trip works).
|
||||
r = admin_client.put(
|
||||
f"{API}/suggestions", json={"value": '["How do I deploy gitea?"]'}
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["db_value"] == '["How do I deploy gitea?"]'
|
||||
assert db.execute(
|
||||
text("SELECT value FROM app_settings WHERE key = 'suggestions'")
|
||||
).scalar() == '["How do I deploy gitea?"]'
|
||||
# (Clean up the row — the cross-field pin below counts rows.)
|
||||
assert admin_client.delete(f"{API}/suggestions").status_code == 200
|
||||
|
||||
# The cross-field pair (only breaks in combination): store
|
||||
# relevance_threshold=0.2 (valid alone), then lexical_support_floor=0.9
|
||||
# — above the DB-overridden threshold: the SECOND PUT 422s and the
|
||||
# first row is unchanged.
|
||||
r = admin_client.put(f"{API}/relevance_threshold", json={"value": "0.2"})
|
||||
assert r.status_code == 200, r.text
|
||||
r = admin_client.put(f"{API}/lexical_support_floor", json={"value": "0.9"})
|
||||
assert r.status_code == 422
|
||||
assert "lexical_support_floor" in r.json()["detail"]
|
||||
value = db.execute(
|
||||
text("SELECT value FROM app_settings WHERE key = 'relevance_threshold'")
|
||||
).scalar()
|
||||
assert value == "0.2" # the first row is unchanged
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 1
|
||||
# The failed write did not disturb the live view.
|
||||
assert get_settings().relevance_threshold == 0.2
|
||||
|
||||
|
||||
def test_secrets_are_masked_and_write_only(admin_client, db: Session) -> None:
|
||||
"""LOCKED A4: a stored secret reports ``masked``/``has_value`` only —
|
||||
``value``/``db_value`` are ``None`` and the plaintext appears NOWHERE
|
||||
in the raw response text (asserted on ``response.text``, not the
|
||||
parsed model); a blank secret PUT 422s (a self-lockout — DELETE is
|
||||
the explicit clear); a DELETE clears it."""
|
||||
db.execute(
|
||||
text("INSERT INTO app_settings (key, value) VALUES ('llm_api_key', :v)"),
|
||||
{"v": "sk-secret-12345"},
|
||||
)
|
||||
db.commit()
|
||||
invalidate_settings()
|
||||
|
||||
r = admin_client.get(API)
|
||||
assert r.status_code == 200
|
||||
item = _item(r.json(), "llm_api_key")
|
||||
assert item["masked"] is True
|
||||
assert item["value"] is None
|
||||
assert item["db_value"] is None
|
||||
assert item["has_value"] is True # a non-empty value is in effect
|
||||
assert item["has_override"] is True
|
||||
assert item["source"] == "db"
|
||||
assert item["default"] == "" # masked: the code default is never shown
|
||||
assert "sk-secret-12345" not in r.text # NOWHERE in the raw body
|
||||
# The other write-only fields are masked too (set in the test env).
|
||||
for key in ("admin_password", "session_secret", "database_url"):
|
||||
other = _item(r.json(), key)
|
||||
assert other["masked"] is True and other["value"] is None
|
||||
|
||||
# A blank secret PUT is a 422 (an empty API key / session secret
|
||||
# would self-lockout — DELETE is the explicit clear).
|
||||
r = admin_client.put(f"{API}/llm_api_key", json={"value": " "})
|
||||
assert r.status_code == 422
|
||||
assert "BOR_LLM_API_KEY must not be blank" in r.json()["detail"]
|
||||
assert db.execute(
|
||||
text("SELECT value FROM app_settings WHERE key = 'llm_api_key'")
|
||||
).scalar() == "sk-secret-12345" # unchanged
|
||||
|
||||
# A DELETE clears the row — the effective value falls back to the
|
||||
# env value (whatever this deployment's env/.env provides — the
|
||||
# test only pins that it is NOT the stored secret and that the
|
||||
# item reports the fallback state).
|
||||
r = admin_client.delete(f"{API}/llm_api_key")
|
||||
assert r.status_code == 200
|
||||
assert r.json()["source"] == "default" # no BOR_LLM_API_KEY in the PROCESS env
|
||||
assert r.json()["has_override"] is False
|
||||
assert r.json()["value"] is None
|
||||
assert r.json()["has_value"] == bool(get_settings().llm_api_key)
|
||||
assert get_settings().llm_api_key != "sk-secret-12345" # the override is gone
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
||||
|
||||
|
||||
def test_reset_to_env_and_clear_all(admin_client, db: Session) -> None:
|
||||
"""Reset semantics: DELETE ``/{key}`` deletes the row (idempotent — a
|
||||
double Reset is a success, not a 404) and the env value wins again;
|
||||
DELETE ``""`` clears every row (``{"cleared": 3}``) and
|
||||
``totals.overridden`` is back to 0."""
|
||||
r = admin_client.put(f"{API}/relevance_threshold", json={"value": "0.5"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert get_settings().relevance_threshold == 0.5
|
||||
|
||||
r = admin_client.delete(f"{API}/relevance_threshold")
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["source"] == "env" # BOR_RELEVANCE_THRESHOLD is set (0.30)
|
||||
assert r.json()["has_override"] is False
|
||||
assert get_settings().relevance_threshold == 0.30 # the env value wins again
|
||||
# Idempotent: the double Reset is a success, not a 404.
|
||||
r = admin_client.delete(f"{API}/relevance_threshold")
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
# Three rows, then "Clear all overrides".
|
||||
for key, value in (("relevance_threshold", "0.5"), ("llm_retries", "7"), ("top_n_docs", "3")):
|
||||
r = admin_client.put(f"{API}/{key}", json={"value": value})
|
||||
assert r.status_code == 200, r.text
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 3
|
||||
|
||||
r = admin_client.delete(API)
|
||||
assert r.status_code == 200
|
||||
assert r.json() == {"cleared": 3}
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
||||
assert get_settings().relevance_threshold == 0.30 # env values win again
|
||||
r = admin_client.get(API)
|
||||
assert r.json()["totals"]["overridden"] == 0
|
||||
|
||||
|
||||
def test_kill_switch_disables_the_mechanism_not_the_panel(
|
||||
admin_client, db: Session, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""D6: with ``BOR_SETTINGS_DB=0`` the catalogue reports
|
||||
``overlay_enabled`` false (the panel shows a banner instead of
|
||||
pretending a save took effect), a PUT STILL stores the row, and
|
||||
``get_settings()`` still reports the ENV value — the switch disables
|
||||
the mechanism, not the panel."""
|
||||
r = admin_client.put(f"{API}/relevance_threshold", json={"value": "0.5"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert get_settings().relevance_threshold == 0.5 # the switch was ON
|
||||
# Deterministic env base for the pin (machine-independent):
|
||||
monkeypatch.setenv("BOR_LLM_RETRIES", "3")
|
||||
invalidate_settings()
|
||||
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "0")
|
||||
invalidate_settings()
|
||||
|
||||
r = admin_client.get(API)
|
||||
assert r.status_code == 200
|
||||
assert r.json()["overlay_enabled"] is False
|
||||
assert r.json()["totals"]["overridden"] == 0 # no row is IN EFFECT
|
||||
item = _item(r.json(), "relevance_threshold")
|
||||
assert item["source"] == "env" # the row exists but is not in effect
|
||||
assert item["has_override"] is True
|
||||
assert item["db_value"] == "0.5" # the stored string is visible (non-secret)
|
||||
assert get_settings().relevance_threshold == 0.30 # the env value stands
|
||||
|
||||
# The panel stays writable: the PUT stores the row …
|
||||
r = admin_client.put(f"{API}/llm_retries", json={"value": "7"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 2
|
||||
# … but the env value stays in effect while the switch is off.
|
||||
assert get_settings().llm_retries == 3 # the env value (pinned 3 above)
|
||||
|
||||
|
||||
def test_catalogue_stays_under_the_byte_budget(admin_client) -> None:
|
||||
"""The whole catalogue stays under the phase-167 INFO bar (64 KB —
|
||||
``BOR_LOG_LARGE_RESPONSE_BYTES``), so ``GET /api/admin/settings``
|
||||
never floods the log (75 items ≈ 20 KB; the audit script re-verifies
|
||||
the 256 KB budget in task 06)."""
|
||||
r = admin_client.get(API)
|
||||
assert r.status_code == 200
|
||||
assert len(r.content) < 65_536
|
||||
|
||||
|
||||
def test_unknown_delete_is_a_404(admin_client) -> None:
|
||||
"""A DELETE of an UNKNOWN SETTING is a 404 like the PUT (a typo in
|
||||
the key is a client bug; the idempotency pin is about a known key
|
||||
whose row is already gone — the double Reset in
|
||||
``test_reset_to_env_and_clear_all``)."""
|
||||
r = admin_client.delete(f"{API}/bogus_key")
|
||||
assert r.status_code == 404
|
||||
assert r.json() == {"detail": "Unknown setting bogus_key"}
|
||||
|
||||
|
||||
def test_write_log_lines_carry_never_the_value(
|
||||
admin_client, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""AGENTS rule 10 / A4: one line per write (``settings action=…`` on
|
||||
the ``app.api.admin_settings`` logger) carrying the key and the
|
||||
action — NEVER the value (a secret write must not leak the
|
||||
plaintext into the log)."""
|
||||
caplog.set_level(logging.INFO, logger="app.api.admin_settings")
|
||||
|
||||
r = admin_client.put(f"{API}/llm_api_key", json={"value": "sk-log-secret-xyz"})
|
||||
assert r.status_code == 200, r.text
|
||||
set_lines = [rec.message for rec in caplog.records
|
||||
if rec.name == "app.api.admin_settings"]
|
||||
assert "settings action=set key=llm_api_key status=200" in set_lines
|
||||
assert "sk-log-secret-xyz" not in "\n".join(set_lines)
|
||||
|
||||
caplog.clear()
|
||||
r = admin_client.delete(f"{API}/llm_api_key")
|
||||
assert r.status_code == 200
|
||||
assert "settings action=clear key=llm_api_key" in [
|
||||
rec.message for rec in caplog.records
|
||||
if rec.name == "app.api.admin_settings"
|
||||
]
|
||||
|
||||
caplog.clear()
|
||||
admin_client.put(f"{API}/top_n_docs", json={"value": "3"})
|
||||
r = admin_client.delete(API)
|
||||
assert r.status_code == 200 and r.json() == {"cleared": 1}
|
||||
assert "settings action=clear_all count=1" in [
|
||||
rec.message for rec in caplog.records
|
||||
if rec.name == "app.api.admin_settings"
|
||||
]
|
||||
|
||||
|
||||
def test_bad_row_inserted_directly_never_bricks_boot(
|
||||
db: Session, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""Completion criterion (00_phase, item 5): a malformed row written
|
||||
straight into ``app_settings`` (bypassing the API's 422 gate — the
|
||||
operator SQL, the failed migration) must never brick boot. A FRESH
|
||||
``create_app()`` — the full boot path: fail-loud auth gate, resolver
|
||||
prime, middleware wiring — runs with the bad row present, the row is
|
||||
SKIPPED with a logged reason, the rest of the table still applies,
|
||||
and ``/api/health`` is 200 (A5)."""
|
||||
# One bad row (the API's 422 gate would refuse it) + one good row:
|
||||
# the bad one is skipped, the good one stands.
|
||||
db.execute(text(
|
||||
"INSERT INTO app_settings (key, value)"
|
||||
" VALUES ('relevance_threshold', 'not-a-number')"
|
||||
))
|
||||
db.execute(text(
|
||||
"INSERT INTO app_settings (key, value) VALUES ('top_n_docs', '3')"
|
||||
))
|
||||
db.commit()
|
||||
caplog.set_level(logging.ERROR, logger="app.settings")
|
||||
|
||||
# The boot path with the bad row in the table — must not raise.
|
||||
app = create_app()
|
||||
|
||||
s = get_settings()
|
||||
env_only = Settings() # the env value the skipped key falls back to
|
||||
assert s.relevance_threshold == env_only.relevance_threshold
|
||||
assert s.top_n_docs == 3 # the rest of the table still applies
|
||||
assert any(
|
||||
"settings override ignored" in rec.message
|
||||
and "relevance_threshold" in rec.message
|
||||
for rec in caplog.records
|
||||
)
|
||||
|
||||
r = TestClient(app).get("/api/health")
|
||||
assert r.status_code == 200
|
||||
|
||||
|
||||
def test_catalog_covers_every_settings_field(admin_client) -> None:
|
||||
"""The catalogue's keys are exactly ``Settings.model_fields`` (the
|
||||
API-level face of the unit completeness pin — a new setting cannot
|
||||
be missing from the panel's data)."""
|
||||
r = admin_client.get(API)
|
||||
assert r.status_code == 200
|
||||
assert {item["key"] for item in r.json()["items"]} == set(Settings.model_fields)
|
||||
assert set(CATALOG) == set(Settings.model_fields)
|
||||
@@ -0,0 +1,217 @@
|
||||
"""Phase 168 (task 03): the end-to-end proof of "no restart", in-process.
|
||||
|
||||
After ``invalidate_settings()`` (the save-side wrapper), the NEXT
|
||||
request behaves under the new value — pinned for the three settings
|
||||
that are NOT covered by the cache clear alone or that prove it end
|
||||
to end: the retrieval honesty gate (``relevance_threshold`` — the
|
||||
per-use read, ``app/api/chat.py``'s ``plan_turn`` gate), the shared
|
||||
``LLMClient`` (rebuilt: model + timeout from the new ``Settings``
|
||||
snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
|
||||
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
|
||||
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
|
||||
the ``scripts/``-consumer surface (``import_extension_set``).
|
||||
|
||||
The rows are inserted directly through the ``db`` session (the admin
|
||||
API is task 04); the save itself is ``invalidate_settings()`` —
|
||||
exactly what task 04's PUT/DELETE call after the write.
|
||||
|
||||
Real Postgres (``podman compose up -d db``); no LLM involved.
|
||||
|
||||
Requires: podman compose up -d db
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api import chat as chat_api
|
||||
from app.config import get_settings, invalidate_settings
|
||||
from app.core import settings_overlay
|
||||
from app.core.settings_overlay import register_invalidator
|
||||
from app.models import Document
|
||||
from app.rag.retriever import RetrievedChunk
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_state(
|
||||
db: Session, monkeypatch: pytest.MonkeyPatch
|
||||
) -> Iterator[None]:
|
||||
"""A clean ``app_settings`` table + a fresh overlay/hook slate.
|
||||
|
||||
``reset_state_for_tests()`` clears the process-wide hook list (a
|
||||
prior suite's test may have wiped the hook ``app.main`` registered
|
||||
at import — re-wire what ``create_app()`` wires: the chat runtime
|
||||
reset), then the table is emptied and the switch is ON for the
|
||||
test. Teardown deletes the rows and invalidates (runs the reset
|
||||
hook — a test-built ``LLMClient``/semaphore snapshot never leaks
|
||||
into a later suite) + clears the cache; ``monkeypatch`` then
|
||||
restores ``BOR_SETTINGS_DB`` and the next resolution is the
|
||||
conftest-pinned env-only view.
|
||||
"""
|
||||
settings_overlay.reset_state_for_tests()
|
||||
register_invalidator(chat_api.reset_runtime_caches)
|
||||
db.execute(text("DELETE FROM app_settings"))
|
||||
db.commit()
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
invalidate_settings()
|
||||
yield
|
||||
db.execute(text("DELETE FROM app_settings"))
|
||||
db.commit()
|
||||
invalidate_settings()
|
||||
get_settings.cache_clear()
|
||||
|
||||
|
||||
def _save(db: Session, key: str, value: str) -> None:
|
||||
"""One upserted override row (the panel's Save, minus the API)."""
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO app_settings (key, value) VALUES (:k, :v)"
|
||||
" ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value"
|
||||
),
|
||||
{"k": key, "v": value},
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _doc(title: str) -> Document:
|
||||
return Document(
|
||||
id=uuid.uuid4(),
|
||||
source="Homelab",
|
||||
path=f"{title.lower().replace(' ', '-')}.md",
|
||||
full_path="/tmp/doc.md",
|
||||
title=title,
|
||||
content=f"{title} content body for the gate pin",
|
||||
summary=None,
|
||||
content_hash="0" * 64,
|
||||
created_at=datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC),
|
||||
)
|
||||
|
||||
|
||||
def _chunk(doc: Document, score: float, cosine: float) -> RetrievedChunk:
|
||||
"""A non-FTS candidate: *cosine* is the honesty-gate input."""
|
||||
return RetrievedChunk(
|
||||
chunk_id=uuid.uuid4(),
|
||||
position=0,
|
||||
content=doc.content[:32],
|
||||
score=score,
|
||||
document=doc,
|
||||
cosine=cosine,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the per-use read: the A8 gate behaves under the saved threshold
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_relevance_threshold_db_value_bites_in_the_gate(db: Session) -> None:
|
||||
base = get_settings().relevance_threshold
|
||||
if base >= 0.95:
|
||||
pytest.skip("env threshold too close to 1.0 to bracket a DB override")
|
||||
# A DB value over the base; a cosine BETWEEN them — grounded under
|
||||
# the base, deflected under the DB value (no FTS: fts_hits = 0).
|
||||
db_value = min(base + 0.2, 0.9)
|
||||
cosine = (base + db_value) / 2
|
||||
chunk = _chunk(_doc("Gate Pin"), score=0.1, cosine=cosine)
|
||||
assert chat_api.plan_turn([chunk], get_settings()).deflected is False
|
||||
_save(db, "relevance_threshold", str(db_value))
|
||||
invalidate_settings()
|
||||
assert get_settings().relevance_threshold == db_value
|
||||
# the SAME chunk, the SAME function, the overlay-fed settings —
|
||||
# the gate now deflects (no process restart in between)
|
||||
assert chat_api.plan_turn([chunk], get_settings()).deflected is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the LLM client snapshot: Save swaps the shared LLMClient
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_llm_client_rebuilt_from_saved_values(db: Session) -> None:
|
||||
_save(db, "llm_chat_model", "gpt-test")
|
||||
_save(db, "llm_timeout", "7")
|
||||
invalidate_settings()
|
||||
llm = chat_api.get_llm()
|
||||
# the rebuilt client holds the DB-overridden snapshot…
|
||||
assert llm.settings.llm_chat_model == "gpt-test"
|
||||
assert llm.settings.llm_timeout == 7.0
|
||||
# …and its AsyncOpenAI transport was built from it (base_url +
|
||||
# the timeout's connect value — the way llm.py's __init__ builds
|
||||
# it; the openai client enforces a trailing slash on base_url)
|
||||
assert llm._client.base_url == ( # pyright: ignore[reportPrivateUsage]
|
||||
llm.settings.llm_base_url.rstrip("/") + "/"
|
||||
)
|
||||
assert llm._client._client.timeout.connect == 7.0 # pyright: ignore[reportPrivateUsage]
|
||||
|
||||
|
||||
def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
|
||||
_save(db, "llm_chat_model", "gpt-first")
|
||||
invalidate_settings()
|
||||
assert chat_api.get_llm().settings.llm_chat_model == "gpt-first"
|
||||
# the NEXT save re-points the singleton (not just the first)
|
||||
_save(db, "llm_chat_model", "gpt-second")
|
||||
invalidate_settings()
|
||||
llm = chat_api.get_llm()
|
||||
assert llm.settings.llm_chat_model == "gpt-second"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the chat concurrency semaphore follows the saved cap
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_chat_semaphore_follows_saved_cap(db: Session) -> None:
|
||||
base_cap = get_settings().chat_max_concurrent
|
||||
first = 3 if base_cap != 3 else 5
|
||||
_save(db, "chat_max_concurrent", str(first))
|
||||
invalidate_settings()
|
||||
assert get_settings().chat_max_concurrent == first
|
||||
sem = chat_api._get_chat_semaphore()
|
||||
assert sem._value == first # pyright: ignore[reportPrivateUsage]
|
||||
# the next save re-points the semaphore too
|
||||
second = 5 if first == 3 else 3
|
||||
_save(db, "chat_max_concurrent", str(second))
|
||||
invalidate_settings()
|
||||
rebuilt = chat_api._get_chat_semaphore()
|
||||
assert rebuilt is not sem
|
||||
assert rebuilt._value == second # pyright: ignore[reportPrivateUsage]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the kill switch: rows present, switch off → the env values are back
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_kill_switch_returns_env_values(
|
||||
db: Session, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
base = get_settings().input_placeholder
|
||||
_save(db, "input_placeholder", "DB-WINS-PLACEHOLDER")
|
||||
invalidate_settings()
|
||||
assert get_settings().input_placeholder == "DB-WINS-PLACEHOLDER"
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "0")
|
||||
invalidate_settings()
|
||||
assert get_settings().input_placeholder == base
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the scripts/ consumer: the importer reads the same overlay
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_import_extensions_override_reaches_script_consumer(db: Session) -> None:
|
||||
base = get_settings().import_extension_set
|
||||
# a REAL change — guard against a dev .env that already carries
|
||||
# exactly this set (a no-op would still pass the final assert)
|
||||
if base == {".md", ".txt", ".csv"}:
|
||||
saved, expect = "md,txt,csv,ini", {".md", ".txt", ".csv", ".ini"}
|
||||
else:
|
||||
saved, expect = "md,txt,csv", {".md", ".txt", ".csv"}
|
||||
_save(db, "import_extensions", saved)
|
||||
invalidate_settings()
|
||||
assert get_settings().import_extension_set == expect
|
||||
@@ -88,6 +88,7 @@ SHELL_BACKED_PAGES = {
|
||||
"/history.html": "index.html", # phase 76 task 03
|
||||
"/tokens.html": "index.html", # phase 79 task 06
|
||||
"/theme.html": "index.html", # phase 91 task 04
|
||||
"/settings.html": "index.html", # phase 168 task 05
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
"""Integration: migration 0028 (app_settings) schema contract
|
||||
(phase 168, task 01).
|
||||
|
||||
Drives the **real Alembic engine** against the live dev database
|
||||
(``podman compose up -d db``), mirroring the house pattern of
|
||||
``test_migration_0017.py`` (information_schema assertions on the state
|
||||
the migration must leave). The tests target the 0027 → 0028 step
|
||||
explicitly so later migrations cannot break them:
|
||||
|
||||
* upgrade 0027 → 0028 → the ``app_settings`` table exists with the
|
||||
full column contract — ``key`` VARCHAR(100) NOT NULL and the
|
||||
primary key (one row per BARE ``Settings`` field name; the PK is
|
||||
the uniqueness guard — two rows for one setting would make
|
||||
"which one wins" undefined), ``value`` TEXT NOT NULL (the env-
|
||||
native string form; the reset is a DELETE, never an empty value),
|
||||
``updated_at`` TIMESTAMPTZ NOT NULL with the now() server default
|
||||
(house style) — while the 0027 schema (``documents.
|
||||
page_strip_mode``) survives;
|
||||
* inserted rows round-trip (``relevance_threshold`` / ``"0.55"`` and
|
||||
``import_extensions`` / ``"md,txt"``);
|
||||
* a duplicate ``key`` raises ``IntegrityError`` (the PK pin) and a
|
||||
NULL ``value`` is rejected (NOT NULL);
|
||||
* downgrade to 0027 → the table is GONE (A13 — reversible), the rest
|
||||
of the schema (``documents.page_strip_mode``, ``ui_settings``)
|
||||
survives;
|
||||
* upgrade back to 0028 → the table is back (round-trip).
|
||||
|
||||
The ``alembic`` fixture guarantees the DB ends at head even if a test
|
||||
fails or the process is interrupted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from alembic.config import Config
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from alembic import command
|
||||
from app.db import db_available
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def alembic(db: Session) -> Iterator[Config]:
|
||||
"""Real Alembic config bound to the dev DB (URL from app settings).
|
||||
|
||||
Starts at head (repairs an interrupted earlier run); teardown
|
||||
upgrades to head no matter what happened, so the dev DB is never
|
||||
left below head.
|
||||
"""
|
||||
if not db_available():
|
||||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||||
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
|
||||
cfg.set_main_option("script_location", "alembic")
|
||||
command.upgrade(cfg, "head")
|
||||
try:
|
||||
yield cfg
|
||||
finally:
|
||||
command.upgrade(cfg, "head")
|
||||
|
||||
|
||||
def _version(db: Session) -> str | None:
|
||||
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
|
||||
|
||||
|
||||
def _table_exists(db: Session, table: str) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
text("SELECT 1 FROM information_schema.tables WHERE table_name = :t"),
|
||||
{"t": table},
|
||||
).scalar()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _column(db: Session, table: str, column: str) -> tuple[Any, ...] | None:
|
||||
"""(data_type, is_nullable, column_default, character_maximum_length)
|
||||
for one table column."""
|
||||
row = db.execute(
|
||||
text(
|
||||
"SELECT data_type, is_nullable, column_default, character_maximum_length"
|
||||
" FROM information_schema.columns"
|
||||
" WHERE table_name = :t AND column_name = :c"
|
||||
),
|
||||
{"t": table, "c": column},
|
||||
).fetchone()
|
||||
return tuple(row) if row is not None else None
|
||||
|
||||
|
||||
def _pk_columns(db: Session, table: str) -> list[str]:
|
||||
"""The table's PRIMARY KEY columns in ordinal position."""
|
||||
rows = db.execute(
|
||||
text(
|
||||
"SELECT kcu.column_name"
|
||||
" FROM information_schema.table_constraints tc"
|
||||
" JOIN information_schema.key_column_usage kcu"
|
||||
" ON kcu.constraint_name = tc.constraint_name"
|
||||
" AND kcu.table_name = tc.table_name"
|
||||
" WHERE tc.table_name = :t"
|
||||
" AND tc.constraint_type = 'PRIMARY KEY'"
|
||||
" ORDER BY kcu.ordinal_position"
|
||||
),
|
||||
{"t": table},
|
||||
).fetchall()
|
||||
return [r[0] for r in rows]
|
||||
|
||||
|
||||
def _clear_rows(db: Session) -> None:
|
||||
db.execute(text("DELETE FROM app_settings"))
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_upgrade_to_0028_creates_app_settings(db: Session, alembic: Config) -> None:
|
||||
"""Upgrade 0027 → 0028: the table exists with the full column
|
||||
contract (``key`` VARCHAR(100) NOT NULL + the sole PK; ``value``
|
||||
TEXT NOT NULL; ``updated_at`` TIMESTAMPTZ NOT NULL with the now()
|
||||
server default), and the table is ABSENT at 0027 while the 0027
|
||||
``documents`` schema survives the upgrade."""
|
||||
command.downgrade(alembic, "0027") # start from the pre-0028 state
|
||||
assert _version(db) == "0027"
|
||||
assert not _table_exists(db, "app_settings"), (
|
||||
"app_settings must be absent at 0027"
|
||||
)
|
||||
|
||||
command.upgrade(alembic, "0028")
|
||||
assert _version(db) == "0028", "alembic_version must be at 0028"
|
||||
assert _table_exists(db, "app_settings"), "the table must exist at 0028"
|
||||
|
||||
key = _column(db, "app_settings", "key")
|
||||
assert key is not None, "app_settings.key is missing"
|
||||
assert key[0] == "character varying", "key must be VARCHAR"
|
||||
assert key[1] == "NO", "key must be NOT NULL (PK)"
|
||||
assert key[2] is None, "key must have no server default"
|
||||
assert key[3] == 100, "key must be String(100)"
|
||||
|
||||
value = _column(db, "app_settings", "value")
|
||||
assert value is not None, "app_settings.value is missing"
|
||||
assert value[0] == "text", "value must be TEXT (the env-native string form)"
|
||||
assert value[1] == "NO", "value must be NOT NULL (reset is a DELETE, never empty)"
|
||||
|
||||
updated = _column(db, "app_settings", "updated_at")
|
||||
assert updated is not None, "app_settings.updated_at is missing"
|
||||
assert updated[0] == "timestamp with time zone", "updated_at must be TIMESTAMPTZ"
|
||||
assert updated[1] == "NO", "updated_at must be NOT NULL"
|
||||
assert updated[2] is not None and "now" in str(updated[2]), (
|
||||
"updated_at must carry the now() server default (house style)"
|
||||
)
|
||||
|
||||
assert _pk_columns(db, "app_settings") == ["key"], (
|
||||
"key must be the sole primary key (the uniqueness guard —"
|
||||
" two rows for one setting make 'which one wins' undefined)"
|
||||
)
|
||||
|
||||
# The 0027 schema survives the additive upgrade.
|
||||
strip = _column(db, "documents", "page_strip_mode")
|
||||
assert strip is not None, "documents.page_strip_mode (0027) must survive"
|
||||
assert strip[3] == 16, "page_strip_mode must stay String(16)"
|
||||
|
||||
|
||||
def test_inserted_rows_round_trip(db: Session, alembic: Config) -> None:
|
||||
"""At 0028, the env-native string forms round-trip verbatim
|
||||
(``relevance_threshold`` / ``"0.55"``, ``import_extensions`` /
|
||||
``"md,txt"``) and ``updated_at`` is stamped by the server default."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO app_settings (key, value)"
|
||||
" VALUES ('relevance_threshold', '0.55')"
|
||||
)
|
||||
)
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO app_settings (key, value)"
|
||||
" VALUES ('import_extensions', 'md,txt')"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
rows = db.execute(
|
||||
text(
|
||||
"SELECT key, value, updated_at"
|
||||
" FROM app_settings ORDER BY key"
|
||||
)
|
||||
).fetchall()
|
||||
assert len(rows) == 2, "both rows must be stored"
|
||||
assert rows[0][0] == "import_extensions", "key must round-trip verbatim"
|
||||
assert rows[0][1] == "md,txt", "the CSV string form must round-trip"
|
||||
assert rows[0][2] is not None, "updated_at must be stamped (server default)"
|
||||
assert rows[1][0] == "relevance_threshold"
|
||||
assert rows[1][1] == "0.55", "the numeric string form must round-trip"
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_duplicate_key_is_rejected(db: Session, alembic: Config) -> None:
|
||||
"""The PK is the uniqueness guard: two rows for one setting would
|
||||
make 'which one wins' undefined, so the second insert must raise
|
||||
``IntegrityError``."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
db.execute(
|
||||
text("INSERT INTO app_settings (key, value) VALUES ('llm_retries', '3')")
|
||||
)
|
||||
db.commit()
|
||||
with pytest.raises(IntegrityError):
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO app_settings (key, value)"
|
||||
" VALUES ('llm_retries', '5')"
|
||||
)
|
||||
)
|
||||
db.rollback() # the IntegrityError aborts the open transaction
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_null_value_is_rejected(db: Session, alembic: Config) -> None:
|
||||
"""``value`` is NOT NULL: 'clear the override' is a DELETE of the
|
||||
row, never a NULL/empty value, so the resolver never has to
|
||||
distinguish 'no override' from 'override to empty'."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
with pytest.raises(IntegrityError):
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO app_settings (key, value)"
|
||||
" VALUES ('chat_max_concurrent', NULL)"
|
||||
)
|
||||
)
|
||||
db.rollback() # the IntegrityError aborts the open transaction
|
||||
stored = db.execute(
|
||||
text("SELECT count(*) FROM app_settings")
|
||||
).scalar()
|
||||
assert stored == 0, "the rejected row must not be stored"
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_downgrade_to_0027_drops_the_table(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade 0028 → 0027: the table is gone (A13 — fully
|
||||
reversible) while the rest of the schema survives (the 0027
|
||||
``documents.page_strip_mode`` column, ``ui_settings``,
|
||||
``api_tokens``). The values are re-enterable from the panel —
|
||||
they are an override of the env, not a source of truth."""
|
||||
command.downgrade(alembic, "0027")
|
||||
assert _version(db) == "0027"
|
||||
assert not _table_exists(db, "app_settings"), "app_settings must be dropped"
|
||||
|
||||
strip = _column(db, "documents", "page_strip_mode")
|
||||
assert strip is not None and strip[3] == 16, (
|
||||
"documents.page_strip_mode (0027) must survive the downgrade"
|
||||
)
|
||||
semantic = _column(db, "ui_settings", "ok_bg")
|
||||
assert semantic is not None and semantic[3] == 7, (
|
||||
"ui_settings.ok_bg must survive the downgrade"
|
||||
)
|
||||
token_col = _column(db, "api_tokens", "token_hash")
|
||||
assert token_col is not None and token_col[0] == "character varying", (
|
||||
"api_tokens.token_hash must survive the downgrade"
|
||||
)
|
||||
|
||||
|
||||
def test_upgrade_round_trip_restores_the_table(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade to 0027, then upgrade back to 0028: the table is back
|
||||
with the column contract and PK intact (A13 round-trip)."""
|
||||
command.downgrade(alembic, "0027")
|
||||
command.upgrade(alembic, "0028")
|
||||
assert _version(db) == "0028", "round-trip upgrade must land at 0028"
|
||||
assert _table_exists(db, "app_settings"), "the table must be back"
|
||||
|
||||
key = _column(db, "app_settings", "key")
|
||||
assert key is not None, "app_settings.key must be back"
|
||||
assert key[0] == "character varying", "key must be VARCHAR after the round-trip"
|
||||
assert key[1] == "NO", "key must be NOT NULL after the round-trip"
|
||||
assert key[3] == 100, "key must be String(100) after the round-trip"
|
||||
value = _column(db, "app_settings", "value")
|
||||
assert value is not None, "app_settings.value must be back"
|
||||
assert value[0] == "text", "value must be TEXT after the round-trip"
|
||||
assert value[1] == "NO", "value must be NOT NULL after the round-trip"
|
||||
assert _pk_columns(db, "app_settings") == ["key"]
|
||||
@@ -0,0 +1,174 @@
|
||||
"""Unit: the admin settings router's pure helpers (phase 168, task 04)
|
||||
— the env-native stringification (``_display``), the control-kind
|
||||
derivation (``_field_type``), the ``_item`` builder's masked and
|
||||
default branches, and the LOCKED A1 static pins: the module never
|
||||
opens a file (``.env`` is never written), never shells out and never
|
||||
``eval``s a value.
|
||||
|
||||
The DB-driven behavior (the four routes, the candidate-set validation,
|
||||
the auth matrix) is pinned in
|
||||
``tests/integration/test_admin_settings_api.py``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
import re
|
||||
|
||||
import pytest
|
||||
|
||||
from app.api.admin_settings import _display, _field_type, _item
|
||||
from app.config import Settings
|
||||
from app.core.settings_overlay import LIST_FIELDS
|
||||
|
||||
|
||||
def test_display_bools_are_the_env_spelling() -> None:
|
||||
"""bool → ``"true"``/``"false"`` — checked BEFORE int (``isinstance(True, int)``
|
||||
is ``True`` — a bool field must never stringify through ``str()``'s
|
||||
``"True"``)."""
|
||||
assert _display("stream_thinking", True) == "true"
|
||||
assert _display("stream_thinking", False) == "false"
|
||||
# An int value for an int field is the plain str (no bool confusion).
|
||||
assert _display("llm_retries", 3) == "3"
|
||||
|
||||
|
||||
def test_display_list_field_is_json() -> None:
|
||||
"""The one ``list[str]`` field (``suggestions``) stringifies as the
|
||||
JSON array the env spelling would carry (K2: stored in the same
|
||||
JSON form, pydantic parses it exactly like the env var)."""
|
||||
assert set(LIST_FIELDS) == {"suggestions"}
|
||||
assert _display("suggestions", ["a", "b"]) == '["a", "b"]'
|
||||
assert _display("suggestions", []) == "[]"
|
||||
|
||||
|
||||
def test_display_scalars_are_plain_strings() -> None:
|
||||
"""Everything else is ``str()`` — floats, ints, strings, in the
|
||||
env-native form (what ``app_settings`` stores, D1)."""
|
||||
assert _display("relevance_threshold", 0.55) == "0.55"
|
||||
assert _display("llm_timeout", 7.0) == "7.0"
|
||||
assert _display("app_name", "Brain of Reese") == "Brain of Reese"
|
||||
assert _display("import_extensions", "md,txt") == "md,txt"
|
||||
|
||||
|
||||
def test_field_type_covers_all_kinds() -> None:
|
||||
"""The panel control kind per annotation: ``bool`` before ``int``
|
||||
(a bool annotation is never misread as int), the one list field is
|
||||
``json``, the rest fall through to their annotation."""
|
||||
assert _field_type("stream_thinking") == "bool"
|
||||
assert _field_type("llm_retries") == "int"
|
||||
assert _field_type("relevance_threshold") == "float"
|
||||
assert _field_type("app_name") == "str"
|
||||
assert _field_type("suggestions") == "json"
|
||||
# Every field resolves to one of the five wire kinds.
|
||||
kinds = {"int", "float", "bool", "str", "json"}
|
||||
for key in Settings.model_fields:
|
||||
assert _field_type(key) in kinds
|
||||
|
||||
|
||||
def test_item_masks_secret_fields() -> None:
|
||||
"""LOCKED A4: a secret field's item is write-only on the wire —
|
||||
``masked=True``, ``value=None``, ``db_value=None``,
|
||||
``default=""`` (the code default is never shown for a secret),
|
||||
``has_value`` = is a non-empty value in effect at all (the test
|
||||
env sets a non-empty admin password / session secret)."""
|
||||
effective = Settings()
|
||||
assert effective.admin_password # tests/conftest.py pins it non-empty
|
||||
item = _item("admin_password", {}, effective)
|
||||
assert item.masked is True
|
||||
assert item.value is None
|
||||
assert item.db_value is None
|
||||
assert item.default == ""
|
||||
assert item.has_value is True
|
||||
assert item.secret is True
|
||||
assert item.key == "admin_password"
|
||||
assert item.env_var == "BOR_ADMIN_PASSWORD"
|
||||
# A stored row for a secret field never exposes the text either.
|
||||
# (The effective value is forced via an init kwarg — the operator's
|
||||
# local `.env` may legitimately carry a key, so the test does not
|
||||
# assume a specific effective value.)
|
||||
item = _item("llm_api_key", {"llm_api_key": "sk-x"}, Settings(llm_api_key="sk-x"))
|
||||
assert item.masked is True
|
||||
assert item.value is None
|
||||
assert item.db_value is None
|
||||
assert item.has_value is True
|
||||
assert item.has_override is True
|
||||
item = _item("llm_api_key", {}, Settings(llm_api_key=""))
|
||||
assert item.masked is True
|
||||
assert item.value is None
|
||||
assert item.db_value is None
|
||||
assert item.has_value is False # no value in effect at all
|
||||
|
||||
|
||||
def test_item_reports_env_default_and_db_sources(
|
||||
monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""The ``source`` badge: ``db`` for an in-effect overridable row,
|
||||
``env`` when the ``BOR_*`` var is in the process environment,
|
||||
``default`` otherwise — and a non-overridable row is NEVER an
|
||||
override (LOCKED A3: the overlay drops it, so a hand-inserted
|
||||
``database_url`` row is not in effect)."""
|
||||
import os
|
||||
|
||||
effective = Settings()
|
||||
# "env": tests/conftest.py sets BOR_RELEVANCE_THRESHOLD.
|
||||
assert os.environ.get("BOR_RELEVANCE_THRESHOLD") is not None
|
||||
item = _item("relevance_threshold", {}, effective)
|
||||
assert item.source == "env"
|
||||
# The value is the EFFECTIVE float stringified ("0.30" → 0.3 → "0.3").
|
||||
assert item.value == str(float(os.environ["BOR_RELEVANCE_THRESHOLD"]))
|
||||
assert item.has_override is False
|
||||
# "default": no BOR_DOCS_WORK_DIR in the test environment.
|
||||
assert os.environ.get("BOR_DOCS_WORK_DIR") is None
|
||||
item = _item("docs_work_dir", {}, effective)
|
||||
assert item.source == "default"
|
||||
assert item.value == effective.docs_work_dir
|
||||
# "db": the row is in effect only while the overlay switch is ON
|
||||
# (the conftest pin is 0 — flip it for the pin, it restores after).
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
item = _item("relevance_threshold", {"relevance_threshold": "0.9"}, effective)
|
||||
assert item.source == "db"
|
||||
assert item.has_override is True
|
||||
# Non-overridable: a stored row is not an override (LOCKED A3) —
|
||||
# even with the switch on.
|
||||
item = _item("database_url", {"database_url": "postgresql://x"}, effective)
|
||||
assert item.has_override is False
|
||||
assert item.masked is True
|
||||
# The code default stringifies (non-secret fields).
|
||||
item = _item("llm_retries", {}, effective)
|
||||
assert item.default == "3"
|
||||
assert item.value == "3"
|
||||
assert item.db_value is None
|
||||
|
||||
|
||||
def test_item_db_value_is_the_stored_string() -> None:
|
||||
"""A stored non-secret override exposes ``db_value`` (the stored
|
||||
string, D1) and ``has_override`` — even while the overlay switch
|
||||
is OFF (the row is stored but not in effect: the badge is
|
||||
env/default, the panel's ``overlay_enabled`` flag explains the
|
||||
rest)."""
|
||||
effective = Settings()
|
||||
item = _item("llm_retries", {"llm_retries": "7"}, effective)
|
||||
assert item.has_override is True
|
||||
assert item.db_value == "7"
|
||||
assert item.source == "default" # switch off (the conftest pin): not in effect
|
||||
|
||||
|
||||
def test_module_never_writes_files_or_shells_out() -> None:
|
||||
"""LOCKED A1 + the module's third invariant (the unit pin): the
|
||||
router never opens a file (``.env`` is never written by any code
|
||||
path), never shells out and never ``eval``s a value. (The docstring
|
||||
legitimately mentions ``.env`` — the pins are on the CALLS, the
|
||||
``test_settings_overlay`` house style.)"""
|
||||
source = (
|
||||
pathlib.Path(__file__).resolve().parents[2]
|
||||
/ "app"
|
||||
/ "api"
|
||||
/ "admin_settings.py"
|
||||
).read_text()
|
||||
assert re.search(r"\bopen\s*\(", source) is None, (
|
||||
"app/api/admin_settings.py must never open() a file (LOCKED A1: "
|
||||
".env is never written)"
|
||||
)
|
||||
assert "subprocess" not in source
|
||||
assert "os.system" not in source
|
||||
assert re.search(r"\beval\s*\(", source) is None
|
||||
assert "import shutil" not in source and "shutil." not in source
|
||||
@@ -218,6 +218,7 @@ def test_html_pages_include_history() -> None:
|
||||
"/history.html",
|
||||
"/tokens.html", # phase 79 task 06: the admin tokens page (shell route)
|
||||
"/theme.html", # phase 91 task 04: the admin theme page (shell route)
|
||||
"/settings.html", # phase 168 task 05: the admin settings page (shell route)
|
||||
"/shared.html", # phase 51: the shared page's static path
|
||||
"/doc-edit.html", # phase 59: the doc edit screen (task 06)
|
||||
):
|
||||
|
||||
@@ -128,8 +128,11 @@ def test_view_map_covers_the_shell_paths() -> None:
|
||||
assert '"/theme.html": "theme"' in view_body, (
|
||||
"phase 91 task 04 folds the Theme view into the shell"
|
||||
)
|
||||
assert '"/settings.html": "settings"' in view_body, (
|
||||
"phase 168 task 05 folds the Settings view into the shell"
|
||||
)
|
||||
# The view names are the #view-<name> section slugs in index.html.
|
||||
for name in ("chat", "tuning", "history", "tokens", "theme"):
|
||||
for name in ("chat", "tuning", "history", "tokens", "theme", "settings"):
|
||||
assert f'id="view-{name}"' in _html(), f"missing the #view-{name} section"
|
||||
|
||||
|
||||
@@ -226,6 +229,10 @@ def test_only_non_chat_views_have_lazy_modules() -> None:
|
||||
assert 'theme: () => import("./theme.js")' in mods_body, (
|
||||
"the Theme view module is lazy-imported on first show (phase 91)"
|
||||
)
|
||||
assert 'settings: () => import("./settings.js")' in mods_body, (
|
||||
"the Settings view module is lazy-imported on first show "
|
||||
"(phase 168, a static specifier the bundle inlines)"
|
||||
)
|
||||
assert '"chat"' not in mods_body, "the chat view has no lazy module"
|
||||
assert 'import("./app.js")' not in js, "app.js must never be lazy-imported"
|
||||
|
||||
@@ -288,6 +295,11 @@ def test_router_writes_active_state_title_and_meta() -> None:
|
||||
"Set the palette and branding — the theme is baked into every served page, "
|
||||
"live on the first paint."
|
||||
) in js
|
||||
assert 'settings: "Settings · Brain of Reese"' in js
|
||||
assert (
|
||||
"Edit any application setting — a saved value wins over the environment, "
|
||||
"live on save; reset hands control back."
|
||||
) in js
|
||||
# The brand composition (phase 39's window.BOR_BRAND, read at
|
||||
# write time — never a hardcoded stamp).
|
||||
assert 'window.BOR_BRAND || "Brain of Reese"' in js
|
||||
|
||||
@@ -4,6 +4,8 @@ from __future__ import annotations
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import app.main as main_mod
|
||||
from app.config import get_settings
|
||||
from app.core import settings_overlay
|
||||
|
||||
|
||||
def test_create_app_warns_and_serves_api_only_without_static_dir(
|
||||
@@ -30,3 +32,22 @@ def test_create_app_wires_caching_middleware() -> None:
|
||||
# reach for ``__name__`` the same way starlette's own __repr__ does.
|
||||
middleware_names = [getattr(mw.cls, "__name__", "") for mw in app2.user_middleware]
|
||||
assert "CachingMiddleware" in middleware_names
|
||||
|
||||
|
||||
def test_create_app_boots_with_the_switch_on(monkeypatch) -> None:
|
||||
"""Phase 168 (task 02/03): the boot prime with the kill switch ON
|
||||
— the ``settings overrides loaded count=…`` branch (the OFF branch
|
||||
is exercised by every other suite's app import; the overlay's
|
||||
engine is fail-soft, so a DB-down unit environment still boots).
|
||||
``create_app`` also wires the chat runtime reset hook here (task
|
||||
03) — the ``register_invalidator`` pin lives in
|
||||
``test_settings_live_apply.py``."""
|
||||
settings_overlay.reset_state_for_tests()
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
app2 = main_mod.create_app()
|
||||
assert app2 is not None
|
||||
# restore: the switch reverts via monkeypatch; the cache clear
|
||||
# ensures a boot-time row set never leaks into later suites'
|
||||
# settings, and the hook slate is fresh for them too.
|
||||
settings_overlay.reset_state_for_tests()
|
||||
get_settings.cache_clear()
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
"""Phase 168 (task 02): the settings catalogue is complete and consistent.
|
||||
|
||||
The Settings panel (task 05) is GENERATED from
|
||||
``app/core/settings_catalog.py`` — never hand-written — so these pins
|
||||
are load-bearing: ``set(CATALOG) == set(Settings.model_fields)`` fails
|
||||
the build the moment a new ``Settings`` field lands without its
|
||||
metadata (it can never silently disappear from the panel), the group
|
||||
table is closed both directions, and the capability sets are exactly
|
||||
the LOCKED decisions (A3/A4).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from app.config import Settings
|
||||
from app.core.settings_catalog import (
|
||||
CATALOG,
|
||||
GROUPS,
|
||||
NON_OVERRIDABLE,
|
||||
RESTART_REQUIRED,
|
||||
SECRETS,
|
||||
SettingMeta,
|
||||
env_var,
|
||||
)
|
||||
|
||||
|
||||
def test_catalog_covers_exactly_model_fields() -> None:
|
||||
"""The completeness pin (both directions, the diff in the message)."""
|
||||
catalog_keys = set(CATALOG)
|
||||
field_names = set(Settings.model_fields)
|
||||
missing = sorted(field_names - catalog_keys)
|
||||
extra = sorted(catalog_keys - field_names)
|
||||
assert not missing, (
|
||||
f"Settings fields missing from the catalogue (invisible in the "
|
||||
f"panel): {missing}"
|
||||
)
|
||||
assert not extra, (
|
||||
f"Catalogue entries with no Settings field (stale): {extra}"
|
||||
)
|
||||
assert set(CATALOG) == set(Settings.model_fields)
|
||||
|
||||
|
||||
def test_groups_closed_both_directions() -> None:
|
||||
catalog_groups = {meta.group for meta in CATALOG.values()}
|
||||
only_in_groups = sorted(set(GROUPS) - catalog_groups)
|
||||
only_in_catalog = sorted(catalog_groups - set(GROUPS))
|
||||
assert not only_in_groups, f"GROUPS entries with no catalogue member: {only_in_groups}"
|
||||
assert not only_in_catalog, f"Catalogue groups missing from GROUPS: {only_in_catalog}"
|
||||
|
||||
|
||||
def test_every_meta_carries_its_own_key() -> None:
|
||||
for key, meta in CATALOG.items():
|
||||
assert isinstance(meta, SettingMeta)
|
||||
assert meta.key == key
|
||||
|
||||
|
||||
def test_secrets_are_exactly_the_masked_set() -> None:
|
||||
# LOCKED A4: write-only, masked — the 3 named secrets +
|
||||
# database_url (read-only, but its URL carries credentials).
|
||||
assert set(SECRETS) == {
|
||||
"llm_api_key",
|
||||
"admin_password",
|
||||
"session_secret",
|
||||
"database_url",
|
||||
}
|
||||
|
||||
|
||||
def test_non_overridable_is_exactly_locked_a3() -> None:
|
||||
# LOCKED A3: database_url builds the engine that would read the
|
||||
# override (chicken-and-egg brick); embedding_dim is fixed in the
|
||||
# chunks table by A6 (a dim override would corrupt retrieval).
|
||||
assert set(NON_OVERRIDABLE) == {"database_url", "embedding_dim"}
|
||||
|
||||
|
||||
def test_restart_required_is_the_boot_wired_set() -> None:
|
||||
# LOCKED A4: logging, the gzip middleware (compress_min_bytes), the
|
||||
# static mount (static_dir), create_engine (db_pool_*), and
|
||||
# SessionMiddleware (session_*).
|
||||
assert set(RESTART_REQUIRED) == {
|
||||
"log_level",
|
||||
"compress_min_bytes",
|
||||
"static_dir",
|
||||
"db_pool_size",
|
||||
"db_pool_max_overflow",
|
||||
"db_pool_recycle",
|
||||
"session_secret",
|
||||
"session_max_age",
|
||||
"session_cookie",
|
||||
}
|
||||
|
||||
|
||||
def test_env_var_is_bare_key_upper_prefixed() -> None:
|
||||
assert env_var("relevance_threshold") == "BOR_RELEVANCE_THRESHOLD"
|
||||
for key in CATALOG:
|
||||
assert env_var(key).startswith("BOR_")
|
||||
|
||||
|
||||
def test_labels_nonempty_and_unique() -> None:
|
||||
labels = [meta.label for meta in CATALOG.values()]
|
||||
assert all(label.strip() for label in labels), "empty label in the catalogue"
|
||||
duplicates = sorted({label for label in labels if labels.count(label) > 1})
|
||||
assert not duplicates, f"duplicate labels: {duplicates}"
|
||||
@@ -0,0 +1,608 @@
|
||||
"""Unit: the phase-168 Settings view wiring (task 05) — the static
|
||||
source-level pins for the shell's eighth view.
|
||||
|
||||
The browser behavior is E2E-gated (the phase-168 story suite —
|
||||
``test_admin_settings_panel.py``, task 06); here we pin the
|
||||
source-level invariants the "views of one document" architecture
|
||||
depends on, so a silent regression is caught without a browser
|
||||
(house pattern: ``tests/unit/test_thinking_scroll.py`` /
|
||||
``test_frontend_router.py`` read the sources and assert on their
|
||||
mechanisms).
|
||||
|
||||
Pinned (task 05 spec):
|
||||
|
||||
* ``index.html`` carries the ``#view-settings`` section (ship-hidden
|
||||
+ inert + focusable — the WCAG pair, AGENTS.md rule 5) with the
|
||||
``#settings-gate`` (ship-hidden, its Sign in returning to the
|
||||
Settings view), the ship-hidden ``#settings-content`` holding the
|
||||
``#settings-disabled`` kill-switch banner (D6), the ONE
|
||||
``role="status"`` region, the ``role="alert"`` error line, the
|
||||
client-side search input, the ``.table-wrap`` catalogue with the
|
||||
visually-hidden caption + the four-column thead, and the action
|
||||
row (``#settings-save`` — ships disabled — ``#settings-clear-all``
|
||||
+ the aria-hidden ``#settings-dirty`` hint); the ``#nav-settings``
|
||||
link ships hidden (admin-only).
|
||||
* ``router.js`` maps ``/settings.html`` in ALL THREE tables
|
||||
(VIEW / VIEW_PATH / VIEW_MODULES — a static specifier so the
|
||||
Containerfile's esbuild stage inlines it into the router bundle)
|
||||
+ the per-view title/description (the router is the SINGLE writer
|
||||
of both).
|
||||
* ``app/main.py``'s shell tuple and ``app/core/caching.py``'s
|
||||
``HTML_PAGES`` both list ``/settings.html`` (the no-cache +
|
||||
``?v=`` contract, A19).
|
||||
* ``header.js`` reveals ``#nav-settings`` keyed off ``admin`` (NOT
|
||||
``signedIn`` — a token user never sees it), the same two-line
|
||||
null-safe pattern as the other admin-only links.
|
||||
* ``settings.js`` builds the DOM with ``createElement`` /
|
||||
``textContent`` — NO ``innerHTML`` (template-literal or not) and
|
||||
NO ``http(s)://`` reference (rule 6 — everything local; the
|
||||
no-HTML-strings house rule).
|
||||
* ``styles.css`` declares ``.settings-table`` + the three
|
||||
``.settings-src.is-*`` variants + the dirty-row marker and adds
|
||||
NO new ``:root`` variable (the phase-156 B3 rule — the badges
|
||||
are solid existing tokens).
|
||||
* the FOUR pages that carry the Theme link (index / document /
|
||||
login / shared) now carry the Settings link too (the phase-34
|
||||
one-bar contract — test_nav_consistency pins the parity).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||||
ASSETS = FRONTEND / "assets"
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _read(path: Path) -> str:
|
||||
assert path.is_file(), f"missing {path}"
|
||||
return path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _html() -> str:
|
||||
return _read(FRONTEND / "index.html")
|
||||
|
||||
|
||||
# ---------- the shell markup: the #view-settings section ----------
|
||||
|
||||
|
||||
def test_settings_view_section_ships_hidden_inert_focusable() -> None:
|
||||
"""The #view-settings section lives in the shell's single main,
|
||||
ships hidden AND inert (the WCAG a11y pair — a hidden view must
|
||||
not receive focus or keyboard traversal), and is focusable
|
||||
(tabindex="-1" — the router focuses it on user-initiated
|
||||
switches)."""
|
||||
html = _html()
|
||||
view = html.find('<section class="view" id="view-settings"')
|
||||
assert view != -1, "the #view-settings section must be in the shell"
|
||||
tag_end = html.find(">", view)
|
||||
tag = html[view:tag_end]
|
||||
assert "hidden" in tag and "inert" in tag, (
|
||||
"the folded view ships hidden AND inert"
|
||||
)
|
||||
assert 'tabindex="-1"' in tag, "the target view is focusable"
|
||||
assert 'aria-label="Settings"' in tag, "the view is labelled"
|
||||
main_end = html.find("</main>", view)
|
||||
assert view < main_end, "the view section lives inside the single main"
|
||||
# The 72rem .container — no new width token (PLAN §7.1).
|
||||
body = html[view:main_end]
|
||||
assert 'class="container settings-shell"' in body, (
|
||||
"the view rides the standard 72rem .container (no new width)"
|
||||
)
|
||||
|
||||
|
||||
def test_settings_page_head_states_the_rule_in_the_owner_terms() -> None:
|
||||
"""The page-head: <h1>Settings</h1> + a .page-sub that states
|
||||
the rule in the owner's terms — the value is stored in the
|
||||
database and wins over the environment variables; clearing hands
|
||||
control back to the environment (and the .env file is never
|
||||
written — LOCKED A1)."""
|
||||
html = _html()
|
||||
view = html.find('<section class="view" id="view-settings"')
|
||||
main_end = html.find("</main>", view)
|
||||
body = html[view:main_end]
|
||||
assert "<h1>Settings</h1>" in body
|
||||
sub = re.search(r'<p class="page-sub">([\s\S]*?)</p>', body)
|
||||
assert sub, "the .page-sub must exist"
|
||||
text = " ".join(sub.group(1).split())
|
||||
assert "stored in the database" in text, (
|
||||
"the sub states: the value is stored in the database"
|
||||
)
|
||||
assert "win over" in text, "the sub states: it wins over the env vars"
|
||||
assert "hands control back to the environment" in text, (
|
||||
"the sub states: clearing hands control back to the environment"
|
||||
)
|
||||
|
||||
|
||||
def test_settings_gate_is_the_tokens_gate_shape_ships_hidden() -> None:
|
||||
"""The anonymous sign-in gate: the EXACT #tokens-gate shape —
|
||||
the .sources-gate visual language, ship-hidden, its heading
|
||||
naming what is admin-only, and its Sign in returning to the
|
||||
Settings view (?next=/settings.html — the no-JS fallback)."""
|
||||
html = _html()
|
||||
view = html.find('<section class="view" id="view-settings"')
|
||||
main_end = html.find("</main>", view)
|
||||
body = html[view:main_end]
|
||||
gate = re.search(r'<section[^>]*id="settings-gate"[^>]*>', body)
|
||||
assert gate, "the #settings-gate must exist"
|
||||
assert "hidden" in gate.group(0), "#settings-gate must ship hidden"
|
||||
assert 'class="sources-gate"' in gate.group(0), (
|
||||
"the gate reuses the .sources-gate visual language"
|
||||
)
|
||||
assert 'aria-labelledby="settings-gate-title"' in gate.group(0)
|
||||
assert "<h2 id=\"settings-gate-title\">Sign in to change settings</h2>" in body
|
||||
assert 'href="/login.html?next=/settings.html"' in body, (
|
||||
"the gate's Sign in returns to the Settings view (no-JS fallback)"
|
||||
)
|
||||
|
||||
|
||||
def test_settings_content_scaffold_and_feedback_regions() -> None:
|
||||
"""The ship-hidden #settings-content holds, in order: the
|
||||
#settings-disabled kill-switch banner (role=alert, ship-hidden —
|
||||
D6: the mechanism is off, not the panel), the ONE status region
|
||||
(role=status, aria-live=polite — the #tokens-status precedent),
|
||||
the role=alert error line (ship-hidden — the #theme-error
|
||||
precedent), the search row (label + type=search input), the
|
||||
.table-wrap catalogue (the visually-hidden caption + the
|
||||
four-column thead with the visually-hidden Actions header), and
|
||||
the action row (Save — ships DISABLED — Clear all + the
|
||||
aria-hidden dirty hint)."""
|
||||
html = _html()
|
||||
view = html.find('<section class="view" id="view-settings"')
|
||||
main_end = html.find("</main>", view)
|
||||
body = html[view:main_end]
|
||||
content = re.search(r'<div[^>]*id="settings-content"[^>]*>', body)
|
||||
assert content and "hidden" in content.group(0), (
|
||||
"#settings-content must ship hidden (anonymous-safe)"
|
||||
)
|
||||
banner = re.search(
|
||||
r'<div[^>]*class="kb-banner is-error"[^>]*id="settings-disabled"[^>]*>'
|
||||
r"|<div[^>]*id=\"settings-disabled\"[^>]*>",
|
||||
body,
|
||||
)
|
||||
assert banner, "the #settings-disabled banner must exist"
|
||||
btag_match = re.search(r'<div[^>]*id="settings-disabled"[^>]*>', body)
|
||||
assert btag_match, "the #settings-disabled banner tag must match"
|
||||
btag = btag_match.group(0)
|
||||
assert 'role="alert"' in btag and "hidden" in btag, (
|
||||
"the banner is a ship-hidden role=alert"
|
||||
)
|
||||
status = re.search(
|
||||
r'<span[^>]*id="settings-status"[^>]*>', body
|
||||
)
|
||||
assert status, "the #settings-status region must exist"
|
||||
stag = status.group(0)
|
||||
assert 'role="status"' in stag and 'aria-live="polite"' in stag, (
|
||||
"the ONE status region (the #tokens-status precedent)"
|
||||
)
|
||||
error = re.search(r'<p[^>]*id="settings-error"[^>]*>', body)
|
||||
assert error, "the #settings-error line must exist"
|
||||
etag = error.group(0)
|
||||
assert 'role="alert"' in etag and "hidden" in etag, (
|
||||
"the error line is a ship-hidden role=alert"
|
||||
)
|
||||
# The search row: the label + the type=search input (client-side,
|
||||
# the phase-101 D4 contract — no fetch).
|
||||
assert '<label for="settings-filter">Search settings</label>' in body
|
||||
filter_input = re.search(
|
||||
r'<input[^>]*id="settings-filter"[^>]*>', body, re.S
|
||||
)
|
||||
assert filter_input, "the #settings-filter input must exist"
|
||||
assert 'type="search"' in filter_input.group(0), (
|
||||
"the filter is a type=search input"
|
||||
)
|
||||
# The scroll card: the .table-wrap + the visually-hidden caption
|
||||
# + the four-column thead (the Actions header visually-hidden —
|
||||
# the row buttons carry their own aria-labels).
|
||||
assert 'class="table-wrap"' in body, (
|
||||
"the catalogue rides the .table-wrap card (the phase-156 "
|
||||
"bounded scroll + sticky header contract)"
|
||||
)
|
||||
table = re.search(r'<table[^>]*class="settings-table"[^>]*>', body)
|
||||
assert table, "the .settings-table must exist"
|
||||
thead = body[body.find('class="settings-table"') : body.find("</thead>")]
|
||||
assert '<caption class="visually-hidden">' in thead, (
|
||||
"the caption is visually-hidden (the phase-86 table pattern)"
|
||||
)
|
||||
assert thead.count('<th scope="col">') == 4, (
|
||||
"the table is four columns (Setting | Value | Source | Actions)"
|
||||
)
|
||||
assert '<th scope="col">Setting</th>' in thead
|
||||
assert '<th scope="col">Value</th>' in thead
|
||||
assert '<th scope="col">Source</th>' in thead
|
||||
assert (
|
||||
'<th scope="col"><span class="visually-hidden">Actions</span></th>'
|
||||
in thead
|
||||
), "the Actions header is visually-hidden (the row buttons are labelled)"
|
||||
# The action row: Save (type=button, ships DISABLED — no dirty
|
||||
# rows yet), Clear all (type=button), the aria-hidden dirty hint.
|
||||
save = re.search(r'<button[^>]*id="settings-save"[^>]*>', body)
|
||||
assert save, "the #settings-save button must exist"
|
||||
stag = save.group(0)
|
||||
assert 'type="button"' in stag and "disabled" in stag, (
|
||||
"Save is a type=button that ships disabled (no dirty rows)"
|
||||
)
|
||||
clear_all = re.search(r'<button[^>]*id="settings-clear-all"[^>]*>', body)
|
||||
assert clear_all, "the #settings-clear-all button must exist"
|
||||
assert 'type="button"' in clear_all.group(0)
|
||||
assert "Clear all overrides" in body
|
||||
hint = re.search(r'<span[^>]*id="settings-dirty"[^>]*>', body)
|
||||
assert hint, "the #settings-dirty hint must exist"
|
||||
assert 'aria-hidden="true"' in hint.group(0), (
|
||||
"the hint is aria-hidden (the rows + the status carry the info)"
|
||||
)
|
||||
|
||||
|
||||
def test_settings_nav_link_ships_hidden_in_the_shell() -> None:
|
||||
"""The #nav-settings link (the shell's eighth view) ships hidden
|
||||
(admin-only) and UNstamped — the router is the single writer of
|
||||
the active state, and a token user (role "user") must never see
|
||||
the link (header.js reveals it for admin only)."""
|
||||
html = _html()
|
||||
match = re.search(r'<a[^>]*id="nav-settings"[^>]*>', html)
|
||||
assert match, "the shell must carry the #nav-settings nav link"
|
||||
link = match.group(0)
|
||||
assert 'href="/settings.html"' in link, "the Settings link's href"
|
||||
assert "hidden" in link, "#nav-settings ships hidden (admin-only)"
|
||||
assert "is-active" not in link, "no static active stamp on the Settings link"
|
||||
|
||||
|
||||
def test_nav_settings_sits_after_nav_theme() -> None:
|
||||
"""The one-bar contract: the Settings link is the LAST admin link
|
||||
(after #nav-theme) — the navbar order is part of the
|
||||
test_nav_consistency inventory, so the position is pinned."""
|
||||
html = _html()
|
||||
theme_i = html.find('id="nav-theme"')
|
||||
settings_i = html.find('id="nav-settings"')
|
||||
sign_in_i = html.find('id="sign-in-link-mobile"')
|
||||
assert 0 <= theme_i < settings_i < sign_in_i, (
|
||||
"the Settings link sits after Theme and before the auth links"
|
||||
)
|
||||
|
||||
|
||||
# ---------- router.js: /settings.html in ALL THREE tables ----------
|
||||
|
||||
|
||||
def test_router_js_maps_settings_in_all_three_tables() -> None:
|
||||
"""router.js: VIEW maps /settings.html → settings, VIEW_PATH maps
|
||||
settings → /settings.html, and VIEW_MODULES lazy-imports
|
||||
./settings.js with a STATIC specifier (so the Containerfile's
|
||||
esbuild --bundle of router.js inlines it — settings.js needs no
|
||||
new esbuild line because no page references it directly;
|
||||
tests/integration/test_containerfile_assets.py proves the
|
||||
coverage). The router is the SINGLE writer of the per-view
|
||||
title/meta — settings has both entries."""
|
||||
js = _read(ASSETS / "router.js")
|
||||
view_start = js.find("const VIEW = {")
|
||||
view_body = js[view_start : js.find("\n}", view_start)]
|
||||
assert '"/settings.html": "settings"' in view_body, (
|
||||
"phase 168 task 05 folds the Settings view into the shell"
|
||||
)
|
||||
path_start = js.find("const VIEW_PATH = {")
|
||||
path_body = js[path_start : js.find("\n}", path_start)]
|
||||
assert 'settings: "/settings.html"' in path_body, (
|
||||
"VIEW_PATH must map settings → /settings.html"
|
||||
)
|
||||
mods_start = js.find("const VIEW_MODULES = {")
|
||||
mods_body = js[mods_start : js.find("\n}", mods_start)]
|
||||
assert 'settings: () => import("./settings.js")' in mods_body, (
|
||||
"the Settings view module is lazy-imported on first show "
|
||||
"(a static specifier — the bundle inlines it)"
|
||||
)
|
||||
# The per-view head values (the router's single-writer contract).
|
||||
assert 'settings: "Settings · Brain of Reese"' in js, (
|
||||
"the Settings view's title (carried through brandName)"
|
||||
)
|
||||
assert (
|
||||
"Edit any application setting — a saved value wins over the environment, "
|
||||
"live on save; reset hands control back."
|
||||
) in js, "the Settings view's meta description"
|
||||
|
||||
|
||||
# ---------- the server-side registration (shell route + caching) ----------
|
||||
|
||||
|
||||
def test_shell_route_and_html_pages_carry_settings() -> None:
|
||||
"""app/main.py's _shell_routes tuple and caching.HTML_PAGES both
|
||||
list /settings.html — without the tuple the deep link 404s
|
||||
(the static mount has no settings.html file), without the
|
||||
HTML_PAGES entry it would serve unversioned asset refs, which
|
||||
the immutable-for-a-year asset caching would pin to stale CSS
|
||||
after a deploy (the phase-35 lesson, the A19 contract)."""
|
||||
main_py = _read(REPO / "app" / "main.py")
|
||||
app_start = main_py.find("def create_app")
|
||||
mount_i = main_py.find('app.mount("/", StaticFiles')
|
||||
assert 0 <= app_start < mount_i, "create_app + the static mount must exist"
|
||||
routes_block = main_py[app_start:mount_i]
|
||||
assert '"/settings.html"' in routes_block, (
|
||||
"main.py's shell tuple must list /settings.html (deep link)"
|
||||
)
|
||||
caching_py = _read(REPO / "app" / "core" / "caching.py")
|
||||
pages_start = caching_py.find("HTML_PAGES: tuple")
|
||||
pages_end = caching_py.find("\n)\n", pages_start)
|
||||
pages_block = caching_py[pages_start:pages_end]
|
||||
assert '"/settings.html"' in pages_block, (
|
||||
"caching.HTML_PAGES must list /settings.html (no-cache + ?v=)"
|
||||
)
|
||||
# The shell route comes AFTER the API router registration and
|
||||
# BEFORE the static mount (routes-first — the docstring contract).
|
||||
include_i = main_py.find("app.include_router(admin_settings_router")
|
||||
assert 0 <= app_start < include_i < mount_i, (
|
||||
"the shell route sits between the API routers and the static mount"
|
||||
)
|
||||
|
||||
|
||||
# ---------- header.js: the reveal is admin-keyed, null-safe ----------
|
||||
|
||||
|
||||
def test_header_js_reveals_settings_link_for_admin_only() -> None:
|
||||
"""header.js reveals #nav-settings for role admin — the same
|
||||
ship-hidden / reveal-for-admin contract as the other admin-only
|
||||
links: the two-line reveal (`hidden = !admin`) sits in
|
||||
initSharedHeader, null-safe (a page without the link is a
|
||||
no-op), and the gate is the `admin` flag (role === "admin") —
|
||||
a token user (role "user") never sees the link. The lookup +
|
||||
reveal sit AFTER the whoami resolution (the admin flag exists
|
||||
only once fetchWhoami has settled)."""
|
||||
header_js = _read(ASSETS / "header.js")
|
||||
fn = header_js.find("export async function initSharedHeader")
|
||||
assert fn != -1, "initSharedHeader must exist"
|
||||
body = header_js[fn:]
|
||||
lookup = body.find('document.querySelector("#nav-settings")')
|
||||
assert lookup != -1, "header.js must look up #nav-settings"
|
||||
reveal = body.find("navSettings.hidden = !admin")
|
||||
assert 0 <= lookup < reveal, (
|
||||
"the reveal must be the two-line pattern: null-safe lookup, "
|
||||
'then hidden = !admin (the admin flag — role === "admin")'
|
||||
)
|
||||
whoami = body.find("const whoami = await fetchWhoami()")
|
||||
admin_flag = body.find('const admin = whoami.role === "admin"')
|
||||
assert 0 <= whoami < admin_flag < lookup, (
|
||||
"the reveal keys off the resolved admin flag"
|
||||
)
|
||||
# The gate is `admin`, NOT `signedIn` (a token user never sees
|
||||
# it — the phase-79 contract).
|
||||
reveal_line = body[lookup:reveal + 60]
|
||||
assert "signedIn" not in reveal_line, (
|
||||
"the reveal must key off admin (a token user never sees the link)"
|
||||
)
|
||||
|
||||
|
||||
# ---------- settings.js: DOM APIs only, no CDN ----------
|
||||
|
||||
|
||||
def test_settings_js_builds_dom_with_element_apis_only() -> None:
|
||||
"""settings.js follows the phase-76 view-module contract:
|
||||
`export async function mount(root)` is the entry, the whoami
|
||||
gate (fetchIsAdmin) runs in mount and the anonymous branch shows
|
||||
the gate + hides the content + RETURNS with NO
|
||||
/api/admin/settings request (the router 403s anonymous), and the
|
||||
bor:view-refresh listener is armed ONLY in the ADMIN branch
|
||||
(after the gate), gated on the started flag. The DOM is built
|
||||
with createElement + textContent — NO innerHTML (template
|
||||
literal or not: the XSS-safe-by-construction house rule) and NO
|
||||
http(s):// reference (rule 6 — everything is local; the fetch
|
||||
paths are same-origin)."""
|
||||
js = _read(ASSETS / "settings.js")
|
||||
assert "export async function mount(root)" in js, (
|
||||
"mount(root) must be the module's entry (the phase-76 fold)"
|
||||
)
|
||||
assert 'import { fetchIsAdmin } from "./header.js";' in js, (
|
||||
"the view imports ONLY the shared cached whoami promise"
|
||||
)
|
||||
mount_i = js.find("export async function mount(root)")
|
||||
gate_i = js.find("if (!(await fetchIsAdmin()))")
|
||||
assert 0 <= mount_i < gate_i, "the whoami gate must run in mount"
|
||||
branch = js[gate_i:js.find("return;", gate_i)]
|
||||
assert "fetch(" not in branch, (
|
||||
"the anonymous branch must not fetch anything"
|
||||
)
|
||||
assert "gateEl.hidden = false" in branch
|
||||
assert "contentEl.hidden = true" in branch
|
||||
# The re-show refresh: armed in the ADMIN branch only (after the
|
||||
# gate), gated on the first load (started).
|
||||
listener = js.find('addEventListener("bor:view-refresh"')
|
||||
assert 0 <= gate_i < listener, (
|
||||
"the refresh listener is armed only in the ADMIN branch (after the gate)"
|
||||
)
|
||||
assert re.search(r"if \(started\) void loadSettings\(\)", js), (
|
||||
"the listener is gated on the first load (started)"
|
||||
)
|
||||
# No HTML-string DOM building (the house rule) — innerHTML is
|
||||
# absent from the file entirely (a template-literal innerHTML
|
||||
# would also match this check).
|
||||
assert "innerHTML" not in js, (
|
||||
"the DOM is built with createElement + textContent — no "
|
||||
"innerHTML anywhere (XSS-safe by construction)"
|
||||
)
|
||||
# No CDN (rule 6): no http(s):// reference in the module — the
|
||||
# fetch paths are same-origin (/api/admin/settings…).
|
||||
assert not re.search(r"https?://", js), (
|
||||
"no CDN — every reference is local (AGENTS.md rule 6)"
|
||||
)
|
||||
# The API surface: GET the catalogue, PUT one value, DELETE one /
|
||||
# DELETE all (the phase-168 API contract, task 04).
|
||||
assert 'fetch("/api/admin/settings")' in js, "the catalogue GET"
|
||||
assert "fetch(`/api/admin/settings/${key}`" in js, (
|
||||
"the per-key PUT / DELETE"
|
||||
)
|
||||
# The §7.4 lifecycle: the save button is aria-busy + labelled
|
||||
# while in flight, re-enabled in the same path (success AND
|
||||
# failure — the saveInFlight guard).
|
||||
assert 'setAttribute("aria-busy", "true")' in js
|
||||
assert "removeAttribute(\"aria-busy\")" in js
|
||||
assert 'textContent = "Saving…"' in js
|
||||
|
||||
|
||||
def test_settings_js_dirty_tracking_and_stop_at_first_failure() -> None:
|
||||
"""The K4 contract, pinned at source level: the dirty state is a
|
||||
module Map (key → value); markDirty compares the control value
|
||||
against the SERVED value (a masked field's served value is "" —
|
||||
an empty input means "leave unchanged", never an accidental
|
||||
wipe); Save walks the dirty rows IN DOM ORDER, PUTs { value },
|
||||
and breaks at the first failure (the 422 detail lands in the
|
||||
role=alert region, the load re-applies the remaining edits,
|
||||
focus goes to the offending input); a successful save lands the
|
||||
`Saved N settings.` status line + the restart-required
|
||||
suffix."""
|
||||
js = _read(ASSETS / "settings.js")
|
||||
assert "const dirty = new Map();" in js, (
|
||||
"the dirty state is a module Map"
|
||||
)
|
||||
mark = js.find("function markDirty(")
|
||||
mark_body = js[mark:js.find("\n }", mark)]
|
||||
assert 'item.secret ? "" : item.value ?? ""' in mark_body, (
|
||||
"the served value is '' for a masked field (never an accidental wipe)"
|
||||
)
|
||||
save = js.find("async function saveSettings()")
|
||||
save_body = js[save:js.find("\n }", save)]
|
||||
assert "for (const row of dataRows())" in save_body, (
|
||||
"Save walks the dirty rows in DOM order"
|
||||
)
|
||||
put_i = save_body.find("JSON.stringify({ value })")
|
||||
break_i = save_body.find("break; // stop at the first failure (K4)")
|
||||
assert 0 <= put_i < break_i, "the loop breaks at the first failure (K4)"
|
||||
assert "showError(" in save_body, "the 422 detail lands in the error region"
|
||||
assert "input.focus()" in save_body, (
|
||||
"focus goes to the offending input (K4)"
|
||||
)
|
||||
assert '`Saved ${saved.length} settings.`' in save_body, (
|
||||
"the one status line (role=status)"
|
||||
)
|
||||
assert "applies on the next restart" in save_body, (
|
||||
"a saved restart-required row says so (the owner is never "
|
||||
"told 'saved' and then sees nothing change)"
|
||||
)
|
||||
# The reset + clear-all contract: the DELETE one/all + the
|
||||
# settled status lines.
|
||||
assert "`${envVar} is back to the environment value.`" in js, (
|
||||
"the per-row Reset status line"
|
||||
)
|
||||
assert "Cleared ${cleared} overrides." in js, (
|
||||
"the Clear-all status line (N from the response)"
|
||||
)
|
||||
# The two-step confirm (the house history-confirm-* classes,
|
||||
# focus to Yes, Escape / No cancels).
|
||||
assert 'label.textContent = "Clear all overrides?"' in js
|
||||
assert 'yes.className = "history-confirm-yes"' in js
|
||||
assert 'no.className = "history-confirm-no"' in js
|
||||
assert "yes.focus()" in js, "focus moves to Yes (keyboard confirm)"
|
||||
assert "e.key === \"Escape\"" in js, "Escape cancels (the task contract)"
|
||||
|
||||
|
||||
# ---------- styles.css: the settings block, no new :root tokens ----------
|
||||
|
||||
|
||||
def test_styles_css_settings_block_uses_existing_tokens_only() -> None:
|
||||
"""styles.css declares the .settings-table (full width — the
|
||||
.git-sources-table language, no skinny list) + the three
|
||||
.settings-src.is-* variants (SOLID existing tokens — B3) + the
|
||||
dirty-row marker, and adds NO new :root variable (the phase-156
|
||||
B3 rule — the badges are text, the color is decoration; a themed
|
||||
page's #bor-theme override of the token handles the color)."""
|
||||
css = _read(ASSETS / "styles.css")
|
||||
assert "---------- Settings view (phase 168) ----------" in css, (
|
||||
"the Settings view block must exist"
|
||||
)
|
||||
block = re.search(r"\.settings-table \{([\s\S]*?)\n\}", css)
|
||||
assert block, "styles.css must style .settings-table"
|
||||
assert "width: 100%" in block.group(1), (
|
||||
"the catalogue is full width (no skinny list — PLAN §7.1)"
|
||||
)
|
||||
for variant, bg in (
|
||||
("is-db", "var(--brand-soft)"),
|
||||
("is-env", "var(--surface)"),
|
||||
("is-default", "var(--line)"),
|
||||
):
|
||||
rule = re.search(
|
||||
rf"\.settings-src\.{variant} \{{([\s\S]*?)\n\}}", css
|
||||
)
|
||||
assert rule, f"styles.css must style .settings-src.{variant}"
|
||||
body = rule.group(1)
|
||||
assert bg in body, f".settings-src.{variant} is the solid {bg} token (B3)"
|
||||
# The badge is TEXT (B5): the text is never color-alone — the
|
||||
# variant always pairs the background with an existing ink
|
||||
# token (never a color declaration without a text color).
|
||||
assert "color: var(--" in body, (
|
||||
f".settings-src.{variant} carries an existing ink token (text, not color alone)"
|
||||
)
|
||||
# The dirty-row marker (the --brand left border on the row).
|
||||
assert "settings-row.is-dirty" in css, (
|
||||
"the dirty-row marker must exist (unsaved survives grayscale)"
|
||||
)
|
||||
dirty = re.search(r"\.settings-row\.is-dirty[^{]*\{([\s\S]*?)\n\}", css)
|
||||
assert dirty and "var(--brand)" in dirty.group(1), (
|
||||
"the dirty marker is the --brand left border"
|
||||
)
|
||||
# No new :root variable (B3, the phase-156 rule): the settings
|
||||
# block must not declare a custom property — only var(--…)
|
||||
# references of the pre-existing tokens (a themed page's
|
||||
# #bor-theme override of the token handles the color).
|
||||
settings_block = css[css.find("Settings view (phase 168)"): css.find(
|
||||
"Shared page (phase 51"
|
||||
)]
|
||||
declarations = re.findall(r"^\s{2,}--[\w-]+\s*:", settings_block, re.M)
|
||||
assert not declarations, (
|
||||
f"the settings block must not declare :root variables (B3): "
|
||||
f"{declarations}"
|
||||
)
|
||||
# The touch targets: the ≥44px rules (the house language) + the
|
||||
# mobile 640px block stacks the search + action rows.
|
||||
for selector in ("#settings-filter", ".settings-control"):
|
||||
rule = re.search(rf"{re.escape(selector)} \{{([\s\S]*?)\n\}}", css)
|
||||
assert rule, f"styles.css must style {selector}"
|
||||
assert "min-height: 44px" in rule.group(1), f"{selector}: the ≥44px target"
|
||||
mobile = re.search(
|
||||
r"@media \(max-width: 640px\) \{([\s\S]*?)\n\}\n", css
|
||||
)
|
||||
assert mobile, "the 640px media query must exist"
|
||||
mbody = mobile.group(1)
|
||||
assert ".settings-search { flex-direction: column; align-items: stretch; }" in mbody
|
||||
assert "#settings-save { width: 100%; }" in mbody, (
|
||||
"the Save pill is full width on phones (the ≥44px target stands)"
|
||||
)
|
||||
|
||||
|
||||
# ---------- the four Theme-bearing pages carry the Settings link ----------
|
||||
|
||||
|
||||
def test_settings_link_ships_on_every_theme_bearing_page() -> None:
|
||||
"""The phase-34 one-bar contract — the SAME nav ships on every
|
||||
page (test_nav_consistency pins the header inventory PARITY
|
||||
across the shell pages, the document viewer, and the login
|
||||
page), so #nav-settings (ship-hidden, admin-only) must be in
|
||||
the #app-nav of EVERY header-bearing page: the shell +
|
||||
document.html + login.html + shared.html. The doc-edit flow
|
||||
page ships the reduced header (no admin links at all) and is
|
||||
out of the contract."""
|
||||
for page in (
|
||||
FRONTEND / "index.html",
|
||||
FRONTEND / "document.html",
|
||||
FRONTEND / "login.html",
|
||||
FRONTEND / "shared.html",
|
||||
):
|
||||
text = _read(page)
|
||||
# The page must carry BOTH links — the Theme link (the
|
||||
# phase-91 precedent) and the Settings link (this task).
|
||||
theme = re.search(r'<a[^>]*id="nav-theme"[^>]*>', text)
|
||||
assert theme, f"{page.name} must carry the #nav-theme nav link"
|
||||
match = re.search(r'<a[^>]*id="nav-settings"[^>]*>', text)
|
||||
assert match, (
|
||||
f"{page.name} must carry the #nav-settings nav link (one-bar)"
|
||||
)
|
||||
link = match.group(0)
|
||||
assert 'href="/settings.html"' in link, (
|
||||
f"{page.name}: the Settings link's href"
|
||||
)
|
||||
assert "hidden" in link, (
|
||||
f"{page.name}: #nav-settings ships hidden (admin-only)"
|
||||
)
|
||||
assert "is-active" not in link, (
|
||||
f"{page.name}: no static active stamp on the Settings link"
|
||||
)
|
||||
# The Settings link sits AFTER the Theme link (the order
|
||||
# test_nav_consistency's inventory compares).
|
||||
assert text.find('id="nav-theme"') < text.find('id="nav-settings"'), (
|
||||
f"{page.name}: the Settings link follows the Theme link"
|
||||
)
|
||||
@@ -0,0 +1,188 @@
|
||||
"""Phase 168 (task 03): Save re-points the two in-process singletons.
|
||||
|
||||
The "takes effect on Save" contract (D3): ``invalidate_settings()``
|
||||
clears the ``get_settings()`` cache (the ~70 per-use settings) AND runs
|
||||
the registered reset hooks — ``app.api.chat.reset_runtime_caches``
|
||||
(nulls the shared ``LLMClient`` and the chat concurrency semaphore,
|
||||
re-registered by ``create_app()`` at the composition root because
|
||||
``app/core`` must never import ``app/api``, PLAN §3). These pins are
|
||||
unit level: the chat globals are set directly, the overlay hooks are
|
||||
registered directly (the autouse fixture gives every test a clean hook
|
||||
list and restores every global it touches — a leaked ``LLMClient``
|
||||
holding a test settings snapshot would poison the next suite's
|
||||
singleton).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import app.main as app_main
|
||||
from app.api import chat as chat_api
|
||||
from app.config import get_settings, invalidate_settings
|
||||
from app.core import settings_overlay
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_state() -> Iterator[None]:
|
||||
"""Clean hook list + saved chat globals, restored on teardown.
|
||||
|
||||
``reset_state_for_tests()`` clears the process-wide hook list (a
|
||||
prior suite's test may have wiped the hook ``app.main`` registered
|
||||
at import — these tests register their own); the ``chat_api``
|
||||
globals are saved/restored around the test so a built ``LLMClient``
|
||||
or semaphore never leaks into another test's singleton slot.
|
||||
"""
|
||||
settings_overlay.reset_state_for_tests()
|
||||
saved = {
|
||||
"_llm": chat_api._llm,
|
||||
"_chat_semaphore": chat_api._chat_semaphore,
|
||||
"_chat_active": chat_api._chat_active,
|
||||
}
|
||||
yield
|
||||
chat_api._llm = saved["_llm"]
|
||||
chat_api._chat_semaphore = saved["_chat_semaphore"]
|
||||
chat_api._chat_active = saved["_chat_active"]
|
||||
settings_overlay.reset_state_for_tests()
|
||||
# monkeypatch restores BOR_* env first (finalized before this
|
||||
# fixture) — the cache clear re-resolves to the conftest-pinned
|
||||
# env-only view.
|
||||
get_settings.cache_clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# reset_runtime_caches — nulls exactly the two settings snapshots
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_reset_nulls_llm_and_semaphore() -> None:
|
||||
chat_api._llm = object()
|
||||
chat_api._chat_semaphore = object()
|
||||
chat_api._chat_active = 7
|
||||
chat_api.reset_runtime_caches()
|
||||
assert chat_api._llm is None
|
||||
assert chat_api._chat_semaphore is None
|
||||
# _chat_active counts live turns — a reset must not touch it (the
|
||||
# new chat_max_concurrent applies as permits are taken).
|
||||
assert chat_api._chat_active == 7
|
||||
|
||||
|
||||
def test_reset_is_idempotent() -> None:
|
||||
chat_api.reset_runtime_caches()
|
||||
chat_api.reset_runtime_caches() # a double Save must not raise
|
||||
assert chat_api._llm is None
|
||||
assert chat_api._chat_semaphore is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# get_llm / _get_chat_semaphore — lazy rebuild from the CURRENT settings
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_get_llm_singleton_contract() -> None:
|
||||
chat_api._llm = None
|
||||
first = chat_api.get_llm()
|
||||
second = chat_api.get_llm()
|
||||
# The singleton contract still holds — the same instance until a
|
||||
# reset (the pin that stops an "always construct" shortcut that
|
||||
# would break connection pooling).
|
||||
assert first is second
|
||||
|
||||
|
||||
def test_get_llm_rebuilds_after_reset() -> None:
|
||||
chat_api._llm = None
|
||||
first = chat_api.get_llm()
|
||||
chat_api.reset_runtime_caches()
|
||||
second = chat_api.get_llm()
|
||||
assert second is not first
|
||||
assert second.settings is get_settings()
|
||||
|
||||
|
||||
def test_get_chat_semaphore_rebuilds_after_reset(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
chat_api._chat_semaphore = None
|
||||
base = chat_api._get_chat_semaphore()
|
||||
base_value = base._value # pyright: ignore[reportPrivateUsage]
|
||||
# Drive the cap to a value different from the current one (the
|
||||
# env is the base — the kill switch is off, conftest pin), so the
|
||||
# rebuild's new _value is unambiguous.
|
||||
target = 3 if base_value != 3 else 5
|
||||
monkeypatch.setenv("BOR_CHAT_MAX_CONCURRENT", str(target))
|
||||
get_settings.cache_clear()
|
||||
chat_api.reset_runtime_caches()
|
||||
rebuilt = chat_api._get_chat_semaphore()
|
||||
assert rebuilt is not base
|
||||
assert rebuilt._value == target # pyright: ignore[reportPrivateUsage]
|
||||
# restore: the env reverts via monkeypatch, and the fixture's cache
|
||||
# clear re-resolves the settings.
|
||||
get_settings.cache_clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the create_app() wiring — the composition root registers the hook
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_create_app_wires_the_reset_hook() -> None:
|
||||
"""``register_invalidator(reset_runtime_caches)`` inside
|
||||
``create_app`` — the static pin that stops the wiring being
|
||||
"cleaned up" (without it a Save silently stops retuning the LLM
|
||||
client)."""
|
||||
src = Path(app_main.__file__).read_text()
|
||||
start = src.index("def create_app(")
|
||||
end = src.index("\napp = create_app()", start)
|
||||
body = src[start:end]
|
||||
assert "register_invalidator(reset_runtime_caches)" in body
|
||||
|
||||
|
||||
def test_app_core_imports_no_app_api() -> None:
|
||||
"""Dependency direction (PLAN §3): ``app/core/`` never imports an
|
||||
``app.api`` symbol — the hook exists so the composition root can
|
||||
wire the reset without inverting it."""
|
||||
core = Path(app_main.__file__).resolve().parent / "core"
|
||||
for py in sorted(core.glob("*.py")):
|
||||
src = py.read_text()
|
||||
for line in src.splitlines():
|
||||
stripped = line.strip()
|
||||
assert not (
|
||||
stripped.startswith("from app.api")
|
||||
or stripped.startswith("import app.api")
|
||||
), f"{py.name} imports app.api (PLAN §3 direction)"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# invalidate_settings — cache clear + hook run, a failed reset stands
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_invalidate_runs_hooks_and_re_resolves(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
calls: list[str] = []
|
||||
settings_overlay.register_invalidator(lambda: calls.append("hook"))
|
||||
get_settings.cache_clear()
|
||||
get_settings() # a cached hit under the ORIGINAL env
|
||||
monkeypatch.setenv("BOR_APP_NAME", "Live-Apply-Check")
|
||||
invalidate_settings() # clears the cache + runs the hooks
|
||||
assert calls == ["hook"]
|
||||
# the re-resolution sees the NEW env — the cache really was cleared
|
||||
assert get_settings().app_name == "Live-Apply-Check"
|
||||
|
||||
|
||||
def test_invalidate_survives_a_raising_hook(
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""A failed reset must not turn a committed save into a 500 — the
|
||||
exception is logged, never propagated."""
|
||||
|
||||
def _boom() -> None:
|
||||
raise RuntimeError("boom")
|
||||
|
||||
settings_overlay.register_invalidator(_boom)
|
||||
with caplog.at_level(logging.ERROR, logger="app.settings"):
|
||||
invalidate_settings() # must not raise
|
||||
assert any("settings invalidator failed" in r.getMessage() for r in caplog.records)
|
||||
@@ -0,0 +1,427 @@
|
||||
"""Phase 168 (task 02): the DB-over-env overlay resolver.
|
||||
|
||||
Pure/unit level — no live DB: the ``load_overrides`` failure paths use
|
||||
a stubbed engine, the kill switch is flipped via ``os.environ`` (it is
|
||||
read per call), and the env base comes from the process environment
|
||||
(pinned by ``tests/conftest.py`` — ``BOR_SETTINGS_DB=0``).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from sqlalchemy.exc import OperationalError, ProgrammingError
|
||||
|
||||
from app.config import Settings
|
||||
from app.core import settings_overlay
|
||||
from app.core.settings_overlay import (
|
||||
_coerce,
|
||||
enabled,
|
||||
load_overrides,
|
||||
merge,
|
||||
register_invalidator,
|
||||
reset_state_for_tests,
|
||||
resolve_settings,
|
||||
run_invalidators,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_state() -> None:
|
||||
reset_state_for_tests()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# enabled() — the BOR_SETTINGS_DB kill switch (D6)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_enabled_matrix(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.delenv("BOR_SETTINGS_DB", raising=False)
|
||||
assert enabled() is True
|
||||
for value in ("0", "false", "no", "off"):
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", value)
|
||||
assert enabled() is False, value
|
||||
for value in ("1", "true"):
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", value)
|
||||
assert enabled() is True, value
|
||||
# read per call — a flip is visible without any cache clear
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "off")
|
||||
assert enabled() is False
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
assert enabled() is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _coerce — env-native strings; the one list[str] field is JSON
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_coerce_scalars_are_raw_strings() -> None:
|
||||
# pydantic-settings lax-coerces on Settings() — the raw string goes
|
||||
# through exactly like the env var spelling.
|
||||
assert _coerce("relevance_threshold", "0.55") == "0.55"
|
||||
assert _coerce("llm_retries", "7") == "7"
|
||||
assert _coerce("stream_thinking", "false") == "false"
|
||||
assert _coerce("import_extensions", "md,txt") == "md,txt"
|
||||
|
||||
|
||||
def test_coerce_suggestions_json() -> None:
|
||||
assert set(settings_overlay.LIST_FIELDS) == {"suggestions"}
|
||||
assert _coerce("suggestions", '["a", "b"]') == ["a", "b"]
|
||||
with pytest.raises(ValueError):
|
||||
_coerce("suggestions", "not a json array")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# merge — layer rows over the env base, fail soft per key (LOCKED A5)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_merge_happy_path_env_base() -> None:
|
||||
base = Settings()
|
||||
merged, skipped = merge({"relevance_threshold": "0.55"})
|
||||
assert skipped == []
|
||||
assert merged.relevance_threshold == 0.55
|
||||
assert isinstance(merged.relevance_threshold, float)
|
||||
# everything else is byte-equal to the env base
|
||||
for name in Settings.model_fields:
|
||||
if name != "relevance_threshold":
|
||||
assert getattr(merged, name) == getattr(base, name), name
|
||||
|
||||
|
||||
def test_merge_skips_unknown_and_non_overridable() -> None:
|
||||
base = Settings()
|
||||
merged, skipped = merge(
|
||||
{
|
||||
"no_such_field": "x", # orphan row from a removed setting
|
||||
"database_url": base.database_url, # LOCKED A3 — read-only
|
||||
"embedding_dim": "512", # LOCKED A3 — read-only
|
||||
}
|
||||
)
|
||||
keys = {key for key, _ in skipped}
|
||||
assert keys == {"no_such_field", "database_url", "embedding_dim"}
|
||||
# the base values stand — a hand-inserted row never hijacks them
|
||||
assert merged.database_url == base.database_url
|
||||
assert merged.embedding_dim == base.embedding_dim
|
||||
assert merged.embedding_dim == 768
|
||||
|
||||
|
||||
def test_merge_drops_one_invalid_value_keeps_the_rest() -> None:
|
||||
base = Settings()
|
||||
merged, skipped = merge(
|
||||
{
|
||||
"relevance_threshold": "not-a-float",
|
||||
"llm_retries": "7",
|
||||
}
|
||||
)
|
||||
assert [key for key, _ in skipped] == ["relevance_threshold"]
|
||||
assert merged.llm_retries == 7
|
||||
# the bad row falls back to the env/base value
|
||||
assert merged.relevance_threshold == base.relevance_threshold
|
||||
|
||||
|
||||
def test_merge_malformed_suggestions_json_is_skipped_not_fatal() -> None:
|
||||
base = Settings()
|
||||
merged, skipped = merge({"suggestions": "definitely not json"})
|
||||
assert [key for key, _ in skipped] == ["suggestions"]
|
||||
assert merged.suggestions == base.suggestions
|
||||
assert isinstance(merged.suggestions, list)
|
||||
|
||||
|
||||
def test_merge_valid_suggestions_json_overrides() -> None:
|
||||
merged, skipped = merge({"suggestions": '["only this"]'})
|
||||
assert skipped == []
|
||||
assert merged.suggestions == ["only this"]
|
||||
|
||||
|
||||
def test_merge_cross_field_violation_skipped_not_raised(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setenv("BOR_RELEVANCE_THRESHOLD", "0.30")
|
||||
monkeypatch.setenv("BOR_LEXICAL_SUPPORT_FLOOR", "0.15")
|
||||
# floor 0.9 > threshold 0.55 — the validator's cross-field guard
|
||||
# (app/config.py) fires; the FLOOR row is dropped, the threshold
|
||||
# override stands, the floor falls back to the env value.
|
||||
merged, skipped = merge(
|
||||
{"relevance_threshold": "0.55", "lexical_support_floor": "0.9"}
|
||||
)
|
||||
keys = {key for key, _ in skipped}
|
||||
assert "lexical_support_floor" in keys
|
||||
assert merged.relevance_threshold == 0.55
|
||||
assert merged.lexical_support_floor == 0.15 # env fallback
|
||||
|
||||
|
||||
def test_merge_env_collision_cross_field_terminates(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
# The env floor (0.15) is above the DB-overridden threshold (0.1):
|
||||
# the validator blames the FLOOR, which has no override — its value
|
||||
# comes from env and cannot be dropped. The resolver must find a
|
||||
# single override drop that heals the set (the threshold row)
|
||||
# instead of looping: one unresolvable collision costs the
|
||||
# colliding row, the rest of the table stands (LOCKED A5 fail-soft).
|
||||
monkeypatch.setenv("BOR_RELEVANCE_THRESHOLD", "0.30")
|
||||
monkeypatch.setenv("BOR_LEXICAL_SUPPORT_FLOOR", "0.15")
|
||||
base = Settings()
|
||||
merged, skipped = merge({"relevance_threshold": "0.1"})
|
||||
keys = {key for key, _ in skipped}
|
||||
assert "relevance_threshold" in keys
|
||||
assert merged.relevance_threshold == base.relevance_threshold
|
||||
assert merged.lexical_support_floor == base.lexical_support_floor
|
||||
|
||||
|
||||
def test_merge_bool_spellings_behave_like_env() -> None:
|
||||
merged_false, skipped = merge({"stream_thinking": "false"})
|
||||
assert skipped == []
|
||||
assert merged_false.stream_thinking is False
|
||||
merged_zero, _ = merge({"stream_thinking": "0"})
|
||||
assert merged_zero.stream_thinking is False
|
||||
# untouched fields keep the base value
|
||||
base = Settings()
|
||||
assert merged_false.llm_retries == base.llm_retries
|
||||
|
||||
|
||||
def test_merge_empty_rows_is_the_env_base() -> None:
|
||||
base = Settings()
|
||||
merged, skipped = merge({})
|
||||
assert skipped == []
|
||||
assert merged == base
|
||||
|
||||
|
||||
def test_load_overrides_success_path(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
class _Rows:
|
||||
def __iter__(self):
|
||||
return iter((("llm_retries", "9"), ("stream_thinking", "false")))
|
||||
|
||||
class _FakeConn:
|
||||
def execute(self, _stmt: object) -> _Rows:
|
||||
return _Rows()
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> bool:
|
||||
return False
|
||||
|
||||
class _FakeEngine:
|
||||
def connect(self) -> _FakeConn: # type: ignore[no-untyped-def]
|
||||
return _FakeConn()
|
||||
|
||||
monkeypatch.setattr(settings_overlay, "_get_engine", lambda: _FakeEngine())
|
||||
assert load_overrides() == {"llm_retries": "9", "stream_thinking": "false"}
|
||||
|
||||
|
||||
def test_get_engine_creates_once_and_caches() -> None:
|
||||
# create_engine is lazy (no connection at creation) — the second
|
||||
# call must return the SAME engine (the once-per-process engine).
|
||||
first = settings_overlay._get_engine()
|
||||
second = settings_overlay._get_engine()
|
||||
assert first is second
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# merge — the defensive fallback (a validator change in a later release
|
||||
# blames a field no override owns and no single drop heals the set: the
|
||||
# env base stands, logged)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_merge_unresolvable_collision_returns_base_logged(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
from typing import Any, cast
|
||||
|
||||
from pydantic import ValidationError
|
||||
|
||||
class _StubSettings:
|
||||
"""Raises like a broken candidate set; the final no-args call
|
||||
(the env-base fallback) succeeds."""
|
||||
|
||||
model_fields = Settings.model_fields # merge() pre-checks it
|
||||
calls = 0
|
||||
|
||||
def __init__(self, **_kwargs: object) -> None:
|
||||
_StubSettings.calls += 1
|
||||
if _StubSettings.calls < 3:
|
||||
# the runtime shape (msg/error keys) is wider than the
|
||||
# pyright stub's InitErrorDetails — cast it
|
||||
raise ValidationError.from_exception_data(
|
||||
"Settings",
|
||||
cast(
|
||||
Any,
|
||||
[
|
||||
{
|
||||
"type": "missing",
|
||||
"loc": ("lexical_support_floor",),
|
||||
"msg": "a later-release validator",
|
||||
"error": ValueError("a later-release validator"),
|
||||
"input_value": 0.35,
|
||||
}
|
||||
],
|
||||
),
|
||||
)
|
||||
|
||||
monkeypatch.setattr(settings_overlay, "Settings", _StubSettings)
|
||||
merged, skipped = merge({"relevance_threshold": "0.1"})
|
||||
assert merged is not None # the env base (the stub's final call)
|
||||
reasons = {key: reason for key, reason in skipped}
|
||||
assert "unresolvable cross-field collision" in (
|
||||
reasons.get("relevance_threshold", "")
|
||||
or reasons.get("lexical_support_floor", "")
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# load_overrides — fail-soft, once-per-process (K3 + D5)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _FailingEngine:
|
||||
"""A stub engine whose connect() raises the given exception."""
|
||||
|
||||
def __init__(self, exc: Exception) -> None:
|
||||
self._exc = exc
|
||||
|
||||
def connect(self) -> None: # type: ignore[no-untyped-def]
|
||||
raise self._exc
|
||||
|
||||
|
||||
def test_load_overrides_generic_failure_warns_once(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
monkeypatch.setattr(
|
||||
settings_overlay,
|
||||
"_get_engine",
|
||||
lambda: _FailingEngine(
|
||||
OperationalError("SELECT 1", {}, OSError("connection refused"))
|
||||
),
|
||||
)
|
||||
with caplog.at_level(logging.WARNING, logger="app.settings"):
|
||||
assert load_overrides() == {}
|
||||
assert load_overrides() == {} # the second miss is silent
|
||||
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
|
||||
assert len(warnings) == 1
|
||||
assert "settings overrides unavailable — env values in effect" in warnings[0].msg
|
||||
# the exception class is named, the value is not
|
||||
assert "OperationalError" in warnings[0].getMessage()
|
||||
|
||||
|
||||
def test_load_overrides_missing_table_debugs_once(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
monkeypatch.setattr(
|
||||
settings_overlay,
|
||||
"_get_engine",
|
||||
lambda: _FailingEngine(
|
||||
ProgrammingError(
|
||||
"SELECT key, value FROM app_settings",
|
||||
{},
|
||||
Exception('relation "app_settings" does not exist'),
|
||||
)
|
||||
),
|
||||
)
|
||||
with caplog.at_level(logging.DEBUG, logger="app.settings"):
|
||||
assert load_overrides() == {}
|
||||
assert load_overrides() == {} # the second miss is silent
|
||||
debugs = [
|
||||
r
|
||||
for r in caplog.records
|
||||
if r.levelno == logging.DEBUG and "app_settings" in r.getMessage()
|
||||
]
|
||||
assert len(debugs) == 1
|
||||
|
||||
|
||||
def test_reset_state_restores_the_warning(
|
||||
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
monkeypatch.setattr(
|
||||
settings_overlay,
|
||||
"_get_engine",
|
||||
lambda: _FailingEngine(
|
||||
OperationalError("SELECT 1", {}, OSError("connection refused"))
|
||||
),
|
||||
)
|
||||
with caplog.at_level(logging.WARNING, logger="app.settings"):
|
||||
load_overrides() # first miss: warns
|
||||
reset_state_for_tests() # clears the once-per-process flag
|
||||
load_overrides() # second miss: warns again
|
||||
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
|
||||
assert len(warnings) == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# resolve_settings — the whole overlay, recursion guard (D2)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_settings_disabled_returns_env_base(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "0")
|
||||
assert resolve_settings() == Settings()
|
||||
|
||||
|
||||
def test_resolve_settings_enabled_uses_the_rows(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
|
||||
monkeypatch.setattr(
|
||||
settings_overlay, "load_overrides", lambda: {"llm_retries": "9"}
|
||||
)
|
||||
settings = resolve_settings()
|
||||
assert settings.llm_retries == 9
|
||||
|
||||
|
||||
def test_resolve_settings_never_reaches_back_into_config() -> None:
|
||||
"""The recursion guard, source-pinned: the overlay builds
|
||||
``Settings()`` directly — app.config's resolver delegates HERE."""
|
||||
src = Path(settings_overlay.__file__).read_text()
|
||||
assert "get_settings(" not in src
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the invalidator registry (task 03's seam)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_invalidators_run_in_order_and_survive_a_raising_hook(
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
calls: list[str] = []
|
||||
|
||||
def _boom() -> None:
|
||||
raise RuntimeError("boom")
|
||||
|
||||
register_invalidator(lambda: calls.append("first"))
|
||||
register_invalidator(_boom)
|
||||
register_invalidator(lambda: calls.append("third"))
|
||||
with caplog.at_level(logging.ERROR, logger="app.settings"):
|
||||
run_invalidators() # must not raise
|
||||
assert calls == ["first", "third"]
|
||||
assert any(
|
||||
"settings invalidator failed" in r.getMessage() for r in caplog.records
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# LOCKED A1 — .env is never written by the app
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_phase_files_never_write_the_env_file() -> None:
|
||||
"""No ``open(`` / ``write_text(`` in the phase-168 modules — the
|
||||
panel writes Postgres only (LOCKED A1); ``app/api/admin_settings.py``
|
||||
(task 04) is pinned in the same way once it exists."""
|
||||
root = Path(settings_overlay.__file__).resolve().parents[2]
|
||||
for rel in (
|
||||
"app/core/settings_overlay.py",
|
||||
"app/config.py",
|
||||
"app/api/admin_settings.py",
|
||||
):
|
||||
path = root / rel
|
||||
if not path.exists():
|
||||
continue
|
||||
src = path.read_text()
|
||||
assert "open(" not in src, (
|
||||
f"{rel} must never open() a file (LOCKED A1: .env is never "
|
||||
f"written by the app)"
|
||||
)
|
||||
assert "write_text(" not in src, f"{rel} must never write a file (LOCKED A1)"
|
||||
Reference in New Issue
Block a user