**Phase 168 — Admin Settings panel — final verification: PASS** 1. **Implemented/verified:** `/settings.html` (admin-only) with the 75-field catalogue (`app/core/settings_catalog.py`), DB-over-env overrides in Postgres (`app_settings`, migration 0028 at head, `settings_overlay.py` resolver, `app/api/admin_settings.py` — 4 admin-only routes), live-apply on save with no restart + restart-required badges, Reset/Clear all, write-only secrets, `.env`-never-written, `BOR_SETTINGS_DB=0` kill switch. 2. **Phase E2E: 12/12** — run with `E2E_APP_PORT=8200` (ports 8123/8124 are held by an unrelated "undercooked" project; no process killed, no code change — the documented env override). 3. **Regression set: all 7 suites green in isolation** (theme tab, API tokens, configurable brand, global tuning, admin auth, cache busting, wide desktop column — 38 tests). 4. **Full suite: unit+integration exit 0 (4066 passed), app/ coverage 99%** (gate >90%); **ruff clean, pyright 0 errors / 0 warnings.** 5. **Log contract (rule 10) verified live:** per-turn + per-response byte-budget lines present; settings write lines carry key + action, **never the value**. 6. **Byte-budget audit: all endpoints under the 262,144 B budget**; settings catalogue 23,513 B (gzip 3,326). 7. **Defect fixed (one):** criterion 5 had no integration pin — added `test_bad_row_inserted_directly_never_bricks_boot` to `tests/integration/test_admin_settings_api.py` (malformed row inserted directly → fresh `create_app()` boots, row skipped with logged reason, rest of table applies, `/api/health` 200) — 12/12 green. 8. **Docs pass:** `README.md` and `AGENTS.md` reviewed in full — both consistent with the as-built feature (Settings page table, Settings section, Configuration-reference note, `.env.example` block); **no changes required**. 9. **Cleanup:** audit app killed (by PID); `app_settings` left empty (0 rows); all changes left in the working tree (no commit, per protocol). 10. **Completion criteria:** all 7 checkboxes verified green (criterion 7's commit is the harness's). 11. **Next pending phase:** `170_edit_bubble_sizing`.
1138 lines
51 KiB
Python
1138 lines
51 KiB
Python
"""Phase 168 E2E (Playwright): the admin Settings panel — the live-on-Save
|
|
contract, proven in a real browser.
|
|
|
|
Source: ``TODO.md`` L3 — "I want an admin panel that can edit anything in
|
|
.env and save it to the database. The database value should override the
|
|
env var. The change should take effect on clicking 'save' on that page."
|
|
|
|
Run in isolation (DB must be up: ``podman compose up -d db``):
|
|
|
|
uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov
|
|
|
|
Test → contract mapping (one story, one phase, one isolated file):
|
|
|
|
1. ``test_anonymous_sees_gate_and_no_data`` — the gate: anonymous
|
|
meets ``#settings-gate`` (sign-in link ``?next=/settings.html``)
|
|
with ``#settings-content`` hidden, ``#nav-settings`` hidden, and
|
|
``GET /api/admin/settings`` 403s — no setting label or value
|
|
ever reaches the served page.
|
|
2. ``test_token_user_never_sees_the_panel`` — the phase-79 B5
|
|
contract: a token user (issued by the admin, driven through the
|
|
real token gate) never sees ``#nav-settings``, meets the gate,
|
|
and 403s the API.
|
|
3. ``test_admin_sees_groups_and_source_badges`` — the catalogue:
|
|
the groups render (``Retrieval & honesty`` / ``LLM (aipi)``),
|
|
the ``.table-wrap`` card carries the bounded-scroll + sticky-
|
|
``thead`` contract, ``input_placeholder`` shows the served env
|
|
value with the ``env`` badge, ``top_n_docs`` the code default
|
|
with the ``default`` badge, ``llm_api_key`` renders a password
|
|
input (empty value, the "stored" placeholder) and
|
|
``database_url`` / ``embedding_dim`` render read-only (masked
|
|
empty password / disabled read-only Reset — the URL's
|
|
credentials never reach the DOM).
|
|
4. ``test_save_takes_effect_without_restart`` — the TODO's own
|
|
acceptance test: a value saved through the panel shows up in the
|
|
running app in the same browser session with no restart (the
|
|
composer's ``placeholder`` attribute is the saved value), the
|
|
badge flips to ``DB``, the other rows stay at the env value
|
|
(only one row was saved) and the DB value survives a reload of
|
|
``/settings.html``.
|
|
5. ``test_saved_value_reaches_a_real_turn`` — the live read on the
|
|
retrieval path: ``relevance_threshold`` saved from the panel
|
|
flips a fixture turn's honesty verdict (the mock's
|
|
token-overlap cosine for "Tell me about basalt" is ~0.25 with
|
|
ZERO FTS hits, so the threshold bar is the ONLY gate leg —
|
|
saving 0.20 grounds, saving 0.27 deflects, ``query_log.
|
|
top_score`` is unchanged between the two turns, and the Reset
|
|
hands the pre-save (env 0.30) verdict back).
|
|
6. ``test_invalid_value_is_refused_and_nothing_changes`` — A5: a
|
|
batch with the invalid ``lexical_support_floor = "-0.5"`` is a
|
|
422 whose detail names the field in the ``role="alert"`` line
|
|
(the invalid value must be DOM-reachable — the ``float`` rows
|
|
are ``type="number"`` inputs, so the number IS typed; a
|
|
non-number cannot be retained by the browser, the server's
|
|
coercion 422 is a raw-API path); the second dirty row
|
|
(``source_usefulness_floor = "0.5"`` — a valid value that must
|
|
NOT be stored) keeps its typed value in the form (K4), both
|
|
badges stay ``env``, ``GET /api/admin/settings`` still reports
|
|
``source=env`` for both and the clean ``input_placeholder`` is
|
|
untouched server-side. (The batch walks the dirty rows in DOM
|
|
order and stops at the first failure — the floor row precedes
|
|
the usefulness row in the ``Retrieval & honesty`` group.)
|
|
7. ``test_cross_field_pair_is_refused`` — the cross-field rule
|
|
(A5/D5): with a DB-saved ``relevance_threshold = 0.2``, saving
|
|
``lexical_support_floor = 0.9`` 422s (the detail names the
|
|
field) and the stored pair never becomes effective (the
|
|
catalogue shows only the first saved value; ``overridden == 1``).
|
|
8. ``test_secret_never_reaches_the_browser`` — LOCKED A4: a
|
|
sentinel key stored through the API never appears in the
|
|
reloaded page, in the served HTML or in the raw
|
|
``GET /api/admin/settings`` body; the row's password input
|
|
shows the "stored" placeholder with a live Reset (the
|
|
``has_override`` render), and a panel-side save rotates the
|
|
key without ever painting it.
|
|
9. ``test_reset_hands_control_back_to_the_env`` — the Reset
|
|
deletes the row: the status line, the badge back to ``env``
|
|
and the footer on ``/`` reading the env value again.
|
|
10. ``test_clear_all_overrides`` — the two-step confirm: Escape
|
|
cancels (the overrides survive), confirming lands
|
|
``Cleared 3 overrides.`` and every badge falls back to
|
|
``env``/``default``.
|
|
11. ``test_disabled_switch_is_announced`` — D6 at the
|
|
SUBPROCESS level (a second app on its own port with
|
|
``BOR_SETTINGS_DB=0`` — the stronger variant): the
|
|
``#settings-disabled`` banner is visible, a save still STORES
|
|
the row but the app's behavior is unchanged (the footer on
|
|
``/`` stays the env value, the badge falls back to ``env``).
|
|
12. ``test_view_layout_and_accessibility`` — PLAN §7.1: no
|
|
horizontal page overflow at 375px and 1440px, the table is
|
|
≥80% of the 72rem container (not a skinny list), the
|
|
``restart required`` flag sits on ``db_pool_size``'s row, the
|
|
keyboard walk reaches filter → first input → (an enabled)
|
|
Reset → Save → Clear all in DOM order, and the whole test
|
|
issues ZERO external requests (rule 6).
|
|
|
|
DB isolation: the shared e2e Postgres keeps ``app_settings`` (the
|
|
overlay rows — a leftover row would retune every later suite's app
|
|
through the conftest pin's failure mode), ``ui_settings`` (the
|
|
single row the caching middleware reads for EVERY served page — the
|
|
effective-strings resolution ``/api/config`` uses) and
|
|
``api_tokens`` rows across suites. An autouse fixture truncates
|
|
``app_settings`` and ``ui_settings`` and deletes the ``e2e-``-
|
|
labeled tokens before AND after every test (never a TRUNCATE on
|
|
``api_tokens`` — the shared DB may hold the owner's real tokens);
|
|
the module teardown pins ``app_settings`` empty. The module app
|
|
opts BACK IN over the conftest's global ``BOR_SETTINGS_DB=0`` pin
|
|
(``env["BOR_SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py``
|
|
opt-in pattern) and pins the branding base to DISTINCTIVE values
|
|
(``Env placeholder for panel E2E`` / ``Env footer for panel E2E``)
|
|
so "the DB row now wins" is a visible DOM change, not a no-op
|
|
against the code default. A module-scope ``.env`` mtime guard pins
|
|
LOCKED A1: no code path the suite drives opens the env file for
|
|
writing.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
import threading
|
|
from collections.abc import Iterator
|
|
from typing import Any
|
|
|
|
import httpx
|
|
import pytest
|
|
from playwright.sync_api import Browser, BrowserContext, Page, expect
|
|
from sqlalchemy import select, text
|
|
|
|
from app.config import Settings
|
|
from app.db import SessionLocal
|
|
from app.models import QueryLog
|
|
from app.rag.importer import import_sources
|
|
from app.rag.llm import LLMClient
|
|
from e2e.auth_helpers import login, login_with_token
|
|
from e2e.conftest import (
|
|
ADMIN_PASSWORD,
|
|
APP_PORT,
|
|
REPO,
|
|
SESSION_SECRET,
|
|
USE_REAL_LLM,
|
|
_wait_http,
|
|
)
|
|
|
|
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
|
|
|
#: The kill-switch app (test 11) — a second, real app subprocess on
|
|
#: its own port with ``BOR_SETTINGS_DB=0`` (the stronger variant of
|
|
#: the task's subprocess-level option; the route-interception
|
|
#: fallback was not needed).
|
|
DISABLED_PORT = APP_PORT + 1
|
|
DISABLED_URL = f"http://127.0.0.1:{DISABLED_PORT}"
|
|
|
|
#: The distinctive env base (task 06): "the DB row now wins" must be
|
|
#: a VISIBLE DOM change, not a no-op against the code default.
|
|
ENV_PLACEHOLDER = "Env placeholder for panel E2E"
|
|
ENV_FOOTER = "Env footer for panel E2E"
|
|
|
|
#: The save's own acceptance value (test 4).
|
|
SAVED_PLACEHOLDER = "DB placeholder — panel saved this"
|
|
|
|
#: The sentinel secret (test 8) — a value that must appear NOWHERE
|
|
#: outside the DB.
|
|
SENTINEL_KEY = "sk-e2e-settings-sentinel-0123456789abcd"
|
|
ROTATED_KEY = "sk-e2e-settings-rotated-9876543210fedc"
|
|
|
|
#: Test 5 — the live-threshold question. The fixture corpus has NO
|
|
#: question whose mock cosine sits in [0.30, 0.31] (verified against
|
|
#: the real importer + mock embeddings: the mock's token-overlap
|
|
#: scores top out at ~0.61 for the on-topic battery), so the
|
|
#: task-pinned proxy is used: ``query_log.top_score`` is unchanged
|
|
#: while the deflected/grounded decision flips when the saved bar
|
|
#: moves above/below it. "Tell me about basalt" scores ~0.25 with
|
|
#: ZERO FTS hits (verified the same way) — with no lexical
|
|
#: corroboration the threshold bar is the ONLY gate leg. The bars
|
|
#: are derived from the measured top score (``top - 0.05`` /
|
|
#: ``top + 0.02``) so the flip is robust to the exact mock score.
|
|
BASELALT = "Tell me about basalt"
|
|
ENV_BAR = 0.30 # the app's env bar (the pre-save classification)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Per-module app env (the tuning/theme pattern — the task's exact line
|
|
# changes over the test_admin_theme_tab.py fixture)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _boot_app(port: int, settings_db: str) -> subprocess.Popen[bytes]:
|
|
"""One module app (the shared env block + the task's pins), on
|
|
``port`` with the overlay ``settings_db`` (``"1"`` / ``"0"``)."""
|
|
env = dict(os.environ)
|
|
env.pop("DEBUGPY", None)
|
|
env["BOR_ENVIRONMENT"] = "e2e"
|
|
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
|
env["BOR_LLM_BASE_URL"] = (
|
|
"https://aipi.reeseapps.com/v1"
|
|
if USE_REAL_LLM
|
|
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
|
|
)
|
|
# The E2E-calibrated gate pins (the conftest pattern — the mock's
|
|
# token-overlap distribution, explicit so an operator's local
|
|
# .env cannot leak a different gate into the suite's app).
|
|
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
|
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
|
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
|
env["BOR_LLM_RETRY_DELAY"] = "0"
|
|
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
|
|
env.setdefault(
|
|
"BOR_DATABASE_URL",
|
|
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
|
)
|
|
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
|
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
|
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
|
env["BOR_DOCS_REPO"] = ""
|
|
env["BOR_SUGGESTIONS"] = json.dumps(
|
|
Settings.model_fields["suggestions"].default
|
|
)
|
|
# The app name stays the CODE default (the theme-suite pin — the
|
|
# assertions below name it).
|
|
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
|
|
# Phase 168 (task 06): the module opts BACK IN over the conftest's
|
|
# global BOR_SETTINGS_DB=0 pin (the test_pictorial_pdf.py pattern)
|
|
# — or stays OUT for the kill-switch app (test 11).
|
|
env["BOR_SETTINGS_DB"] = settings_db
|
|
# The distinctive env base: "the DB row now wins" is a visible
|
|
# DOM change, not a no-op against the default.
|
|
env["BOR_INPUT_PLACEHOLDER"] = ENV_PLACEHOLDER
|
|
env["BOR_FOOTER_TEXT"] = ENV_FOOTER
|
|
env["BOR_GIT_SOURCES"] = ""
|
|
return subprocess.Popen(
|
|
[sys.executable, "-m", "uvicorn", "app.main:app",
|
|
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
|
|
cwd=REPO,
|
|
env=env,
|
|
)
|
|
|
|
|
|
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
|
|
proc.terminate()
|
|
try:
|
|
proc.wait(timeout=10)
|
|
except subprocess.TimeoutExpired:
|
|
proc.kill()
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def app_server(mock_llm: int) -> Iterator[str]:
|
|
"""The real app under test — per-module env: the branding vars
|
|
pinned to the DISTINCTIVE E2E values (the panel's saved value
|
|
must visibly WIN over them) and ``BOR_SETTINGS_DB=1`` (the
|
|
module opts back in over the conftest pin)."""
|
|
global _MOCK_PORT
|
|
_MOCK_PORT = mock_llm
|
|
proc = _boot_app(APP_PORT, "1")
|
|
try:
|
|
_wait_http(f"{APP_URL}/api/health")
|
|
yield APP_URL
|
|
finally:
|
|
_stop_app(proc)
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def app_url(app_server: str) -> str:
|
|
return app_server
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def disabled_app_server(mock_llm: int) -> Iterator[str]:
|
|
"""The kill-switch app (test 11): a second, real app subprocess
|
|
on its own port with ``BOR_SETTINGS_DB=0`` — the D6 contract
|
|
proven at the process level, not by route interception."""
|
|
global _MOCK_PORT
|
|
_MOCK_PORT = mock_llm
|
|
proc = _boot_app(DISABLED_PORT, "0")
|
|
try:
|
|
_wait_http(f"{DISABLED_URL}/api/health")
|
|
yield DISABLED_URL
|
|
finally:
|
|
_stop_app(proc)
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def disabled_app_url(disabled_app_server: str) -> str:
|
|
return disabled_app_server
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DB isolation + helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _clean_settings_state() -> None:
|
|
"""Fresh settings + theme + token state per test: truncate
|
|
``app_settings`` (a leftover row is the cross-suite poison this
|
|
phase's D6 exists to prevent) and the single-row
|
|
``ui_settings`` (the middleware reads it for EVERY page — the
|
|
effective-strings resolution ``/api/config`` uses), and delete
|
|
this suite's issued tokens (label-scoped on ``e2e-`` — never a
|
|
TRUNCATE: the shared DB may hold the owner's real tokens)."""
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE app_settings"))
|
|
db.execute(text("TRUNCATE ui_settings"))
|
|
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
|
|
db.commit()
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _clean(db_ready: None) -> Iterator[None]:
|
|
_clean_settings_state()
|
|
yield
|
|
_clean_settings_state()
|
|
# The module's final state: app_settings EMPTY (the D6 poison
|
|
# guard — the next suite's app must boot with no overrides).
|
|
with SessionLocal() as db:
|
|
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
|
|
|
|
|
|
@pytest.fixture(scope="module", autouse=True)
|
|
def _env_never_written() -> Iterator[None]:
|
|
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
|
|
``.env`` for writing (the file is gitignored — a ``git status``
|
|
check cannot see it, so ``stat``'s mtime is the assertion)."""
|
|
env_file = REPO / ".env"
|
|
before = env_file.stat().st_mtime_ns if env_file.exists() else None
|
|
yield
|
|
after = env_file.stat().st_mtime_ns if env_file.exists() else None
|
|
assert before == after, "the app under test must never write .env"
|
|
|
|
|
|
def _cookies(page: Page) -> dict[str, str]:
|
|
"""The session cookies the browser context holds (the test's API
|
|
side sees exactly what that browser sees)."""
|
|
return {
|
|
c["name"]: c["value"]
|
|
for c in page.context.cookies()
|
|
if "name" in c and "value" in c
|
|
}
|
|
|
|
|
|
def _admin_client(app_url: str) -> httpx.Client:
|
|
"""The API side of the form login (the 204 + the signed session
|
|
cookie in the jar)."""
|
|
client = httpx.Client(timeout=60.0)
|
|
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
|
|
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
|
|
return client
|
|
|
|
|
|
def _run_in_thread(coro: Any) -> Any:
|
|
"""Run a coroutine on a worker thread (Playwright's sync API
|
|
keeps an asyncio loop on the test thread — ``asyncio.run`` is
|
|
unavailable in the test body)."""
|
|
box: dict[str, Any] = {}
|
|
|
|
def runner() -> None:
|
|
try:
|
|
box["value"] = asyncio.run(coro)
|
|
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
|
|
box["error"] = e
|
|
|
|
t = threading.Thread(target=runner)
|
|
t.start()
|
|
t.join()
|
|
if "error" in box:
|
|
raise box["error"]
|
|
return box["value"]
|
|
|
|
|
|
def _import_fixtures(mock_port: int) -> Any:
|
|
"""Seed the shared DB with the fixture KB (the
|
|
test_honest_deflection pattern — the mock's genuine
|
|
token-overlap embeddings make the gate deterministic)."""
|
|
kwargs: dict[str, Any] = {
|
|
"_env_file": None,
|
|
"llm_base_url": f"http://127.0.0.1:{mock_port}/v1",
|
|
}
|
|
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
|
return import_sources([REPO / "tests" / "fixtures" / "docs"], LLMClient(settings))
|
|
|
|
|
|
def _reset_db(mock_port: int, seed: bool) -> None:
|
|
"""Truncate the KB (and query log), then optionally re-import
|
|
the fixtures."""
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE chunks, documents, query_log"))
|
|
db.commit()
|
|
if seed:
|
|
_run_in_thread(_import_fixtures(mock_port))
|
|
|
|
|
|
def _chat_done(client: httpx.Client, app_url: str, message: str) -> dict[str, Any]:
|
|
"""One chat turn, raw SSE (the test_honest_deflection pattern) —
|
|
returns the ``done`` frame (the server's own verdict, not the
|
|
model's phrasing)."""
|
|
frames: list[dict[str, Any]] = []
|
|
with client.stream(
|
|
"POST", f"{app_url}/api/chat", json={"message": message}, timeout=120.0
|
|
) as r:
|
|
assert r.status_code == 200, f"chat → {r.status_code}"
|
|
buf = ""
|
|
for part in r.iter_text():
|
|
buf += part
|
|
while "\n\n" in buf:
|
|
frame, buf = buf.split("\n\n", 1)
|
|
if frame.strip().startswith("data:"):
|
|
frames.append(
|
|
json.loads(frame.strip().removeprefix("data:").strip())
|
|
)
|
|
done = [f for f in frames if f.get("type") == "done"]
|
|
assert len(done) == 1, f"exactly one done frame (got {len(done)})"
|
|
return done[0]
|
|
|
|
|
|
def _last_query_log() -> QueryLog:
|
|
with SessionLocal() as db:
|
|
return (
|
|
db.scalars(
|
|
select(QueryLog)
|
|
.order_by(QueryLog.created_at.desc(), QueryLog.id.desc())
|
|
.limit(1)
|
|
)
|
|
.one()
|
|
)
|
|
|
|
|
|
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
|
|
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.status_code == 200, r.text[:200]
|
|
for item in r.json()["items"]:
|
|
if item["key"] == key:
|
|
return item
|
|
raise AssertionError(f"{key} missing from the catalogue")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. The gate: anonymous sees the gate and no data
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_anonymous_sees_gate_and_no_data(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
page.goto(app_url + "/settings.html")
|
|
|
|
# The shell serves the Settings view (the section is the visible
|
|
# surface — the gate inside it is what anonymous sees)…
|
|
expect(page.locator("#view-settings")).to_be_visible(timeout=15_000)
|
|
# …with the sign-in gate up (the link returns to the Settings
|
|
# view — the ?next= convention)…
|
|
expect(page.locator("#settings-gate")).to_be_visible(timeout=15_000)
|
|
expect(page.locator("#settings-gate a.sources-gate-link")).to_have_attribute(
|
|
"href", "/login.html?next=/settings.html"
|
|
)
|
|
# …the panel itself hidden and the admin-only nav link hidden.
|
|
expect(page.locator("#settings-content")).to_be_hidden()
|
|
expect(page.locator("#nav-settings")).to_be_hidden()
|
|
|
|
# The API agrees (no credentials anywhere — 403, the router-wide
|
|
# require_admin; 403 never 401).
|
|
r = httpx.get(f"{app_url}/api/admin/settings", timeout=10)
|
|
assert r.status_code == 403, f"anonymous GET /api/admin/settings → {r.status_code}"
|
|
|
|
# No setting label or value ever reaches the served page (the
|
|
# catalogue renders for the admin branch only — the gate is the
|
|
# entire anonymous surface).
|
|
content = page.content()
|
|
for label in ("Relevance threshold", "Input placeholder", "LLM API key", "DB pool size"):
|
|
assert label not in content, f"{label} leaked into the anonymous page"
|
|
for name in ("BOR_RELEVANCE_THRESHOLD", "BOR_LLM_API_KEY", "BOR_INPUT_PLACEHOLDER"):
|
|
assert name not in content, f"{name} leaked into the anonymous page"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. The phase-79 B5 wall: a token user never sees the panel
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_token_user_never_sees_the_panel(
|
|
page: Page, browser: Browser, app_url: str, db_ready: None
|
|
) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/")
|
|
r = httpx.post(
|
|
f"{app_url}/api/tokens",
|
|
json={"label": "e2e-settings-wall"},
|
|
cookies=_cookies(page),
|
|
timeout=10,
|
|
)
|
|
assert r.status_code == 201, r.text
|
|
token = r.json()["token"]
|
|
|
|
user_ctx: BrowserContext | None = None
|
|
try:
|
|
user_ctx = browser.new_context()
|
|
user = user_ctx.new_page()
|
|
user.set_default_timeout(30_000)
|
|
login_with_token(user, app_url, token, next="/")
|
|
# The token user's shell (role "user" — the header reveals
|
|
# the admin links only for role === "admin")…
|
|
expect(user.locator("#nav-settings")).to_be_hidden()
|
|
# …meets the gate on the Settings view with the panel hidden…
|
|
user.goto(app_url + "/settings.html")
|
|
expect(user.locator("#settings-gate")).to_be_visible(timeout=15_000)
|
|
expect(user.locator("#settings-content")).to_be_hidden()
|
|
# …and 403s the API (authenticated, just not an admin —
|
|
# 403, never 401) — GET and PUT.
|
|
get_status = user.evaluate(
|
|
"() => fetch('/api/admin/settings').then((r) => r.status)"
|
|
)
|
|
assert get_status == 403, f"token-user GET → {get_status}"
|
|
put_status = user.evaluate(
|
|
"""async () => (await fetch('/api/admin/settings/app_name', {
|
|
method: 'PUT',
|
|
headers: {'Content-Type': 'application/json'},
|
|
body: JSON.stringify({value: 'x'}),
|
|
})).status"""
|
|
)
|
|
assert put_status == 403, f"token-user PUT → {put_status}"
|
|
finally:
|
|
if user_ctx is not None:
|
|
user_ctx.close()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. The catalogue: the groups, the bounded card, the source badges
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_admin_sees_groups_and_source_badges(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#nav-settings")).to_be_visible(timeout=15_000)
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
expect(page.locator("#settings-gate")).to_be_hidden()
|
|
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
|
|
|
|
# The catalogue table is INSIDE the .table-wrap card (the
|
|
# phase-156 bounded-scroll + sticky-header contract, inherited —
|
|
# no per-table override).
|
|
assert page.locator(".table-wrap .settings-table").count() == 1
|
|
sticky = page.evaluate(
|
|
"() => getComputedStyle("
|
|
"document.querySelector('.table-wrap .settings-table thead th')).position"
|
|
)
|
|
assert sticky == "sticky", f"the thead th must be sticky (got {sticky!r})"
|
|
max_height = page.evaluate(
|
|
"() => getComputedStyle(document.querySelector('.settings-shell .table-wrap')).maxHeight"
|
|
)
|
|
assert max_height not in ("", "none"), f"the card must be height-bounded ({max_height!r})"
|
|
scrolls = page.evaluate(
|
|
"""() => {
|
|
const el = document.querySelector('.settings-shell .table-wrap');
|
|
return el.scrollHeight > el.clientHeight;
|
|
}"""
|
|
)
|
|
assert scrolls, "75 rows must overflow the bounded card (the scroll must bite)"
|
|
|
|
# The groups render in the server's order (the pinned display
|
|
# groups — at least these two).
|
|
expect(page.locator("tr.settings-group-row", has_text="Retrieval & honesty")).to_be_visible()
|
|
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
|
|
|
|
# input_placeholder: the SERVED env value (the module's distinctive
|
|
# base) + the env badge (BOR_INPUT_PLACEHOLDER is set in the app).
|
|
ph = page.locator('tr[data-key="input_placeholder"]')
|
|
expect(ph.locator(".settings-control")).to_have_value(ENV_PLACEHOLDER)
|
|
expect(ph.locator(".settings-src")).to_have_text("env")
|
|
|
|
# top_n_docs: the CODE default (no BOR_TOP_N_DOCS in the app) +
|
|
# the default badge.
|
|
top_n = page.locator('tr[data-key="top_n_docs"]')
|
|
expect(top_n.locator(".settings-control")).to_have_value(
|
|
str(Settings.model_fields["top_n_docs"].default)
|
|
)
|
|
expect(top_n.locator(".settings-src")).to_have_text("default")
|
|
|
|
# llm_api_key (LOCKED A4): a password input whose value is EMPTY
|
|
# (the plaintext never leaves the server) and whose placeholder
|
|
# is the "stored" hint.
|
|
key_input = page.locator('tr[data-key="llm_api_key"] .settings-control')
|
|
assert key_input.get_attribute("type") == "password"
|
|
assert key_input.input_value() == ""
|
|
assert key_input.get_attribute("placeholder") == "•••••• (stored)"
|
|
|
|
# database_url (LOCKED A3 + masked): the URL's CREDENTIALS never
|
|
# reach the DOM (the masked secret renders an EMPTY password
|
|
# input) and the row is read-only (a disabled Reset with the
|
|
# read-only title).
|
|
du = page.locator('tr[data-key="database_url"]')
|
|
assert du.locator(".settings-control").get_attribute("type") == "password"
|
|
assert du.locator(".settings-control").input_value() == ""
|
|
assert "reese" not in page.content(), "the DB URL's credentials leaked into the DOM"
|
|
expect(du.locator(".settings-reset")).to_be_disabled()
|
|
assert "read-only" in (du.locator(".settings-reset").get_attribute("title") or "")
|
|
|
|
# embedding_dim (LOCKED A3): read-only — a disabled Reset with
|
|
# the read-only title (the dim is fixed at table creation, A6).
|
|
ed = page.locator('tr[data-key="embedding_dim"]')
|
|
expect(ed.locator(".settings-control")).to_have_value("768")
|
|
expect(ed.locator(".settings-reset")).to_be_disabled()
|
|
assert "read-only" in (ed.locator(".settings-reset").get_attribute("title") or "")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. The TODO's own acceptance test: Save takes effect WITHOUT a restart
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_save_takes_effect_without_restart(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
|
|
# Filter to "placeholder" (the client-side filter — no fetch),
|
|
# set the value, press Save settings.
|
|
page.fill("#settings-filter", "placeholder")
|
|
expect(page.locator('tr[data-key="input_placeholder"]')).to_be_visible()
|
|
page.fill('tr[data-key="input_placeholder"] .settings-control', SAVED_PLACEHOLDER)
|
|
expect(page.locator("#settings-save")).to_be_enabled()
|
|
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
# The badge flips to DB (the re-GET settled the row)…
|
|
expect(page.locator('tr[data-key="input_placeholder"] .settings-src')).to_have_text("DB")
|
|
|
|
# The live effect with NO restart: a fresh page load of the chat
|
|
# view — the composer's placeholder attribute is the saved value
|
|
# (the /api/config boot fetch resolved the overlay in the same
|
|
# server process).
|
|
page.goto(app_url + "/")
|
|
expect(page.locator("#message-input")).to_have_attribute(
|
|
"placeholder", SAVED_PLACEHOLDER, timeout=15_000
|
|
)
|
|
# Only ONE row was saved: the footer is still the env value.
|
|
expect(page.locator(".footer-text").first).to_have_text(ENV_FOOTER)
|
|
|
|
# The DB value survives a reload of /settings.html (the panel
|
|
# re-reads the settled server state).
|
|
page.goto(app_url + "/settings.html")
|
|
expect(
|
|
page.locator('tr[data-key="input_placeholder"] .settings-control')
|
|
).to_have_value(SAVED_PLACEHOLDER, timeout=15_000)
|
|
expect(page.locator('tr[data-key="input_placeholder"] .settings-src')).to_have_text("DB")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 5. The live read on the retrieval path: the saved bar flips a REAL
|
|
# turn's verdict (the task's proxy — the corpus has no question in
|
|
# [0.30, 0.31] under the mock; "Tell me about basalt" tops ~0.25
|
|
# with zero FTS hits, so the bar is the only gate leg)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_saved_value_reaches_a_real_turn(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
|
|
# Seed the fixture KB (the deterministic mock embeddings) and the
|
|
# admin API client (the raw-SSE turn's credential).
|
|
_reset_db(mock_llm, seed=True)
|
|
client = _admin_client(app_url)
|
|
|
|
# The PRE-SAVE baseline: env 0.30 > the question's top (~0.25,
|
|
# zero FTS hits) → deflected.
|
|
done = _chat_done(client, app_url, BASELALT)
|
|
assert done["deflected"] is True, "the env 0.30 bar must deflect the fixture question"
|
|
baseline = _last_query_log()
|
|
assert baseline.question == BASELALT
|
|
assert (baseline.fts_hits or 0) == 0, "the proxy needs ZERO lexical corroboration"
|
|
assert 0.15 < baseline.top_score < 0.28, (
|
|
f"the question's top score must sit between the floor and the env bar "
|
|
f"with headroom for the high bar (got {baseline.top_score})"
|
|
)
|
|
|
|
# Save the LOW bar (below the top score, above the floor) from
|
|
# the PANEL: the retrieval path must read the NEW number, not
|
|
# the env 0.30.
|
|
top = baseline.top_score
|
|
low_bar = max(0.16, top - 0.05)
|
|
high_bar = top + 0.02 # above the top score, below the env 0.30 bar
|
|
assert high_bar < ENV_BAR, f"the high bar must stay under the env bar ({high_bar})"
|
|
page.fill("#settings-filter", "Relevance")
|
|
page.fill('tr[data-key="relevance_threshold"] .settings-control', str(low_bar))
|
|
page.click("#settings-save")
|
|
# The save-completion signal: the dirty hint empties (the
|
|
# successful batch clears the map — a NEW transition for the
|
|
# first save; the status line alone would be the same text
|
|
# after the second save and race the in-flight PUT).
|
|
expect(page.locator("#settings-dirty")).to_have_text("", timeout=30_000)
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings."
|
|
)
|
|
done = _chat_done(client, app_url, BASELALT)
|
|
assert done["deflected"] is False, (
|
|
f"the saved {low_bar} bar must ground the question (the live read)"
|
|
)
|
|
low_row = _last_query_log()
|
|
assert (low_row.fts_hits or 0) == 0
|
|
|
|
# Save the HIGH bar (above the top score): the decision FLIPS
|
|
# while the retrieval (top_score) is unchanged — the bar, not the
|
|
# corpus, moved.
|
|
page.fill('tr[data-key="relevance_threshold"] .settings-control', str(high_bar))
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-dirty")).to_have_text("", timeout=30_000)
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings."
|
|
)
|
|
done = _chat_done(client, app_url, BASELALT)
|
|
assert done["deflected"] is True, (
|
|
f"the saved {high_bar} bar must deflect the question (the live read)"
|
|
)
|
|
high_row = _last_query_log()
|
|
assert abs(high_row.top_score - low_row.top_score) < 0.001, (
|
|
f"top_score must be unchanged (corpus identical) — "
|
|
f"{low_row.top_score} → {high_row.top_score}"
|
|
)
|
|
|
|
# Reset (the per-row button — the row now has a live Reset): the
|
|
# PRE-SAVE classification returns (env 0.30 > the top score).
|
|
page.click('tr[data-key="relevance_threshold"] .settings-reset')
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"BOR_RELEVANCE_THRESHOLD is back to the environment value.", timeout=30_000
|
|
)
|
|
done = _chat_done(client, app_url, BASELALT)
|
|
assert done["deflected"] is True, "the reset must restore the pre-save verdict"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 6. A5: an invalid value is refused — the batch stops, nothing is
|
|
# stored, the edits stay in the form
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_invalid_value_is_refused_and_nothing_changes(
|
|
page: Page, app_url: str, db_ready: None
|
|
) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
client = _admin_client(app_url)
|
|
|
|
# Two dirty rows (DOM order: the floor row precedes the
|
|
# usefulness row in the ``Retrieval & honesty`` group):
|
|
# ``lexical_support_floor = "-0.5"`` (the invalid one — a
|
|
# DOM-reachable invalid value: the ``float`` row is a
|
|
# ``type="number"`` input, so the number types and the
|
|
# validator's 422 is the server's) and
|
|
# ``source_usefulness_floor = "0.5"`` (a valid value that must
|
|
# NOT be stored — the batch stops at the first failure, K4).
|
|
# No filter (the labels share no substring — the whole table
|
|
# is visible).
|
|
page.fill('tr[data-key="lexical_support_floor"] .settings-control', "-0.5")
|
|
page.fill('tr[data-key="source_usefulness_floor"] .settings-control', "0.5")
|
|
expect(page.locator("#settings-dirty")).to_have_text("2 unsaved changes")
|
|
page.click("#settings-save")
|
|
|
|
# The role=alert error line names the offending field (the
|
|
# server's 422 detail: "invalid value for
|
|
# lexical_support_floor: … must be >= 0") — and stays visible
|
|
# after the catalogue re-GET (the §7.4 never-stale contract;
|
|
# the previous success line is cleared with it).
|
|
error = page.locator("#settings-error")
|
|
expect(error).to_have_attribute("role", "alert")
|
|
expect(error).to_be_visible(timeout=30_000)
|
|
expect(error).to_contain_text("lexical_support_floor")
|
|
|
|
# K4: the rows keep their typed values (re-applied to the
|
|
# re-rendered controls)…
|
|
expect(
|
|
page.locator('tr[data-key="lexical_support_floor"] .settings-control')
|
|
).to_have_value("-0.5")
|
|
expect(
|
|
page.locator('tr[data-key="source_usefulness_floor"] .settings-control')
|
|
).to_have_value("0.5")
|
|
# …and neither was stored: the badges stay env (the app env sets
|
|
# both vars), the API still reports source=env for both, and the
|
|
# dirty hint shows both edits are still pending.
|
|
expect(
|
|
page.locator('tr[data-key="lexical_support_floor"] .settings-src')
|
|
).to_have_text("env")
|
|
expect(
|
|
page.locator('tr[data-key="source_usefulness_floor"] .settings-src')
|
|
).to_have_text("env")
|
|
expect(page.locator("#settings-dirty")).to_have_text("2 unsaved changes")
|
|
floor = _catalog_item(client, app_url, "lexical_support_floor")
|
|
assert floor["source"] == "env" and floor["value"] == "0.15"
|
|
usefulness = _catalog_item(client, app_url, "source_usefulness_floor")
|
|
assert usefulness["source"] == "env" and usefulness["value"] == "0.15"
|
|
# The clean input_placeholder is untouched server-side (the batch
|
|
# never sent it — and the app's /api/config still serves the env
|
|
# base).
|
|
r = client.get(f"{app_url}/api/config", timeout=10)
|
|
assert r.json()["input_placeholder"] == ENV_PLACEHOLDER
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 7. A5/D5: the cross-field pair is refused — stored, the second save
|
|
# 422s, and the pair never becomes effective
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_cross_field_pair_is_refused(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
client = _admin_client(app_url)
|
|
|
|
# Save relevance_threshold = 0.2 (valid alone — floor 0.15 ≤ 0.2)…
|
|
page.fill("#settings-filter", "Relevance")
|
|
page.fill('tr[data-key="relevance_threshold"] .settings-control', "0.2")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
expect(page.locator('tr[data-key="relevance_threshold"] .settings-src')).to_have_text("DB")
|
|
|
|
# …then lexical_support_floor = 0.9: valid alone, BROKEN in
|
|
# combination (0.9 > the DB-overridden 0.2 — the startup
|
|
# validator's cross-field rule). The candidate-set validation
|
|
# 422s it — and stores NOTHING.
|
|
page.fill("#settings-filter", "Lexical")
|
|
page.fill('tr[data-key="lexical_support_floor"] .settings-control', "0.9")
|
|
page.click("#settings-save")
|
|
error = page.locator("#settings-error")
|
|
expect(error).to_be_visible(timeout=30_000)
|
|
expect(error).to_contain_text("lexical_support_floor")
|
|
# K4: the floor's edit stays in the form; the stored pair never
|
|
# becomes effective — the catalogue shows ONLY the first saved
|
|
# value (threshold db 0.2; floor still the env 0.15).
|
|
expect(
|
|
page.locator('tr[data-key="lexical_support_floor"] .settings-control')
|
|
).to_have_value("0.9")
|
|
threshold = _catalog_item(client, app_url, "relevance_threshold")
|
|
floor = _catalog_item(client, app_url, "lexical_support_floor")
|
|
assert threshold["source"] == "db" and threshold["db_value"] == "0.2"
|
|
assert floor["source"] == "env" and floor["value"] == "0.15"
|
|
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.json()["totals"]["overridden"] == 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 8. LOCKED A4: a secret never reaches the browser
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_secret_never_reaches_the_browser(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
client = _admin_client(app_url)
|
|
|
|
# Store a sentinel key THROUGH THE API (not the DOM)…
|
|
r = client.put(
|
|
f"{app_url}/api/admin/settings/llm_api_key", json={"value": SENTINEL_KEY}
|
|
)
|
|
assert r.status_code == 200, r.text
|
|
assert r.json()["masked"] is True and r.json()["value"] is None
|
|
# …reload /settings.html: the sentinel is ABSENT from the page,
|
|
# the password input shows the "stored" placeholder (an empty
|
|
# value — "leave unchanged"), and the has_override row renders a
|
|
# LIVE Reset.
|
|
page.goto(app_url + "/settings.html")
|
|
key_input = page.locator('tr[data-key="llm_api_key"] .settings-control')
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
assert key_input.input_value() == ""
|
|
assert key_input.get_attribute("placeholder") == "•••••• (stored)"
|
|
assert SENTINEL_KEY not in page.content(), "the sentinel leaked into the page"
|
|
expect(page.locator('tr[data-key="llm_api_key"] .settings-reset')).to_be_enabled()
|
|
# The served HTML and the raw API body agree (the plaintext
|
|
# appears NOWHERE — asserted on the raw text, not the parsed
|
|
# model).
|
|
raw_html = client.get(app_url + "/settings.html", timeout=10).text
|
|
assert SENTINEL_KEY not in raw_html
|
|
raw_api = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert SENTINEL_KEY not in raw_api.text
|
|
|
|
# Set the key FROM THE PANEL's password input + Save: the
|
|
# rotated value is stored (the Reset stays live) and the
|
|
# sentinel STILL never appears in the served HTML or the
|
|
# response body.
|
|
page.fill('tr[data-key="llm_api_key"] .settings-control', ROTATED_KEY)
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
assert SENTINEL_KEY not in page.content()
|
|
assert ROTATED_KEY not in page.content(), "the new secret must never render either"
|
|
raw_html = client.get(app_url + "/settings.html", timeout=10).text
|
|
assert SENTINEL_KEY not in raw_html and ROTATED_KEY not in raw_html
|
|
raw_api = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert SENTINEL_KEY not in raw_api.text and ROTATED_KEY not in raw_api.text
|
|
expect(page.locator('tr[data-key="llm_api_key"] .settings-reset')).to_be_enabled()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 9. The Reset: control goes back to the environment
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_reset_hands_control_back_to_the_env(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
|
|
# A DB override on footer_text (the live effect first)…
|
|
page.fill("#settings-filter", "footer")
|
|
page.fill('tr[data-key="footer_text"] .settings-control', "DB footer — panel override")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
page.goto(app_url + "/")
|
|
expect(page.locator(".footer-text").first).to_have_text(
|
|
"DB footer — panel override", timeout=15_000
|
|
)
|
|
|
|
# …then the row's Reset: the status line, the badge back to
|
|
# env, and the footer on / reading the env value again.
|
|
page.goto(app_url + "/settings.html")
|
|
page.click('tr[data-key="footer_text"] .settings-reset')
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"BOR_FOOTER_TEXT is back to the environment value.", timeout=30_000
|
|
)
|
|
footer = page.locator('tr[data-key="footer_text"]')
|
|
expect(footer.locator(".settings-src")).to_have_text("env")
|
|
expect(footer.locator(".settings-control")).to_have_value(ENV_FOOTER)
|
|
page.goto(app_url + "/")
|
|
expect(page.locator(".footer-text").first).to_have_text(ENV_FOOTER)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 10. Clear all overrides: the two-step confirm (Escape cancels,
|
|
# confirming lands "Cleared 3 overrides.")
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_clear_all_overrides(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
client = _admin_client(app_url)
|
|
|
|
# Save THREE overrides in one batch (the dirty-rows-only save)…
|
|
page.fill('tr[data-key="app_name"] .settings-control', "E2E cleared brain")
|
|
page.fill('tr[data-key="input_placeholder"] .settings-control', SAVED_PLACEHOLDER)
|
|
page.fill('tr[data-key="top_n_docs"] .settings-control', "7")
|
|
expect(page.locator("#settings-dirty")).to_have_text("3 unsaved changes")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 3 settings.", timeout=30_000
|
|
)
|
|
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.json()["totals"]["overridden"] == 3
|
|
|
|
# …the two-step confirm arms (focus moves to the Yes button)…
|
|
page.click("#settings-clear-all")
|
|
confirm = page.locator(".settings-confirm")
|
|
expect(confirm.locator(".history-confirm-text")).to_have_text("Clear all overrides?")
|
|
expect(confirm.locator(".history-confirm-yes")).to_be_visible()
|
|
|
|
# …Escape CANCELS: the button is restored and the overrides
|
|
# SURVIVE (three badges back, the table still overridden).
|
|
page.keyboard.press("Escape")
|
|
expect(page.locator("#settings-clear-all")).to_be_visible(timeout=15_000)
|
|
assert page.locator(".settings-confirm").count() == 0
|
|
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.json()["totals"]["overridden"] == 3, "Escape must not clear"
|
|
|
|
# …confirming lands "Cleared 3 overrides." and every badge falls
|
|
# back to env/default (zero DB badges — app_name /
|
|
# input_placeholder back to their env values, top_n_docs to the
|
|
# code default).
|
|
page.click("#settings-clear-all")
|
|
page.click(".settings-confirm .history-confirm-yes")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Cleared 3 overrides.", timeout=30_000
|
|
)
|
|
assert page.locator(".settings-src.is-db").count() == 0
|
|
expect(page.locator('tr[data-key="app_name"] .settings-src')).to_have_text("env")
|
|
expect(page.locator('tr[data-key="input_placeholder"] .settings-src')).to_have_text("env")
|
|
expect(page.locator('tr[data-key="top_n_docs"] .settings-src')).to_have_text("default")
|
|
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.json()["totals"]["overridden"] == 0
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 11. D6: the kill switch — a second, real app with BOR_SETTINGS_DB=0
|
|
# (the subprocess-level variant; the route-interception fallback
|
|
# was not needed)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_disabled_switch_is_announced(
|
|
page: Page, disabled_app_url: str, db_ready: None
|
|
) -> None:
|
|
page.set_default_timeout(30_000)
|
|
login(page, disabled_app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
|
|
# The banner is visible (role=alert — the mechanism is off, not
|
|
# the panel)…
|
|
banner = page.locator("#settings-disabled")
|
|
expect(banner).to_have_attribute("role", "alert")
|
|
expect(banner).to_be_visible(timeout=15_000)
|
|
# …and the API reports overlay_enabled=false (the badge falls
|
|
# back to env for a stored row — the mechanism is not applying
|
|
# it).
|
|
client = _admin_client(disabled_app_url)
|
|
r = client.get(f"{disabled_app_url}/api/admin/settings", timeout=30.0)
|
|
assert r.json()["overlay_enabled"] is False
|
|
|
|
# A save still STORES the value…
|
|
page.fill("#settings-filter", "footer")
|
|
page.fill('tr[data-key="footer_text"] .settings-control', "DB footer — switch off")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
with SessionLocal() as db:
|
|
row = db.execute(
|
|
text("SELECT value FROM app_settings WHERE key = 'footer_text'")
|
|
).scalar_one()
|
|
assert row == "DB footer — switch off", "the save must still store the row"
|
|
|
|
# …but the app's behavior is UNCHANGED: the footer on / is still
|
|
# the env value (the overlay is off — the stored row is not in
|
|
# effect) and the badge is env, not db.
|
|
page.goto(disabled_app_url + "/")
|
|
expect(page.locator(".footer-text").first).to_have_text(ENV_FOOTER)
|
|
page.goto(disabled_app_url + "/settings.html")
|
|
expect(
|
|
page.locator('tr[data-key="footer_text"] .settings-src')
|
|
).to_have_text("env")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 12. PLAN §7.1 + rule 6: no horizontal overflow at 375/1440, the
|
|
# table is ≥80% of the container, the keyboard walk, zero
|
|
# external requests
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_view_layout_and_accessibility(page: Page, app_url: str, db_ready: None) -> None:
|
|
page.set_default_timeout(30_000)
|
|
externals: list[str] = []
|
|
|
|
def _watch(request: object) -> None:
|
|
url = getattr(request, "url", "")
|
|
if url.startswith(("http://", "https://")) and "127.0.0.1" not in url:
|
|
externals.append(url)
|
|
|
|
page.on("request", _watch)
|
|
login(page, app_url, next="/settings.html")
|
|
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
|
|
|
|
# Save an override (a live Reset exists — the row's Reset is
|
|
# enabled) and re-edit it (a dirty row — Save is enabled for
|
|
# the keyboard walk).
|
|
page.fill('tr[data-key="app_name"] .settings-control', "E2E layout brain")
|
|
page.click("#settings-save")
|
|
expect(page.locator("#settings-status")).to_contain_text(
|
|
"Saved 1 settings.", timeout=30_000
|
|
)
|
|
page.fill('tr[data-key="app_name"] .settings-control', "E2E layout brain 2")
|
|
|
|
# No horizontal page overflow at 375px or at 1440px (the 72rem
|
|
# column at every viewport — the table scrolls INSIDE its card,
|
|
# the page itself never does).
|
|
for width in (375, 1440):
|
|
page.set_viewport_size({"width": width, "height": 900})
|
|
page.wait_for_timeout(150) # the reflow settles before the measure
|
|
overflow = page.evaluate(
|
|
"() => document.documentElement.scrollWidth <= window.innerWidth + 1"
|
|
)
|
|
assert overflow, f"horizontal overflow at {width}px"
|
|
|
|
# At 1440 the table is ≥80% of the 72rem container (not a
|
|
# skinny wasted-space list — PLAN §7.1).
|
|
ratio = page.evaluate(
|
|
"""() => document.querySelector('.settings-shell .table-wrap')
|
|
.getBoundingClientRect().width /
|
|
document.querySelector('.settings-shell').getBoundingClientRect().width"""
|
|
)
|
|
assert ratio >= 0.8, f"the table must be ≥80% of the container (got {ratio:.2f})"
|
|
|
|
# The restart-required flag on db_pool_size's row (literal text
|
|
# — never color alone, B5).
|
|
expect(page.locator('tr[data-key="db_pool_size"] .settings-flag')).to_have_text(
|
|
"restart required"
|
|
)
|
|
|
|
# The keyboard tab order reaches filter → first input → (an
|
|
# enabled) Reset → Save → Clear all, in DOM order. The walk
|
|
# starts at the skip-link — the document's FIRST tabbable
|
|
# element (a blur() would only continue the cycle after the
|
|
# last focused element, so the script focuses the cycle
|
|
# start; every move after it is a real Tab keypress).
|
|
page.set_viewport_size({"width": 1280, "height": 800})
|
|
page.evaluate("() => document.querySelector('a.skip-link').focus()")
|
|
seen: list[str] = []
|
|
for _ in range(200):
|
|
page.keyboard.press("Tab")
|
|
seen.append(
|
|
page.evaluate(
|
|
"""() => {
|
|
const el = document.activeElement;
|
|
if (!el) return "";
|
|
return el.id || (typeof el.className === "string" ? el.className : "");
|
|
}"""
|
|
)
|
|
)
|
|
i_filter = next((i for i, s in enumerate(seen) if s == "settings-filter"), None)
|
|
i_first_input = next((i for i, s in enumerate(seen) if s.startswith("set-")), None)
|
|
i_reset = next((i for i, s in enumerate(seen) if "settings-reset" in s), None)
|
|
i_save = next((i for i, s in enumerate(seen) if s == "settings-save"), None)
|
|
i_clear = next((i for i, s in enumerate(seen) if s == "settings-clear-all"), None)
|
|
assert i_filter is not None, f"the walk never reached the filter (seen={seen[:20]})"
|
|
assert i_first_input is not None, "the walk never reached a settings control"
|
|
assert i_save is not None, "the walk never reached Save (the dirty row enables it)"
|
|
assert i_clear is not None, "the walk never reached Clear all"
|
|
assert i_filter < i_first_input, "filter must precede the first input"
|
|
assert i_first_input < i_save, "the first input must precede Save"
|
|
assert i_save < i_clear, "Save must precede Clear all"
|
|
if i_reset is not None:
|
|
assert i_first_input <= i_reset < i_save, (
|
|
"an enabled Reset sits in the table (between the first input and Save)"
|
|
)
|
|
|
|
# Rule 6: the whole test issued ZERO external requests (the CSP
|
|
# default-src 'self' posture — every asset is local).
|
|
assert externals == [], f"external requests leaked: {externals}"
|