phase: 173_docs_push_creds_split
Build and Push Containers / build-and-push-app (push) Successful in 1m41s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All gates green. Final report:

**Phase 173 — final verification pass (all tasks already complete in `complete/`)**

- Verified the full implementation is in the working tree: `docs_user`/`docs_token` settings + catalogue rows (task 01), `docs_repo` PUT 422 guard + `docs_token` write-only pin (task 02), `run_git(env)` + `push_document(user, token)` credential env (task 03), `settings_migrate.py` wired into `create_app()` (task 04), E2E suite (task 05).
- **Completion criteria:**
  1. GET `/api/admin/settings` lists `docs_user` (plain) + `docs_token` (secret, `value=null`), token never in any body — ✅ `test_docs_token_is_write_only_masked_end_to_end` + E2E sentinel-absent-from-DOM
  2. PUT `docs_repo` with `://user:pass@` → 422 naming the fields; clean URL passes — ✅ integration + E2E UI-side
  3. Token set → `GIT_CURL_USERNAME`/`GIT_CURL_PASSWORD`/`GIT_TERMINAL_PROMPT=0` on every git step; empty → `env=None` — ✅ mocked-git unit tests
  4. Legacy env split once at startup, idempotent, one INFO line, `.env` untouched — ✅ `test_settings_migrate.py` (4 shapes) + `test_docs_push_migrate.py`
  5. Full suite green, coverage >90%, lint clean — ✅
  6. E2E isolated: `test_docs_push_creds.py` 4/4; regression set (panel + layout + response_to_docs) 21/21 — ✅
  7. Commit + phase move — harness's job (per instructions, not mine).
  8. README/AGENTS consistent — ✅ (below)
- Tests/lint: `uv run pytest` exit 0 (4112 passed) · `uv run pytest --cov=app --cov-report=term-missing` TOTAL 99% · `uv run ruff check .` clean · `uv run pyright` 0 errors
- Docs: README.md (incl. Docs-push config section, guard/migration paragraph, `.env.example` block) and AGENTS.md verified consistent in full; fixed two stale docstrings in `app/config.py` + `app/core/settings_catalog.py` (`is_set` → the API's actual `has_value` field).
- No live-infrastructure changes (no live step in this phase). No next pending phase — `todo/` held only 173.
This commit is contained in:
2026-10-10 16:47:39 -04:00
parent f27106a638
commit a86caec821
56 changed files with 3177 additions and 39 deletions
+7 -2
View File
@@ -553,7 +553,7 @@ def test_admin_sees_groups_and_source_badges(page: Page, app_url: str, db_ready:
return el.scrollHeight > el.clientHeight;
}"""
)
assert scrolls, "75 rows must overflow the bounded card (the scroll must bite)"
assert scrolls, "78 rows must overflow the bounded card (the scroll must bite)"
# The groups render in the server's order (the pinned display
# groups — at least these two).
@@ -981,7 +981,12 @@ def test_clear_all_overrides(page: Page, app_url: str, db_ready: None) -> None:
expect(page.locator("#settings-status")).to_contain_text(
"Cleared 3 overrides.", timeout=30_000
)
assert page.locator(".settings-src.is-db").count() == 0
# The re-GET settles the badges (the status line lands BEFORE the
# re-render — the §7.4 lifecycle); wait for the settled state
# rather than snapshotting a mid-flight render (the combined-run
# race — the 78-item catalog's re-GET can outlast the status
# poll's gap under a three-suite run, phase-173 regression pass).
expect(page.locator(".settings-src.is-db")).to_have_count(0, timeout=30_000)
expect(page.locator('tr[data-key="app_name"] .settings-src')).to_have_text("env")
expect(page.locator('tr[data-key="input_placeholder"] .settings-src')).to_have_text("env")
expect(page.locator('tr[data-key="top_n_docs"] .settings-src')).to_have_text("default")
+531
View File
@@ -0,0 +1,531 @@
"""Phase 173 E2E (Playwright): the docs-push credentials — the LEAK CLASS
is dead in the real UI.
Source: owner audit of the production admin Settings panel (2026-10-10)
— ``BOR_DOCS_REPO`` was a plain-text row whose production value was a
git URL with an embedded username + Gitea token: a live credential leak
on the settings page, the raw HTML, and the settings API. This phase
splits the credentials into ``docs_user`` (plain) + ``docs_token``
(write-only, ``secret=True`` — the ``llm_api_key``-style row), guards
the ``docs_repo`` PUT against embedded credentials (422), and migrates
a legacy value once at startup (C1–C5, phase-locked).
Run in isolation (DB must be up: ``podman compose up -d db``; the
suite needs NO LLM — the mock is the house default, A16):
uv run pytest tests/e2e/test_docs_push_creds.py -v --no-cov
Test → contract mapping (one phase, one isolated file — AGENTS rule 4):
1. ``test_rows_render_token_write_only`` — the split shape in the
real UI: the ``Docs push`` group shows a ``Docs repo`` TEXT row
(the clean URL, seeded through the admin API), a ``Docs user``
TEXT row, and a ``Docs token`` row that is WRITE-ONLY — its
input is ``type=password``, NOT pre-filled (value ``""``),
rendered in the phase-168 secret-row style (the ``••••••
(stored)`` placeholder, the ``BOR_DOCS_TOKEN`` aria-label), with
the is-set state visible (DB badge + a live Reset) — and the
seeded token text is ABSENT from the initial page DOM, the
served HTML, and the raw ``GET /api/admin/settings`` body.
2. ``test_save_stays_masked`` — the core assertion: typing a unique
sentinel token into the ``Docs token`` row and saving re-renders
the row masked; a reload of ``/settings.html`` keeps it masked
with the is-set state visible (DB badge, live Reset, API
``has_value``), and the **sentinel is absent from the entire
page DOM** — ``page.content()`` must not contain it (nor must
the served HTML or the raw API body).
3. ``test_reset_unsets`` — the row's Reset hands control back to the
environment: the status line, the Reset back to DISABLED (the
"no saved override" title — the phase-168 not-set marker for a
secret row; the placeholder is the constant ``•••••• (stored)``
for every secret row, set or not), the badge back to ``env``
(the env base is set-but-empty in the module app), the input
empty, the sentinel gone from the DOM, and the API reporting
``has_value=false`` / ``has_override=false``.
4. ``test_creds_url_guard_surfaces_and_stays_clean`` — the C4 guard
UI-side: with a clean ``docs_repo`` saved, typing a
credentials-bearing URL (``https://u:pass@e2e.invalid/o/r.git``)
and saving 422s — the ``role="alert"`` line names the field and
the replacement (the ``Docs user`` / ``Docs token`` fields) —
the input echoes the typed value (K4, the pre-save state, fine),
and after the failed save + reload the row keeps the PREVIOUS
saved value with the typed URL's ``pass`` segment ABSENT from
the page DOM and the API still reporting the clean URL.
DB isolation: the shared e2e Postgres keeps ``app_settings`` (the
overlay rows — a leftover row would retune every later suite's app
through the conftest pin's failure mode) and ``ui_settings`` (the
single row the caching middleware reads for EVERY served page)
across suites. An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before AND
after every test (never a TRUNCATE on ``api_tokens`` — the shared
DB may hold the owner's real tokens); the module teardown pins
``app_settings`` empty. The module app opts BACK IN over the
conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_SETTINGS_DB"]
= "1"`` — the ``test_pictorial_pdf.py`` opt-in pattern) and pins the
docs-push base vars to DISTINCTIVE empties (``BOR_DOCS_REPO`` /
``BOR_DOCS_USER`` / ``BOR_DOCS_TOKEN`` = ``""``) so an operator's
local (gitignored) ``.env`` — loaded by pydantic from
``cwd=REPO`` — cannot leak a credentials-bearing repo (the startup
migration's subject) or a real token into the suite's app: process
env ranks above the ``.env`` file. A module-scope ``.env`` mtime
guard pins LOCKED A1: no code path the suite drives opens the env
file for writing.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import uuid
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
REPO = Path(__file__).resolve().parents[2]
#: The module app's OWN port (the test_response_to_docs.py pattern —
#: the conftest session app owns 8123 in a combined run; a same-port
#: second uvicorn dies on bind and would drive the wrong server).
#: Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_CREDS", "8190"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The test-seed values (test 1) — distinctive so "the DB row now
#: wins" is visible, and the seeded token is the value that must
#: appear NOWHERE outside the DB.
SEED_REPO = "https://gitea.example.com/e2e/docs.git"
SEED_USER = "e2e-docs-push-user"
SEED_TOKEN = "sk-e2e-docs-seed-0123456789abcd"
#: Test 4 — the clean value the row KEEPS, and the credentials-bearing
#: URL the guard must refuse (the ``pass`` segment must never persist).
CLEAN_REPO = "https://gitea.example.com/e2e/docs.git"
CRED_URL = "https://u:pass@e2e.invalid/o/r.git"
#: The phase-168 secret-row placeholder (the frontend's constant for
#: every ``secret=True`` row — settings.js's makeControl).
STORED_PLACEHOLDER = "•••••• (stored)"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning/theme pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The module app (the shared env block + the phase-173 pins):
the overlay ON (``BOR_SETTINGS_DB=1`` — the module opts back in
over the conftest's global pin) and the docs-push base vars
pinned to DISTINCTIVE empties (an operator's local ``.env`` must
not leak a credentials-bearing repo or a real token in)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — explicit so
# an operator's local .env cannot leak a different gate into the
# suite's app; the startup validator needs floor ≤ threshold).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# Phase 173 (this suite): the docs-push base pinned to empties —
# no credentials-bearing repo for the startup migration to act on,
# no real token in effect, and the "env" source badge deterministic
# (the var is set-but-empty → source "env", not "default").
env["BOR_DOCS_REPO"] = ""
env["BOR_DOCS_USER"] = ""
env["BOR_DOCS_TOKEN"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Phase 168: the module opts BACK IN over the conftest's global
# BOR_SETTINGS_DB=0 pin (the test_pictorial_pdf.py pattern).
env["BOR_SETTINGS_DB"] = "1"
env["BOR_GIT_SOURCES"] = ""
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — the overlay ON (``BOR_SETTINGS_DB=1``)
and the docs-push base pinned to empties (the phase-173 pins)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme state per test: truncate ``app_settings``
(a leftover row is the cross-suite poison this phase's overlay
guard exists to prevent) and the single-row ``ui_settings``
(the middleware reads it for EVERY page), and delete this suite's
issued tokens (label-scoped on ``e2e-`` — never a TRUNCATE: the
shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
_clean_settings_state()
yield
_clean_settings_state()
# The module's final state: app_settings EMPTY (the poison guard —
# the next suite's app must boot with no overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _put_setting(
client: httpx.Client, app_url: str, key: str, value: str
) -> httpx.Response:
r = client.put(f"{app_url}/api/admin/settings/{key}", json={"value": value})
assert r.status_code == 200, f"PUT {key} → {r.status_code}: {r.text[:200]}"
return r
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
def _unique_sentinel() -> str:
"""The task's unique sentinel (a per-run uuid4 — "a saved token
never appears anywhere in the page DOM" is asserted against a
value that is unguessable, not a fixed constant)."""
return f"e2e-sentinel-{uuid.uuid4().hex}"
# ---------------------------------------------------------------------------
# 1. The rows render: the clean-URL text row, the user text row, and
# the WRITE-ONLY token row — masked, not pre-filled, absent from
# the initial DOM
# ---------------------------------------------------------------------------
def test_rows_render_token_write_only(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
# Seed all three rows THROUGH THE ADMIN API (before the initial
# page read — the panel's first GET must already see the stored
# values): the clean repo URL, the plain user, and the token
# (whose PUT response is the write-only contract — masked,
# value null).
client = _admin_client(app_url)
_put_setting(client, app_url, "docs_repo", SEED_REPO)
_put_setting(client, app_url, "docs_user", SEED_USER)
r = _put_setting(client, app_url, "docs_token", SEED_TOKEN)
assert r.json()["masked"] is True and r.json()["value"] is None
# The API's read side for the new key (the C1 pin — the phase-168
# secret contract, zero extra code): group / label / secret /
# value=null / has_value, and the plain user's value.
tok = _catalog_item(client, app_url, "docs_token")
assert tok["group"] == "Docs push" and tok["label"] == "Docs token"
assert tok["secret"] is True and tok["value"] is None
assert tok["has_value"] is True and tok["has_override"] is True
user = _catalog_item(client, app_url, "docs_user")
assert user["secret"] is False and user["value"] == SEED_USER
repo = _catalog_item(client, app_url, "docs_repo")
assert repo["value"] == SEED_REPO
# The UI: sign in → the ``Docs push`` group renders with the
# three rows in the catalogue's order…
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator("tr.settings-group-row", has_text="Docs push")
).to_be_visible()
# …the ``Docs repo`` TEXT row (the clean URL, DB badge)…
repo_row = page.locator('tr[data-key="docs_repo"]')
expect(repo_row.locator(".cell-clip")).to_have_text("Docs repo")
repo_input = repo_row.locator(".settings-control")
assert repo_input.get_attribute("type") == "text"
expect(repo_input).to_have_value(SEED_REPO)
expect(repo_row.locator(".settings-src")).to_have_text("DB")
# …the ``Docs user`` TEXT row (the test-seed value, DB badge)…
user_row = page.locator('tr[data-key="docs_user"]')
expect(user_row.locator(".cell-clip")).to_have_text("Docs user")
user_input = user_row.locator(".settings-control")
assert user_input.get_attribute("type") == "text"
expect(user_input).to_have_value(SEED_USER)
expect(user_row.locator(".settings-src")).to_have_text("DB")
# …and the ``Docs token`` row that is WRITE-ONLY: type=password,
# NOT pre-filled (the stored value never reaches the input),
# rendered in the phase-168 secret-row style (the "(stored)"
# placeholder, the aria-label naming the env var), with the
# is-set state visible (the DB badge + a live Reset).
tok_row = page.locator('tr[data-key="docs_token"]')
expect(tok_row.locator(".cell-clip")).to_have_text("Docs token")
tok_input = tok_row.locator(".settings-control")
assert tok_input.get_attribute("type") == "password"
assert tok_input.input_value() == ""
assert tok_input.get_attribute("placeholder") == STORED_PLACEHOLDER
assert tok_input.get_attribute("aria-label") == "BOR_DOCS_TOKEN"
expect(tok_row.locator(".settings-src")).to_have_text("DB")
expect(tok_row.locator(".settings-reset")).to_be_enabled()
# The seeded token text is ABSENT from the initial DOM, the
# served HTML, and the raw API body (the LEAK CLASS — asserted
# on the raw text, not the parsed model).
assert SEED_TOKEN not in page.content(), "the seeded token leaked into the page"
raw_html = client.get(app_url + "/settings.html", timeout=10).text
assert SEED_TOKEN not in raw_html, "the seeded token leaked into the served HTML"
raw_api = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert SEED_TOKEN not in raw_api.text, "the seeded token leaked into the API body"
# ---------------------------------------------------------------------------
# 2. The core assertion: a saved token stays masked — save, reload,
# the sentinel is absent from the entire page DOM
# ---------------------------------------------------------------------------
def test_save_stays_masked(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The row starts NOT set (the module's docs base is set-but-
# empty): an empty password input, the constant "(stored)"
# placeholder, and a DISABLED Reset (the "no saved override"
# title — the phase-168 not-set marker).
tok_row = page.locator('tr[data-key="docs_token"]')
tok_input = tok_row.locator(".settings-control")
assert tok_input.input_value() == ""
expect(tok_row.locator(".settings-reset")).to_be_disabled()
expect(tok_row.locator(".settings-src")).to_have_text("env")
# Type a unique sentinel into the ``Docs token`` row → Save…
sentinel = _unique_sentinel()
page.fill('tr[data-key="docs_token"] .settings-control', sentinel)
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
# …the row re-renders masked: the input is EMPTY again (the
# typed value was sent, never painted back), the placeholder
# stands, and the is-set state is visible (DB badge + a live
# Reset).
assert tok_input.input_value() == ""
assert tok_input.get_attribute("placeholder") == STORED_PLACEHOLDER
expect(tok_row.locator(".settings-src")).to_have_text("DB")
expect(tok_row.locator(".settings-reset")).to_be_enabled()
assert sentinel not in page.content(), "the sentinel leaked into the page"
# Reload /settings.html (the panel re-reads the settled server
# state): the row is STILL masked…
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
tok_input = page.locator('tr[data-key="docs_token"] .settings-control')
assert tok_input.input_value() == ""
assert tok_input.get_attribute("placeholder") == STORED_PLACEHOLDER
# …with the is-set state visible (the DB badge + the live
# Reset)…
tok_row = page.locator('tr[data-key="docs_token"]')
expect(tok_row.locator(".settings-src")).to_have_text("DB")
expect(tok_row.locator(".settings-reset")).to_be_enabled()
# …and the API agrees (has_value, value null, no override
# plaintext).
tok = _catalog_item(client, app_url, "docs_token")
assert tok["has_value"] is True and tok["value"] is None
# THE core assertion: the sentinel is ABSENT from the entire
# page DOM — and from the served HTML and the raw API body.
assert sentinel not in page.content(), "the sentinel leaked into the reloaded page"
raw_html = client.get(app_url + "/settings.html", timeout=10).text
assert sentinel not in raw_html, "the sentinel leaked into the served HTML"
raw_api = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert sentinel not in raw_api.text, "the sentinel leaked into the API body"
# ---------------------------------------------------------------------------
# 3. The Reset unsets: the row returns to the not-set state
# ---------------------------------------------------------------------------
def test_reset_unsets(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Set the token from the panel (the same dirty-row flow)…
sentinel = _unique_sentinel()
page.fill('tr[data-key="docs_token"] .settings-control', sentinel)
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
# …then the row's Reset: the status line…
page.click('tr[data-key="docs_token"] .settings-reset')
expect(page.locator("#settings-status")).to_contain_text(
"BOR_DOCS_TOKEN is back to the environment value.", timeout=30_000
)
# …and the row returns to the NOT-SET state: the Reset is
# DISABLED with the "no saved override" title (the phase-168
# not-set marker for a secret row — the "(stored)" placeholder
# is the frontend's constant for every secret row, set or not,
# so the badge + the Reset ARE the state), the badge is back
# to env (the module's set-but-empty base), the input is empty,
# and the sentinel is gone from the DOM.
tok_row = page.locator('tr[data-key="docs_token"]')
reset = tok_row.locator(".settings-reset")
expect(reset).to_be_disabled()
assert "no saved override" in (reset.get_attribute("title") or "")
expect(tok_row.locator(".settings-src")).to_have_text("env")
assert tok_row.locator(".settings-control").input_value() == ""
assert sentinel not in page.content(), "the sentinel survived the reset"
# The API agrees: the row is gone (has_override false, the
# set-but-empty env base → has_value false, value null).
tok = _catalog_item(client, app_url, "docs_token")
assert tok["has_override"] is False and tok["has_value"] is False
assert tok["value"] is None and tok["source"] == "env"
# ---------------------------------------------------------------------------
# 4. The C4 guard, UI-side: a credentials-bearing URL 422s, the row
# keeps the previous value, and the typed URL never persists
# ---------------------------------------------------------------------------
def test_creds_url_guard_surfaces_and_stays_clean(
page: Page, app_url: str, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Save a CLEAN docs_repo through the panel (the "previous
# value")…
page.fill('tr[data-key="docs_repo"] .settings-control', CLEAN_REPO)
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
# …then type a CREDENTIALS-BEARING URL into the same row and
# save: the batch's single PUT hits the phase-173 guard (C4)…
page.fill('tr[data-key="docs_repo"] .settings-control', CRED_URL)
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
page.click("#settings-save")
# …and the 422 surfaces (the phase-168 error-display contract —
# the role=alert line names the field AND the replacement).
error = page.locator("#settings-error")
expect(error).to_have_attribute("role", "alert")
expect(error).to_be_visible(timeout=30_000)
expect(error).to_contain_text("docs_repo")
expect(error).to_contain_text("Docs user / Docs token")
# K4: the input echoes the typed value (the pre-save state —
# that is fine; the assertion is on the saved/persisted state
# after the failed save + reload).
expect(
page.locator('tr[data-key="docs_repo"] .settings-control')
).to_have_value(CRED_URL)
# The failed save stored NOTHING: the API still reports the
# clean URL as the value (db — the previously saved row).
repo = _catalog_item(client, app_url, "docs_repo")
assert repo["value"] == CLEAN_REPO and repo["source"] == "db"
# Reload: the row KEEPS the previous (clean) value, and the
# typed URL — its ``pass`` segment above all — is ABSENT from
# the page DOM and the raw API body (never shown as a saved
# value).
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="docs_repo"] .settings-control')
).to_have_value(CLEAN_REPO)
assert CRED_URL not in page.content(), "the credentials-bearing URL persisted"
assert "u:pass@" not in page.content(), "the URL's pass segment persisted"
raw_api = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert CRED_URL not in raw_api.text
assert "u:pass@" not in raw_api.text
+166 -3
View File
@@ -9,7 +9,7 @@ The pins (00_phase, Testing & Quality):
* the auth matrix — anonymous AND a signed-in token USER get 403 on ALL
four routes (the router-wide ``require_admin`` closes the surface on
every method), the admin 200s on all four;
* the shape — ``items`` length == ``len(Settings.model_fields)`` (75),
* the shape — ``items`` length == ``len(Settings.model_fields)`` (78),
groups in ``GROUPS`` order (the stable order the E2E pins against),
``totals.count`` / ``overridden`` correct, ``overlay_enabled`` true;
on an empty table a field whose ``BOR_*`` var the test env sets
@@ -152,7 +152,7 @@ def test_auth_matrix_all_four_routes(client: TestClient, db: Session) -> None:
def test_catalogue_shape_and_sources_on_empty_table(admin_client) -> None:
"""The whole catalogue, grouped and stable: 75 items (exactly
"""The whole catalogue, grouped and stable: 78 items (exactly
``Settings.model_fields``), groups in ``GROUPS`` order (each group a
contiguous block, the block order = ``GROUPS``), ``totals`` correct,
``overlay_enabled`` true. On an empty table: the env-var-set field
@@ -418,7 +418,7 @@ def test_kill_switch_disables_the_mechanism_not_the_panel(
def test_catalogue_stays_under_the_byte_budget(admin_client) -> None:
"""The whole catalogue stays under the phase-167 INFO bar (64 KB —
``BOR_LOG_LARGE_RESPONSE_BYTES``), so ``GET /api/admin/settings``
never floods the log (75 items ≈ 20 KB; the audit script re-verifies
never floods the log (78 items ≈ 20 KB; the audit script re-verifies
the 256 KB budget in task 06)."""
r = admin_client.get(API)
assert r.status_code == 200
@@ -602,3 +602,166 @@ def test_llm_thinking_level_row_contract(
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
def test_docs_token_is_write_only_masked_end_to_end(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 173 (task 02, C1): ``docs_token`` is a write-only
``secret=True`` row — the EXISTING generic secret contract pinned
for the new key (no new code): (a) before a save the GET reports
the row in the ``Docs push`` group, ``secret=true``,
``value=null`` (masked) and ``has_value=false`` — with
``docs_user`` a plain ``str`` row at ``""``; (b) a PUT stores it
(200) and the GET reports ``has_value=true`` with the plaintext
NOWHERE in the raw body — the core pin: the token text is absent
from the response JSON, the leak class this phase kills; (c) a
blank PUT 422s (the self-lockout guard — ``DELETE`` is the
explicit clear) and the stored value is unchanged; (d) a DELETE
unsets it (reset to the env value — ``""`` by default).
"""
# A deterministic base (machine-independent, the phase-172 test's
# idiom): an explicit empty process value for both docs
# credentials — the process env RANKS ABOVE the operator's local
# ``.env`` (the conftest pin's reasoning), so this pins the code
# defaults no matter what the machine's ``.env`` carries.
monkeypatch.setenv("BOR_DOCS_USER", "")
monkeypatch.setenv("BOR_DOCS_TOKEN", "")
invalidate_settings()
# (a) the rows: docs_token masked + unset, docs_user plain + empty.
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "docs_token")
assert item["group"] == "Docs push"
assert item["label"] == "Docs token"
assert item["type"] == "str"
assert item["env_var"] == "BOR_DOCS_TOKEN"
assert item["secret"] is True
assert item["restart_required"] is False
assert item["overridable"] is True
assert item["masked"] is True
assert item["value"] is None
assert item["db_value"] is None
assert item["has_value"] is False
assert item["has_override"] is False
assert item["default"] == "" # masked: the code default is never shown
user = _item(r.json(), "docs_user")
assert user["group"] == "Docs push"
assert user["label"] == "Docs user"
assert user["secret"] is False
assert user["masked"] is False
assert user["value"] == ""
assert user["has_value"] is False
# (b) a PUT stores it (200, the RAW string in app_settings) and the
# GET reports has_value=true with the plaintext NOWHERE in the raw
# body (asserted on response.text, not the parsed model).
r = admin_client.put(f"{API}/docs_token", json={"value": "bor-docs-tok-123"})
assert r.status_code == 200, r.text
assert r.json()["masked"] is True
assert r.json()["value"] is None
assert r.json()["has_value"] is True
assert r.json()["source"] == "db"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'docs_token'")
).scalar() == "bor-docs-tok-123" # stored, exactly as sent
assert get_settings().docs_token == "bor-docs-tok-123" # live, no restart
r = admin_client.get(API)
assert r.status_code == 200
item = _item(r.json(), "docs_token")
assert item["has_value"] is True
assert item["value"] is None
assert item["db_value"] is None
assert "bor-docs-tok-123" not in r.text # NOWHERE in the raw body
# (c) a blank PUT is a 422 (a blank token would self-lockout —
# DELETE is the explicit clear) and the stored value is unchanged.
r = admin_client.put(f"{API}/docs_token", json={"value": " "})
assert r.status_code == 422
assert "BOR_DOCS_TOKEN must not be blank" in r.json()["detail"]
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'docs_token'")
).scalar() == "bor-docs-tok-123" # unchanged
# (d) a DELETE unsets it: the row is gone and the effective value is
# back to the env value (the pinned "" — the anonymous state, C5).
r = admin_client.delete(f"{API}/docs_token")
assert r.status_code == 200, r.text
assert r.json()["source"] == "env" # the pinned empty BOR_DOCS_TOKEN
assert r.json()["has_override"] is False
assert r.json()["has_value"] is False
assert get_settings().docs_token == "" # the override is gone
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'docs_token'")
).scalar() == 0 # the row is gone
def test_docs_repo_refuses_embedded_credentials(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 173 (task 02, C4): a ``docs_repo`` PUT with a
``://<user>:<pass>@`` segment is a 422 that NAMES the Docs user /
Docs token fields (asserted in the detail, not just present) and
stores NOTHING — the leak cannot be re-created from the UI. Clean
``https://`` URLs, local paths and ``ssh://`` remotes pass
untouched; a ``user:``-looking subdomain (no trailing ``@``) does
NOT false-positive. A legacy ``.env`` value with embedded
credentials is NOT a PUT — the task-04 startup migration handles
that path."""
# A deterministic base: explicit empty/valid process values for the
# docs settings — the process env ranks above the operator's local
# ``.env`` (the conftest pin's reasoning), so the base is the code
# defaults no matter what the machine's ``.env`` carries (a local
# ``BOR_DOCS_REPO`` with embedded credentials is EXACTLY the leak
# class under test — it must not leak into this pin).
monkeypatch.setenv("BOR_DOCS_REPO", "")
monkeypatch.setenv("BOR_DOCS_BRANCH", "bor-docs")
invalidate_settings()
# The 422: names the replacement fields, stores nothing.
r = admin_client.put(
f"{API}/docs_repo", json={"value": "https://u:pass@gitea.example.com/o/r.git"}
)
assert r.status_code == 422
detail = r.json()["detail"]
assert "Docs user" in detail
assert "Docs token" in detail
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
# The failed write did not disturb the live view.
assert get_settings().docs_repo == ""
# Clean https:// URL: passes (the branch validator is satisfied by
# the non-empty docs_branch default — the repo is set, the token
# shape is valid).
r = admin_client.put(f"{API}/docs_repo", json={"value": "https://gitea.example.com/o/r.git"})
assert r.status_code == 200, r.text
assert r.json()["db_value"] == "https://gitea.example.com/o/r.git"
assert get_settings().docs_repo == "https://gitea.example.com/o/r.git"
assert admin_client.delete(f"{API}/docs_repo").status_code == 200
# A local path: untouched.
r = admin_client.put(f"{API}/docs_repo", json={"value": "/srv/docs-repo"})
assert r.status_code == 200, r.text
assert r.json()["db_value"] == "/srv/docs-repo"
assert admin_client.delete(f"{API}/docs_repo").status_code == 200
# A ``user:``-looking subdomain: the trailing ``@`` requirement keeps
# the regex from false-positiving — passes untouched.
r = admin_client.put(
f"{API}/docs_repo", json={"value": "https://sub.domain.example.com/r.git"}
)
assert r.status_code == 200, r.text
assert r.json()["db_value"] == "https://sub.domain.example.com/r.git"
assert admin_client.delete(f"{API}/docs_repo").status_code == 200
# An ssh:// remote (credentials via SSH keys, never in the URL):
# untouched.
r = admin_client.put(
f"{API}/docs_repo", json={"value": "ssh://git@gitea.example.com/o/r.git"}
)
assert r.status_code == 200, r.text
assert r.json()["db_value"] == "ssh://git@gitea.example.com/o/r.git"
assert admin_client.delete(f"{API}/docs_repo").status_code == 200
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
+203
View File
@@ -0,0 +1,203 @@
"""Integration: the phase-173 (task 04, C3) legacy-credentials
migration, end-to-end against the real ``app_settings`` table.
The shape (the 00_phase / task-04 pins):
* seed env ``BOR_DOCS_REPO=https://u:pass@host/r.git`` (no DB rows)
→ run the migration (the direct call — the ``create_app()`` wiring
is covered by the unit seam in ``test_main.py``) → the migration
returns ``True`` and the three rows land in ``app_settings``
(clean repo / ``docs_user`` / ``docs_token``);
* DB rows SHADOW the env — the effective settings resolve the clean
URL + the credentials FROM the DB (the env value still embeds the
credentials, ``.env`` is never rewritten: the process env is
untouched by the migration);
* idempotent — a second run is a no-op (``False``, the rows
byte-identical);
* the settings API reports the clean ``docs_repo`` (``source="db"``)
and the ``docs_token`` masked (``secret=true``, ``value=null``,
``has_value=true``) with the token text NOWHERE in the raw body —
the leak is gone from the page.
The module's teardown leaves NO row behind (the ``test_admin_settings_
api`` discipline — a surviving row would retune every later suite
through the overlay).
Real Postgres (``podman compose up -d db``); no LLM involved.
Requires: podman compose up -d db
"""
from __future__ import annotations
import os
from collections.abc import Iterator
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.api import chat as chat_api
from app.config import get_settings, invalidate_settings
from app.core import settings_overlay
from app.core.settings_migrate import migrate_legacy_docs_creds
from app.core.settings_overlay import register_invalidator, resolve_settings
from app.main import app as fastapi_app
from tests.conftest import ADMIN_PASSWORD
API = "/api/admin/settings"
#: The legacy value the migration splits (the production shape — the
#: token is distinctive so the "absent from the raw body" pin cannot
#: false-positive on a catalogue label).
_LEGACY = "https://u:tok-legacy-123@host/r.git"
@pytest.fixture(autouse=True)
def clean_state(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> Iterator[None]:
"""A clean ``app_settings`` table + the overlay switch ON + a
fresh overlay/hook slate (the ``test_admin_settings_api`` idiom:
reset the process-wide hook list and re-wire what
``create_app()`` wires). Teardown deletes the rows + invalidates
+ clears the cache — NO row may survive this module (a surviving
row would retune every later suite through the overlay)."""
settings_overlay.reset_state_for_tests()
register_invalidator(chat_api.reset_runtime_caches)
db.execute(text("DELETE FROM app_settings"))
db.commit()
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
monkeypatch.setenv("BOR_DOCS_REPO", "")
invalidate_settings()
get_settings.cache_clear()
yield
db.execute(text("DELETE FROM app_settings"))
db.commit()
invalidate_settings()
get_settings.cache_clear()
settings_overlay.reset_state_for_tests()
def _table(db: Session) -> dict[str, str]:
"""The whole ``app_settings`` table (the display fact, never a
query surface — one SELECT)."""
return {
key: value
for key, value in db.execute(
text("SELECT key, value FROM app_settings")
)
}
def _admin_client() -> TestClient:
"""A fresh client signed in as the admin (the ``_admin_client``
pattern from ``test_ui_settings_api``)."""
client = TestClient(fastapi_app)
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login failed: {r.status_code}"
return client
def _item(body: dict, key: str) -> dict:
"""One catalogue item by key (the GET body's ``items`` list)."""
for item in body["items"]:
if item["key"] == key:
return item
raise AssertionError(f"catalogue item missing: {key}")
def test_legacy_env_is_split_into_db_rows(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The end-to-end shape: seed env (no DB rows) → migration runs →
the three rows land, the effective settings resolve the clean URL
+ the credentials FROM THE DB (the rows shadow the env), the env
is untouched (``.env`` is never rewritten), and the second run is
a no-op (idempotent)."""
monkeypatch.setenv("BOR_DOCS_REPO", _LEGACY)
assert _table(db) == {} # no DB rows (the fixture's clean slate)
assert migrate_legacy_docs_creds() is True
# the three rows — the clean repo + user + token, and NOTHING
# else (the leak class dies: the repo row no longer carries the
# credentials).
rows = _table(db)
assert rows == {
"docs_repo": "https://host/r.git",
"docs_user": "u",
"docs_token": "tok-legacy-123",
}
# the effective settings resolve FROM THE DB (invalidate first —
# the module import primed the cache before this module's rows):
# the clean URL shadows the credentials-bearing env value.
invalidate_settings()
effective = resolve_settings()
assert effective.docs_repo == "https://host/r.git"
assert effective.docs_user == "u"
assert effective.docs_token == "tok-legacy-123"
# the env is untouched — the migration never rewrites the env
# (``.env`` stays the base; the DB rows shadow it).
assert os.environ["BOR_DOCS_REPO"] == _LEGACY
# idempotent: the ``docs_token`` row is present now — a second
# run is a no-op, the rows byte-identical.
assert migrate_legacy_docs_creds() is False
assert _table(db) == rows
def test_migrated_values_are_masked_on_the_settings_api(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Completion criterion (the leak is gone from the page): after a
(b)-shape migration the settings API reports the clean
``docs_repo`` (``source="db"``) + ``docs_token``
``secret=true`` / ``value=null`` / ``has_value=true`` (masked)
with the token text NOWHERE in the raw response body."""
monkeypatch.setenv("BOR_DOCS_REPO", _LEGACY)
assert migrate_legacy_docs_creds() is True
invalidate_settings()
admin = _admin_client()
r = admin.get(API)
assert r.status_code == 200, r.text
repo = _item(r.json(), "docs_repo")
assert repo["value"] == "https://host/r.git" # the clean URL
assert repo["source"] == "db" # the DB row is in effect
assert repo["db_value"] == "https://host/r.git"
token = _item(r.json(), "docs_token")
assert token["secret"] is True
assert token["masked"] is True
assert token["value"] is None # write-only: never the value
assert token["db_value"] is None # a masked row never exposes text
assert token["has_value"] is True # a value IS in effect
assert token["has_override"] is True # the DB row
assert token["source"] == "db"
user = _item(r.json(), "docs_user")
assert user["value"] == "u"
assert user["source"] == "db"
# the core pin: the token text is NOWHERE in the raw body (the
# leak class this phase kills — asserted on response.text, not
# the parsed model; the value is distinctive so the substring
# check cannot false-positive on a catalogue label).
assert "tok-legacy-123" not in r.text
def test_migration_is_a_noop_for_a_clean_env(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The no-op shape end-to-end: a clean env URL (the pre-phase
state) → ``False``, the table stays empty (nothing half-
migrates, nothing written)."""
monkeypatch.setenv("BOR_DOCS_REPO", "https://host/r.git")
assert migrate_legacy_docs_creds() is False
assert _table(db) == {}
invalidate_settings()
assert resolve_settings().docs_repo == "https://host/r.git"
+52 -3
View File
@@ -6,8 +6,8 @@ opens a file (``.env`` is never written), never shells out and never
``eval``s a value.
The DB-driven behavior (the four routes, the candidate-set validation,
the auth matrix) is pinned in
``tests/integration/test_admin_settings_api.py``.
the auth matrix, the phase-173 ``docs_repo`` guard on the PUT path)
is pinned in ``tests/integration/test_admin_settings_api.py``.
"""
from __future__ import annotations
@@ -15,8 +15,14 @@ import pathlib
import re
import pytest
from fastapi import HTTPException
from app.api.admin_settings import _display, _field_type, _item
from app.api.admin_settings import (
_display,
_field_type,
_item,
_refuse_embedded_credentials,
)
from app.config import Settings
from app.core.settings_overlay import LIST_FIELDS
@@ -172,3 +178,46 @@ def test_module_never_writes_files_or_shells_out() -> None:
assert "os.system" not in source
assert re.search(r"\beval\s*\(", source) is None
assert "import shutil" not in source and "shutil." not in source
def test_refuse_embedded_credentials_fires_only_for_docs_repo() -> None:
"""Phase 173 (task 02, C4): the guard 422s for ``docs_repo`` with a
``://<user>:<pass>@`` segment — the detail NAMES the Docs user /
Docs token fields — and passes everything else: other keys (only
``docs_repo`` is guarded), clean ``https://`` URLs, local paths,
``ssh://`` remotes, and a ``user:``-looking subdomain (the
trailing ``@`` requirement keeps it from false-positiving).
An empty user OR an empty pass (``://:pass@`` / ``://u:@``) does
NOT match — the ``+`` quantifiers require both to be non-empty.
"""
# The 422: names the replacement fields.
with pytest.raises(HTTPException) as excinfo:
_refuse_embedded_credentials(
"docs_repo", "https://u:pass@gitea.example.com/o/r.git"
)
assert excinfo.value.status_code == 422
assert "Docs user" in excinfo.value.detail
assert "Docs token" in excinfo.value.detail
# Also a ``git://``-style URL with embedded credentials (the regex
# is scheme-agnostic: it keys on ``://<user>:<pass>@``).
with pytest.raises(HTTPException):
_refuse_embedded_credentials("docs_repo", "git://u:tok@host.example.com/r.git")
# Other keys: never guarded (``docs_user`` may hold any plain text).
_refuse_embedded_credentials("docs_user", "https://u:pass@host.example.com/r.git")
_refuse_embedded_credentials("llm_api_key", "://u:pass@")
# Clean shapes pass: https URL, local path, ssh remote.
_refuse_embedded_credentials("docs_repo", "https://gitea.example.com/o/r.git")
_refuse_embedded_credentials("docs_repo", "/srv/docs-repo")
_refuse_embedded_credentials("docs_repo", "ssh://git@gitea.example.com/o/r.git")
# A subdomain that looks like ``user:`` (no trailing ``@``).
_refuse_embedded_credentials("docs_repo", "https://sub.domain.example.com/r.git")
# Empty user OR empty pass: the ``+`` quantifiers require both
# non-empty — no match (a bare ``://`` is not the leak class).
_refuse_embedded_credentials("docs_repo", "https://:pass@host.example.com/r.git")
_refuse_embedded_credentials("docs_repo", "https://u:@host.example.com/r.git")
# A ``:`` after the ``@`` segment (a URL with no credentials at
# all) and an empty value.
_refuse_embedded_credentials("docs_repo", "https://host.example.com/r.git:3000")
_refuse_embedded_credentials("docs_repo", "")
+36
View File
@@ -665,6 +665,42 @@ def test_docs_branch_env_override(monkeypatch: pytest.MonkeyPatch) -> None:
assert s.docs_base_branch == "master"
def test_docs_creds_defaults_are_empty(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 173 (task 01, C5): no credentials by default — the push is
anonymous/URL-credential-free, exactly as before this phase."""
monkeypatch.delenv("BOR_DOCS_USER", raising=False)
monkeypatch.delenv("BOR_DOCS_TOKEN", raising=False)
s = _settings()
assert s.docs_user == ""
assert s.docs_token == ""
def test_docs_creds_env_overrides(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 173 (task 01): ``BOR_DOCS_USER`` / ``BOR_DOCS_TOKEN``
resolve verbatim — plain ``str``, no coercion."""
monkeypatch.setenv("BOR_DOCS_USER", "alice")
monkeypatch.setenv("BOR_DOCS_TOKEN", "tok")
s = _settings()
assert s.docs_user == "alice"
assert s.docs_token == "tok"
def test_docs_creds_empty_env_stays_empty(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 173 (task 01, C5): an empty (or whitespace) env value stays
``""`` / verbatim — no coercion, no normalization away from empty.
The anonymous state must stay expressible: the settings must NOT
strip the token, or an explicit empty would become a set credential.
(Whitespace is stored verbatim, like ``docs_repo``; the *push*
service decides the empty-vs-set semantics, not the settings.)"""
monkeypatch.setenv("BOR_DOCS_USER", "")
monkeypatch.setenv("BOR_DOCS_TOKEN", "")
s = _settings()
assert s.docs_user == ""
assert s.docs_token == ""
monkeypatch.setenv("BOR_DOCS_TOKEN", " ")
assert _settings().docs_token == " "
def test_docs_work_dir_env_override_is_raw_string(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("BOR_DOCS_WORK_DIR", "/data/bor/docs")
s = _settings()
+169
View File
@@ -9,19 +9,33 @@ ever involved.
The module skips (``pytest.skip``) when ``git --version`` fails — a
machine without git must not see hard failures.
Phase 173 (C2/C5) adds a mocked-``run_git`` section: the FULL 10-step
sequence is exercised against a fake git (no real repo) to pin the
credential transport — no token → every step gets ``env=None``
(byte-identical to the pre-phase call); a token → every step gets
``GIT_CURL_USERNAME`` / ``GIT_CURL_PASSWORD`` / ``GIT_TERMINAL_PROMPT=0``
merged over a controlled ``os.environ`` (``PATH`` survives the
subprocess env-replacement trap); a whitespace-only token = empty;
and a failing step still raises :class:`DocsPushError` with git's
stderr, credential-agnostic.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
from typing import TypedDict
import pytest
import app.core.docs_push as docs_push_mod
from app.core.docs_push import DocsPushError, push_document
from scripts.git_sync import GitSyncError
BASE = "main"
BRANCH = "bor-docs"
REL = "docs/note.md"
CLEAN_REPO = "https://git.example.com/docs.git" # phase 173: the credential-free form
IDENTITY = ("-c", "commit.gpgsign=false", "-c", "user.name=Test", "-c", "user.email=t@example.com")
@@ -198,3 +212,158 @@ def test_unsafe_rel_path_is_refused_before_any_git(bare_repo: Path, tmp_path: Pa
)
# No checkout was even attempted.
assert not (tmp_path / "w").exists()
# --- phase 173: credentials on the wire (C2/C5, mocked git) ---------------
class _GitCall(TypedDict):
"""One recorded ``run_git`` call (argv / cwd / env)."""
argv: list[str]
cwd: Path
env: dict[str, str] | None
def _fake_run_git(
monkeypatch: pytest.MonkeyPatch, push_stderr: str = ""
) -> list[_GitCall]:
"""Monkeypatch ``app.core.docs_push.run_git``; record each call
(argv / cwd / env).
The FULL 10-step sequence runs: clone (the work dir is absent) →
the base fetch → the ``FETCH_HEAD`` capture → the branch fetch
(ALWAYS raises — the branch does not exist on the remote yet,
the swallowed path) → the branch probe → ``checkout -B`` → add →
commit → push (raises when *push_stderr* is set — the
un-swallowed failure, task 03 5d) → the ``HEAD`` capture. Returns
``(BRANCH, "b" * 40)`` on success.
"""
calls: list[_GitCall] = []
def fake_run(
argv: list[str], cwd: Path, env: dict[str, str] | None = None
) -> str:
calls.append({"argv": list(argv), "cwd": cwd, "env": env})
key = tuple(argv)
if key == ("git", "fetch", "--depth", "100", "origin", BRANCH):
raise GitSyncError(
f"git fetch --depth 100 origin {BRANCH} failed (exit 128): "
f"fatal: couldn't find remote ref {BRANCH}"
)
if push_stderr and key == ("git", "push", "origin", BRANCH):
raise GitSyncError(
f"git push origin {BRANCH} failed (exit 128): {push_stderr}"
)
if key[:2] == ("git", "rev-parse"):
return "a" * 40 if key[-1] == "FETCH_HEAD" else "b" * 40
return ""
monkeypatch.setattr(docs_push_mod, "run_git", fake_run)
return calls
def _args(work: Path) -> dict[str, str]:
return dict(
repo=CLEAN_REPO,
base_branch=BASE,
branch=BRANCH,
work_dir=str(work),
rel_path=REL,
content="# Note\n",
commit_message="docs: note",
)
def test_no_token_env_is_none_on_every_step(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Task 03 5a (C5 byte-identity pin): default args (no token) →
EVERY git step of the full sequence gets ``env=None`` — the
pre-phase ``subprocess.run`` shape, unchanged."""
work = tmp_path / "work" # absent — the clone step runs too
calls = _fake_run_git(monkeypatch)
branch, sha = push_document(**_args(work))
assert (branch, sha) == (BRANCH, "b" * 40)
assert len(calls) == 10 # clone, fetch×2, rev-parse×2, branch, checkout, add, commit, push
for call in calls:
assert call["env"] is None
# The CLEAN URL stays in the argv (the credential never appears
# because it is not passed at all).
assert calls[0]["argv"][-2] == CLEAN_REPO
def test_token_env_on_every_step_merged_over_environ(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Task 03 5b (C2): token set → EVERY git step of a successful push
gets the 3-var credential env MERGED OVER a controlled
``os.environ`` — ``PATH`` survives the subprocess
env-replacement trap, the credential never lands in the argv,
and the clean URL stays in it."""
monkeypatch.setenv("PATH", "/usr/bin")
monkeypatch.setenv("BOR_TEST_ENV", "sentinel")
work = tmp_path / "work"
calls = _fake_run_git(monkeypatch)
push_document(**_args(work), user="alice", token="s3cret")
assert len(calls) == 10
for call in calls:
env = call["env"]
assert isinstance(env, dict)
assert env["GIT_CURL_USERNAME"] == "alice"
assert env["GIT_CURL_PASSWORD"] == "s3cret"
assert env["GIT_TERMINAL_PROMPT"] == "0"
assert env["PATH"] == "/usr/bin" # the merge-over-os.environ pin
assert env["BOR_TEST_ENV"] == "sentinel" # the parent env survives
# C2: the credential travels ONLY in the environment.
assert "alice" not in call["argv"]
assert "s3cret" not in call["argv"]
# The clean URL is in the argv, untouched.
assert CLEAN_REPO in calls[0]["argv"]
def test_whitespace_only_token_is_treated_as_empty(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Task 03 5c (the ``.strip()`` rule): a whitespace-only token is
the no-credentials state → every step gets ``env=None``."""
calls = _fake_run_git(monkeypatch)
push_document(**_args(tmp_path / "work"), user="alice", token=" ")
assert len(calls) == 10
for call in calls:
assert call["env"] is None
def test_failing_step_raises_docs_push_error_with_and_without_credentials(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Task 03 5d: a failing step (the push) still raises
:class:`DocsPushError` with git's stderr — the error path is
credential-agnostic (with and without the credential env); the
failing step itself still carries the credential env when a token
is set."""
stderr = "fatal: rejected (simulated non-fast-forward)"
# without credentials (C5 — the pre-phase error path)
calls = _fake_run_git(monkeypatch, push_stderr=stderr)
with pytest.raises(DocsPushError) as excinfo:
push_document(**_args(tmp_path / "w1"))
msg = str(excinfo.value)
assert f"git push origin {BRANCH} failed (exit 128): {stderr}" in msg
assert calls[-1]["env"] is None # the failed push step — no env
# with credentials (the error carries the full stderr for logs;
# the API layer's sanitize_error stays the defense-in-depth)
calls = _fake_run_git(monkeypatch, push_stderr=stderr)
with pytest.raises(DocsPushError) as excinfo:
push_document(**_args(tmp_path / "w2"), user="alice", token="s3cret")
msg = str(excinfo.value)
assert f"git push origin {BRANCH} failed (exit 128): {stderr}" in msg
assert isinstance(calls[-1]["env"], dict)
assert calls[-1]["env"]["GIT_CURL_PASSWORD"] == "s3cret"
+30
View File
@@ -297,6 +297,36 @@ def test_run_git_captures_and_returns_stdout(
assert len(calls) == 1
def test_run_git_default_env_is_none_and_reaches_subprocess(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 173 (C5 byte-identity pin): the default ``env=None`` is
passed through verbatim to ``subprocess.run`` — the pre-phase call
shape, so every sync-path invocation is unchanged."""
calls = _fake_run(monkeypatch, stdout="ok")
assert git_sync.run_git(["git", "version"], cwd=tmp_path) == "ok"
assert len(calls) == 1
assert "env" in calls[0]
assert calls[0]["env"] is None
def test_run_git_explicit_env_reaches_subprocess_verbatim(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 173 (C2): an explicit env dict reaches ``subprocess.run``
verbatim — ``run_git`` neither merges over nor mutates the parent
environment (the env-replacement trap is the caller's contract;
the docs-push sequence does its own ``os.environ`` merge)."""
calls = _fake_run(monkeypatch, stdout="ok")
env = {"A": "b"}
assert git_sync.run_git(["git", "version"], cwd=tmp_path, env=env) == "ok"
assert len(calls) == 1
assert calls[0]["env"] is env # the SAME object — passed through verbatim
assert calls[0]["env"] == {"A": "b"}
# --- phase 135: the branch pin (the argv matrix) ----------------------------
+33
View File
@@ -1,6 +1,8 @@
"""Unit tests: app factory edge cases (no live DB needed)."""
from __future__ import annotations
import logging
from fastapi.testclient import TestClient
import app.main as main_mod
@@ -51,3 +53,34 @@ def test_create_app_boots_with_the_switch_on(monkeypatch) -> None:
# settings, and the hook slate is fresh for them too.
settings_overlay.reset_state_for_tests()
get_settings.cache_clear()
def test_create_app_migration_failure_warns_and_boots(
monkeypatch, caplog
) -> None:
"""Phase 173 (task 04, C3): a migration failure logs a WARNING
and never blocks startup — the app still boots (with the
pre-phase behavior: the migration is a convenience, not a
dependency). The seam: the migration is monkeypatched to raise;
the real migration is a no-op in a DB-less unit environment
(the overlay's fail-soft read + the conftest-pinned empty
``BOR_DOCS_REPO``), so the boot path is otherwise unchanged."""
settings_overlay.reset_state_for_tests()
def _boom() -> bool:
raise RuntimeError("migration hiccup")
monkeypatch.setattr(main_mod, "migrate_legacy_docs_creds", _boom)
with caplog.at_level(logging.WARNING, logger="app"):
app2 = main_mod.create_app()
assert app2 is not None
warnings = [
record.getMessage() for record in caplog.records
if record.levelno == logging.WARNING
]
assert "settings migration failed (RuntimeError)" in warnings
# restore: the hook slate is fresh for the later suites, and the
# cache clear keeps a boot-time row set from leaking into their
# settings.
settings_overlay.reset_state_for_tests()
get_settings.cache_clear()
+36 -1
View File
@@ -54,12 +54,15 @@ def test_every_meta_carries_its_own_key() -> None:
def test_secrets_are_exactly_the_masked_set() -> None:
# LOCKED A4: write-only, masked — the 3 named secrets +
# database_url (read-only, but its URL carries credentials).
# database_url (read-only, but its URL carries credentials) +
# docs_token (phase 173, C1: the docs-push remote token reuses the
# write-only/masked mechanism, llm_api_key-style).
assert set(SECRETS) == {
"llm_api_key",
"admin_password",
"session_secret",
"database_url",
"docs_token",
}
@@ -108,6 +111,38 @@ def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None:
assert "llm_thinking_level" not in NON_OVERRIDABLE
def test_docs_user_is_a_plain_live_overridable_row() -> None:
"""Phase 173 (task 01, C1/C5): ``docs_user`` sits in ``Docs push``
with the ``Docs user`` label and is plain — not ``secret`` (a
plain username), not ``restart_required`` (the push service reads
the setting per push via ``get_settings()`` — live), overridable
(the overlay is generic over ``str`` fields). An empty value is a
valid state (C5: the anonymous push)."""
meta = CATALOG["docs_user"]
assert meta.group == "Docs push"
assert meta.label == "Docs user"
assert meta.secret is False
assert "docs_user" not in SECRETS
assert "docs_user" not in RESTART_REQUIRED
assert "docs_user" not in NON_OVERRIDABLE
def test_docs_token_is_a_secret_live_overridable_row() -> None:
"""Phase 173 (task 01, C1): ``docs_token`` has the ``Docs token``
label and ``secret=True`` — write-only end-to-end via the EXISTING
derived mechanism (``SECRETS`` picks it up automatically; the API
reports ``is_set`` + ``value=null``, a blank PUT 422s, the panel
renders it masked, zero ``settings.js`` code). Live (the push
service reads it per push), overridable (generic ``str`` overlay)."""
meta = CATALOG["docs_token"]
assert meta.group == "Docs push"
assert meta.label == "Docs token"
assert meta.secret is True
assert "docs_token" in SECRETS
assert "docs_token" not in RESTART_REQUIRED
assert "docs_token" not in NON_OVERRIDABLE
def test_labels_nonempty_and_unique() -> None:
labels = [meta.label for meta in CATALOG.values()]
assert all(label.strip() for label in labels), "empty label in the catalogue"
+270
View File
@@ -0,0 +1,270 @@
"""Phase 173 (task 04, C3): the one-time startup migration of a
legacy credentials-bearing ``docs_repo``.
Pure/unit level — no live DB: the ``app_settings`` read is a stubbed
overlay engine (the ``test_settings_overlay`` fake-engine idiom), the
write path is monkeypatched (``settings_migrate._write_rows`` — it
captures the rows AND mirrors them into the stubbed read store, so
``resolve_settings()`` resolves the DB rows the migration wrote — the
DB-shadowing pin), the env base comes from ``os.environ`` (``tests/
conftest.py`` pins ``BOR_DOCS_REPO=`` / ``BOR_SETTINGS_DB=0`` — the
fixture flips the switch on per test), and the overlay state is
reset by ``reset_state_for_tests()``.
"""
from __future__ import annotations
import logging
from collections.abc import Iterator
import pytest
from app.core import settings_migrate, settings_overlay
from app.core.settings_overlay import resolve_settings
#: The ONE INFO line the migration logs when it acts (AGENTS rule 10).
_MIGRATED_LINE = (
"settings: migrated embedded docs-push credentials out of "
"docs_repo into docs_user/docs_token"
)
class _FakeStore:
"""A controllable ``app_settings`` store: the read (a fake overlay
engine) and the write (the monkeypatched ``_write_rows``) share
one dict — a migration write is visible to ``resolve_settings()``
on the next resolution (the DB-shadowing pin)."""
def __init__(self, rows: dict[str, str] | None = None) -> None:
self.rows = dict(rows or {})
self.writes: list[dict[str, str]] = []
def install(self, monkeypatch: pytest.MonkeyPatch) -> None:
store = self
class _Rows:
def __iter__(self) -> object:
return iter(sorted(store.rows.items()))
class _FakeConn:
def execute(self, _stmt: object) -> _Rows:
return _Rows()
def __enter__(self) -> _FakeConn:
return self
def __exit__(self, *exc: object) -> bool:
return False
class _FakeEngine:
def connect(self) -> _FakeConn: # type: ignore[no-untyped-def]
return _FakeConn()
monkeypatch.setattr(
settings_overlay, "_get_engine", lambda: _FakeEngine()
)
def _fake_write(rows: dict[str, str]) -> None:
store.writes.append(dict(rows))
store.rows.update(rows)
monkeypatch.setattr(settings_migrate, "_write_rows", _fake_write)
@pytest.fixture(autouse=True)
def _clean_state(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
settings_overlay.reset_state_for_tests()
monkeypatch.setenv("BOR_SETTINGS_DB", "1")
monkeypatch.setenv("BOR_DOCS_REPO", "")
monkeypatch.setenv("BOR_DOCS_USER", "")
monkeypatch.setenv("BOR_DOCS_TOKEN", "")
yield
settings_overlay.reset_state_for_tests()
def _migrated_lines(caplog: pytest.LogCaptureFixture) -> list[str]:
"""The ``app.settings`` records that carry the migration line."""
return [
record.getMessage()
for record in caplog.records
if _MIGRATED_LINE in record.getMessage()
]
# ---------------------------------------------------------------------------
# (a) the no-op shapes — clean value, malformed segment
# ---------------------------------------------------------------------------
def test_clean_env_url_is_a_noop(monkeypatch: pytest.MonkeyPatch) -> None:
"""(a) a clean env URL with no overrides: ``False``, no rows
written (the pre-phase behavior stands untouched)."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://host/r.git")
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {}
def test_clean_db_override_is_a_noop(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""(a, the DB-source variant) a clean URL arriving as a DB
override (the effective value, not the env): still a no-op."""
store = _FakeStore({"docs_repo": "https://host/r.git"})
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://elsewhere/r.git")
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {"docs_repo": "https://host/r.git"}
def test_malformed_empty_user_is_a_noop(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""(d) a malformed ``://:@…`` (empty user) is NOT the legacy
shape: ``False``, no rows, no log — the value stays as-is
(nothing breaks, nothing half-migrates)."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://:@host/r.git")
with caplog.at_level(logging.INFO, logger="app.settings"):
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {}
assert _migrated_lines(caplog) == []
def test_malformed_empty_pass_is_a_noop(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""(d, the other malformation) an empty pass (``://u:@…``):
the same no-op contract — no rows, no log."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://u:@host/r.git")
with caplog.at_level(logging.INFO, logger="app.settings"):
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {}
assert _migrated_lines(caplog) == []
def test_subdomain_is_not_a_false_positive(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A ``://user:pass@``-shaped SUBDOMAIN (no trailing ``@`` in
the URL) must not match — the character classes + the trailing
``@`` requirement do that job (the same contract the PUT guard
pins)."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://sub.domain.example.com/r.git")
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
# ---------------------------------------------------------------------------
# (b) the migration shape — split, DB-shadowing, the one INFO line
# ---------------------------------------------------------------------------
def test_legacy_env_url_is_split_into_three_rows(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""(b) a legacy env URL with no overrides: ``True``, exactly the
three rows (clean repo / user / token), and
``resolve_settings()`` resolves the clean URL + the credentials
FROM THE DB rows (they shadow the env — the env value still
embeds the credentials, the effective value does not)."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://u:pass@host/r.git")
with caplog.at_level(logging.INFO, logger="app.settings"):
assert settings_migrate.migrate_legacy_docs_creds() is True
assert store.rows == {
"docs_repo": "https://host/r.git",
"docs_user": "u",
"docs_token": "pass",
}
assert len(store.writes) == 1 # one write call, one row set
# DB rows SHADOW the env: the effective value is the clean URL.
effective = resolve_settings()
assert effective.docs_repo == "https://host/r.git"
assert effective.docs_user == "u"
assert effective.docs_token == "pass"
# (e) the INFO line is emitted exactly once — and it carries no
# value (the log discipline, A4).
assert _migrated_lines(caplog) == [_MIGRATED_LINE]
for line in _migrated_lines(caplog):
assert "pass" not in line
def test_second_run_is_a_silent_noop(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""(b + idempotency) after a migration wrote the three rows, the
second run finds the ``docs_token`` row → ``False``, no rows
written, NO second INFO line (the one-line pin, e)."""
store = _FakeStore()
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://u:pass@host/r.git")
with caplog.at_level(logging.INFO, logger="app.settings"):
assert settings_migrate.migrate_legacy_docs_creds() is True
assert settings_migrate.migrate_legacy_docs_creds() is False
assert len(store.writes) == 1
assert _migrated_lines(caplog) == [_MIGRATED_LINE]
def test_legacy_value_from_a_db_override_is_split(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""(b, the DB-source variant) the legacy value arriving as a DB
``docs_repo`` override (the effective value): the same split —
the override row is rewritten clean, the credentials join the
new rows."""
store = _FakeStore({"docs_repo": "https://u:pass@host/r.git"})
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "") # env empty — DB is the source
assert settings_migrate.migrate_legacy_docs_creds() is True
assert store.rows == {
"docs_repo": "https://host/r.git",
"docs_user": "u",
"docs_token": "pass",
}
effective = resolve_settings()
assert effective.docs_repo == "https://host/r.git"
# ---------------------------------------------------------------------------
# (c) the clobber guard — an existing docs_token override stands
# ---------------------------------------------------------------------------
def test_existing_token_override_is_never_clobbered(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""(c) a legacy env URL while a ``docs_token`` override already
exists (owner-set, or a previous migration): ``False`` — the
pre-existing rows are untouched (no clobber)."""
store = _FakeStore({"docs_token": "owner-token"})
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://u:pass@host/r.git")
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {"docs_token": "owner-token"} # byte-identical
# the effective value resolves the owner's token, not the URL's.
assert resolve_settings().docs_token == "owner-token"
def test_existing_token_override_with_clean_repo_is_a_noop(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""(c, the clean variant) a token override with a clean repo
(env or DB): ``False`` — nothing to do, nothing touched."""
store = _FakeStore({"docs_token": "owner-token"})
store.install(monkeypatch)
monkeypatch.setenv("BOR_DOCS_REPO", "https://host/r.git")
assert settings_migrate.migrate_legacy_docs_creds() is False
assert store.writes == []
assert store.rows == {"docs_token": "owner-token"}