phase: 173_docs_push_creds_split
Build and Push Containers / build-and-push-app (push) Successful in 1m41s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All gates green. Final report:

**Phase 173 — final verification pass (all tasks already complete in `complete/`)**

- Verified the full implementation is in the working tree: `docs_user`/`docs_token` settings + catalogue rows (task 01), `docs_repo` PUT 422 guard + `docs_token` write-only pin (task 02), `run_git(env)` + `push_document(user, token)` credential env (task 03), `settings_migrate.py` wired into `create_app()` (task 04), E2E suite (task 05).
- **Completion criteria:**
  1. GET `/api/admin/settings` lists `docs_user` (plain) + `docs_token` (secret, `value=null`), token never in any body — ✅ `test_docs_token_is_write_only_masked_end_to_end` + E2E sentinel-absent-from-DOM
  2. PUT `docs_repo` with `://user:pass@` → 422 naming the fields; clean URL passes — ✅ integration + E2E UI-side
  3. Token set → `GIT_CURL_USERNAME`/`GIT_CURL_PASSWORD`/`GIT_TERMINAL_PROMPT=0` on every git step; empty → `env=None` — ✅ mocked-git unit tests
  4. Legacy env split once at startup, idempotent, one INFO line, `.env` untouched — ✅ `test_settings_migrate.py` (4 shapes) + `test_docs_push_migrate.py`
  5. Full suite green, coverage >90%, lint clean — ✅
  6. E2E isolated: `test_docs_push_creds.py` 4/4; regression set (panel + layout + response_to_docs) 21/21 — ✅
  7. Commit + phase move — harness's job (per instructions, not mine).
  8. README/AGENTS consistent — ✅ (below)
- Tests/lint: `uv run pytest` exit 0 (4112 passed) · `uv run pytest --cov=app --cov-report=term-missing` TOTAL 99% · `uv run ruff check .` clean · `uv run pyright` 0 errors
- Docs: README.md (incl. Docs-push config section, guard/migration paragraph, `.env.example` block) and AGENTS.md verified consistent in full; fixed two stale docstrings in `app/config.py` + `app/core/settings_catalog.py` (`is_set` → the API's actual `has_value` field).
- No live-infrastructure changes (no live step in this phase). No next pending phase — `todo/` held only 173.
This commit is contained in:
2026-10-10 16:47:39 -04:00
parent f27106a638
commit a86caec821
56 changed files with 3177 additions and 39 deletions
+49 -10
View File
@@ -22,6 +22,7 @@ stderr (the ``GitSyncError`` style), and the checkout under
"""
from __future__ import annotations
import os
from pathlib import Path
from scripts.git_sync import GitSyncError, run_git
@@ -61,9 +62,28 @@ def push_document(
rel_path: str,
content: str,
commit_message: str,
user: str = "",
token: str = "",
) -> tuple[str, str]:
"""Commit *content* at *rel_path* and push (ff-only) to *branch*.
*user* / *token* are the docs-push credentials (phase 173, C2):
when *token* is non-blank, EVERY git step of the sequence runs with
git's documented HTTP credential environment —
``GIT_CURL_USERNAME`` / ``GIT_CURL_PASSWORD`` (+
``GIT_TERMINAL_PROMPT=0`` so a headless server fails fast instead
of hanging on a prompt) — merged over ``os.environ`` (the
subprocess env-replacement trap: a dict REPLACES the parent
environment, so the merge is what keeps ``PATH`` and friends).
The credential travels only in that environment — it never appears
in the argv (the clean *repo* URL stays in the argv untouched),
so it is absent from the process list. ``GIT_CURL_*`` is inert for
``ssh://`` / local-path remotes (accepted — the remote decides).
**C5:** when *token* is blank (the default — an anonymous push to
a public or credential-free remote, exactly the pre-phase state),
every ``run_git`` call keeps ``env=None`` and the invocations are
byte-identical to the pre-phase ones.
Step sequence (each step via :func:`scripts.git_sync.run_git`;
``cwd=work_dir`` once the checkout exists):
@@ -108,6 +128,19 @@ def push_document(
"""
_refuse_unsafe_rel_path(rel_path)
work = Path(work_dir).expanduser()
# C5: blank token → env stays None → every invocation below is
# byte-identical to the pre-phase one.
git_env: dict[str, str] | None = None
if token.strip():
# C2: git's documented HTTP credential env — never in the argv.
# GIT_TERMINAL_PROMPT=0 fails fast instead of hanging the headless
# server on a credential prompt.
git_env = {
**os.environ,
"GIT_CURL_USERNAME": user,
"GIT_CURL_PASSWORD": token,
"GIT_TERMINAL_PROMPT": "0",
}
try:
if not (work / ".git").exists():
work.parent.mkdir(parents=True, exist_ok=True)
@@ -123,25 +156,30 @@ def push_document(
str(work),
],
cwd=work.parent,
env=git_env,
)
run_git(["git", "fetch", "--depth", "1", "origin", base_branch], cwd=work)
run_git(["git", "fetch", "--depth", "1", "origin", base_branch], cwd=work, env=git_env)
# Capture the fetched base tip now — a *failed* fetch clears
# FETCH_HEAD, so the symbolic ref cannot be reused later.
base_sha = run_git(["git", "rev-parse", "FETCH_HEAD"], cwd=work).strip()
base_sha = run_git(["git", "rev-parse", "FETCH_HEAD"], cwd=work, env=git_env).strip()
try:
run_git(["git", "fetch", "--depth", "100", "origin", branch], cwd=work)
branch_sha = run_git(["git", "rev-parse", "FETCH_HEAD"], cwd=work).strip()
run_git(
["git", "fetch", "--depth", "100", "origin", branch], cwd=work, env=git_env
)
branch_sha = run_git(
["git", "rev-parse", "FETCH_HEAD"], cwd=work, env=git_env
).strip()
except GitSyncError:
branch_sha = None # the branch does not exist on the remote yet
if run_git(["git", "branch", "--list", branch], cwd=work).strip():
run_git(["git", "checkout", branch], cwd=work)
if run_git(["git", "branch", "--list", branch], cwd=work, env=git_env).strip():
run_git(["git", "checkout", branch], cwd=work, env=git_env)
else:
anchor = branch_sha if branch_sha is not None else base_sha
run_git(["git", "checkout", "-B", branch, anchor], cwd=work)
run_git(["git", "checkout", "-B", branch, anchor], cwd=work, env=git_env)
target = work / rel_path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(content, encoding="utf-8")
run_git(["git", "add", "--", rel_path], cwd=work)
run_git(["git", "add", "--", rel_path], cwd=work, env=git_env)
run_git(
[
"git",
@@ -156,9 +194,10 @@ def push_document(
commit_message,
],
cwd=work,
env=git_env,
)
run_git(["git", "push", "origin", branch], cwd=work)
sha = run_git(["git", "rev-parse", "HEAD"], cwd=work).strip()
run_git(["git", "push", "origin", branch], cwd=work, env=git_env)
sha = run_git(["git", "rev-parse", "HEAD"], cwd=work, env=git_env).strip()
except GitSyncError as err:
raise DocsPushError(str(err)) from err
return branch, sha
+7 -2
View File
@@ -11,8 +11,9 @@ to do with each of the ``Settings`` fields:
presentation metadata (``group`` + ``label``) and its capability
flags:
- ``secret`` — write-only (LOCKED A4): the API reports ``is_set``,
never the plaintext; a saved value is masked on read.
- ``secret`` — write-only (LOCKED A4): the API reports ``has_value``
(and ``masked=True`` / ``value=None``), never the plaintext; a
saved value is masked on read.
- ``restart_required`` — boot-wired (LOCKED A4): the middleware /
engine / logging / static mount read it once at boot, so the
override applies on the NEXT boot (the panel shows the badge).
@@ -301,6 +302,10 @@ CATALOG: dict[str, SettingMeta] = {
"pdf_page_dir": SettingMeta("pdf_page_dir", "PDF", "PDF page dir"),
# --- Docs push ---
"docs_repo": SettingMeta("docs_repo", "Docs push", "Docs repo"),
"docs_user": SettingMeta("docs_user", "Docs push", "Docs user"),
"docs_token": SettingMeta(
"docs_token", "Docs push", "Docs token", secret=True
),
"docs_branch": SettingMeta("docs_branch", "Docs push", "Docs branch"),
"docs_base_branch": SettingMeta(
"docs_base_branch", "Docs push", "Docs base branch"
+116
View File
@@ -0,0 +1,116 @@
"""One-time settings migrations (phase 173).
``migrate_legacy_docs_creds`` splits a legacy ``://user:pass@``
``docs_repo`` value into ``docs_repo`` (clean) + ``docs_user`` +
``docs_token`` — DB rows that SHADOW the env (the settings model's
standing rule: the database wins over the environment). ``.env`` is
never read for writing or rewritten. Idempotent: it runs only while
no ``docs_token`` override exists, so owner-set values are never
clobbered. One INFO line when it acts (AGENTS rule 10).
The read side rides the overlay (``load_overrides`` /
``resolve_settings`` — the same fail-soft engine); the write side is
the PUT route's own write (an ``app_settings`` upsert through the app
session — the overlay exposes no write path). A malformed
``://user:pass@`` segment (an empty user OR pass) is NOT the legacy
shape: the value stays as-is (the pre-phase behavior — nothing breaks,
nothing half-migrates), no rows, no log.
Called ONCE from ``create_app()`` (the composition root, task 04
step 2), after the DB is available; a failure there logs a WARNING
and never blocks startup — the app boots with the pre-phase behavior
(the migration is a convenience, not a dependency).
"""
from __future__ import annotations
import logging
import re
from sqlalchemy import func
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
from app.core.settings_overlay import load_overrides, resolve_settings
from app.db import SessionLocal
from app.models import AppSetting
logger = logging.getLogger("app.settings")
#: ``://<user>:<pass>@`` — the legacy credentials-bearing docs-push
#: remote (the phase-173 leak class). BOTH groups must be non-empty:
#: ``://:@…`` / ``://u:@…`` are malformed, not the legacy shape. The
#: same regex the PUT guard uses (``admin_settings._EMBEDDED_CREDS``)
#: plus the capture groups; a ``://user:pass@`` subdomain cannot match
#: (the character classes exclude ``/`` and the trailing ``@``
#: requirement does the job).
_EMBEDDED_CREDS = re.compile(r"://([^/@:]+):([^@/]+)@")
def _write_rows(rows: dict[str, str]) -> None:
"""Write one row per (key, value) into ``app_settings`` — the PUT
route's own write (INSERT … ON CONFLICT (key) DO UPDATE, the row's
``updated_at`` moves), one session, one commit. The overlay
exposes no write path, so the direct upsert IS the write path.
``.env`` is never touched (LOCKED A1 — the DB rows shadow the env;
the env file stays the base)."""
db: Session = SessionLocal()
try:
for key, value in rows.items():
stmt = pg_insert(AppSetting).values(key=key, value=value)
stmt = stmt.on_conflict_do_update(
index_elements=["key"],
set_={"value": stmt.excluded.value, "updated_at": func.now()},
)
db.execute(stmt)
db.commit()
finally:
db.close()
def migrate_legacy_docs_creds() -> bool:
"""The phase-173 C3 migration — split a legacy
``://user:pass@`` ``docs_repo`` into three ``app_settings`` rows.
(1) read the ``app_settings`` overrides (the overlay's state);
(2) the target is the EFFECTIVE ``docs_repo`` (an override if
present, else the env value — what ``resolve_settings()``
resolves to); (3) match ``://<user>:<pass>@`` with BOTH groups
non-empty — otherwise no-op (a malformed segment stays as-is, the
pre-phase value: nothing breaks, nothing half-migrates); (4) a
``docs_token`` override already present → no-op (idempotent —
owner-set values are never clobbered, and a previous run of this
migration counts as one); (5) otherwise write the three rows
(clean repo / user / token — the PUT route's own write) and log
ONE INFO line. Returns ``True`` when it acted, ``False`` for
every no-op shape. ``.env`` is never read for writing or
rewritten: the DB rows shadow the env (the settings model's
standing rule).
"""
effective = resolve_settings().docs_repo
match = _EMBEDDED_CREDS.search(effective)
if match is None:
# (3) clean value OR malformed ``user:pass@`` (empty user or
# pass) — the pre-phase value stays as-is; no rows, no log.
return False
if "docs_token" in load_overrides():
# (4) idempotent: a token override (owner-set, or this
# migration's own previous run) is never clobbered.
return False
user, token = match.group(1), match.group(2)
# (5) the ``user:pass@`` segment removed, the ``://`` separator
# KEPT — ``https://u:pass@host/r.git`` → ``https://host/r.git``.
clean = effective.replace(match.group(0), "://", 1)
_write_rows(
{
"docs_repo": clean,
"docs_user": user,
"docs_token": token,
}
)
# ONE INFO line when it acts (AGENTS rule 10 — the value is never
# in the line, A4's log discipline).
logger.info(
"settings: migrated embedded docs-push credentials out of "
"docs_repo into docs_user/docs_token"
)
return True