phase: 173_docs_push_creds_split
All gates green. Final report: **Phase 173 — final verification pass (all tasks already complete in `complete/`)** - Verified the full implementation is in the working tree: `docs_user`/`docs_token` settings + catalogue rows (task 01), `docs_repo` PUT 422 guard + `docs_token` write-only pin (task 02), `run_git(env)` + `push_document(user, token)` credential env (task 03), `settings_migrate.py` wired into `create_app()` (task 04), E2E suite (task 05). - **Completion criteria:** 1. GET `/api/admin/settings` lists `docs_user` (plain) + `docs_token` (secret, `value=null`), token never in any body — ✅ `test_docs_token_is_write_only_masked_end_to_end` + E2E sentinel-absent-from-DOM 2. PUT `docs_repo` with `://user:pass@` → 422 naming the fields; clean URL passes — ✅ integration + E2E UI-side 3. Token set → `GIT_CURL_USERNAME`/`GIT_CURL_PASSWORD`/`GIT_TERMINAL_PROMPT=0` on every git step; empty → `env=None` — ✅ mocked-git unit tests 4. Legacy env split once at startup, idempotent, one INFO line, `.env` untouched — ✅ `test_settings_migrate.py` (4 shapes) + `test_docs_push_migrate.py` 5. Full suite green, coverage >90%, lint clean — ✅ 6. E2E isolated: `test_docs_push_creds.py` 4/4; regression set (panel + layout + response_to_docs) 21/21 — ✅ 7. Commit + phase move — harness's job (per instructions, not mine). 8. README/AGENTS consistent — ✅ (below) - Tests/lint: `uv run pytest` exit 0 (4112 passed) · `uv run pytest --cov=app --cov-report=term-missing` TOTAL 99% · `uv run ruff check .` clean · `uv run pyright` 0 errors - Docs: README.md (incl. Docs-push config section, guard/migration paragraph, `.env.example` block) and AGENTS.md verified consistent in full; fixed two stale docstrings in `app/config.py` + `app/core/settings_catalog.py` (`is_set` → the API's actual `has_value` field). - No live-infrastructure changes (no live step in this phase). No next pending phase — `todo/` held only 173.
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
# Phase 173 — Docs-push credentials: split the username/token out of `BOR_DOCS_REPO`
|
||||
|
||||
**Source:** owner audit of the production admin Settings panel (2026-10-10): `BOR_DOCS_REPO` is a plain-text row whose production value is a git URL with an embedded username + Gitea token (`https://user:token@host/…`) — a live credential leak on the settings page, the raw HTML, and the settings API.
|
||||
**Story:** n/a (owner-directed, chat-confirmed 2026-10-10)
|
||||
**Context:** `app/config.py` — the Docs push section (`docs_repo` L539, `docs_branch` L545, `docs_base_branch` L549, `docs_work_dir` L554; the branch validator L831–846 requires non-empty branches only while `docs_repo` is set; `docs_configured` L902–906 is the 409 inert-switch on `docs_repo` alone — **untouched**), `.env.example` L229–234 (the Docs push block — commented optional vars), `app/core/settings_catalog.py` L303–308 (the `Docs push` group: `docs_repo` L303, `docs_branch` L304, `docs_base_branch` L305–307, `docs_work_dir` L308) and the existing secret mechanism (`llm_api_key` L142 `secret=True`; `SECRETS` is DERIVED from `CATALOG` — a `secret=True` entry is write-only end-to-end: the API reports `is_set` + `value=null` (`app/api/admin_settings.py` L147), a blank PUT 422s (L273), and the phase-168/171 panel renders the row masked as `(stored)` with zero `settings.js` code), `scripts/git_sync.py` L320 (`run_git(argv, cwd)` — the ONLY git invocation site, A11; `subprocess.run(argv, cwd=cwd, capture_output=True, text=True)` L331 — no env today), `app/core/docs_push.py` (`push_document` — clone L104–116 / fetch / checkout / add / commit / push, every step via `run_git`; `DocsPushError` carries the full git stderr for logs), `app/api/doc_drafts.py` L299–316 (the push call site; phase 84 `sanitize_error` already strips echoed credentials from the 502 response — defense-in-depth, stays), `app/core/settings_overlay.py` (`load_overrides()` L93 — the raw `app_settings` DB map; `resolve_settings()` L239; `reset_state_for_tests()` L274), `app/main.py` L98–245 (`create_app()` the composition root; `app = create_app()` L245), `tests/unit/test_config.py`, `tests/unit/test_settings_catalog.py`, `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (the generic set/reset/live-apply contract), `tests/e2e/test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 168/171 — the regression set), `tests/e2e/test_response_to_docs.py` (phase 59 — the docs-push E2E).
|
||||
|
||||
## Objective
|
||||
The docs-push remote's credentials live in their own settings — `BOR_DOCS_USER` (plain) and `BOR_DOCS_TOKEN` (write-only, masked like `BOR_LLM_API_KEY`) — while `BOR_DOCS_REPO` holds a clean URL. At push time the credentials travel as git's `GIT_CURL_USERNAME` / `GIT_CURL_PASSWORD` environment variables (never in the command line / process list), a one-time startup migration splits any legacy `://user:pass@` value into the new settings (DB rows shadow the env, `.env` is never rewritten), and a PUT guard refuses a credentials-bearing repo URL so the leak cannot be re-created from the UI. Anonymous pushes (no token) stay byte-identical to today.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **D1 — the three settings (task 01).** `docs_user: str = ""` and `docs_token: str = ""` in the Docs push section of `app/config.py` (directly after `docs_repo` L539, before `docs_branch` L545). Plain `str`, no validator (empty = "no credentials" is a valid state — the repo may be public/anonymous, exactly like today). `.env.example` L231–234 block gains `# BOR_DOCS_USER=` / `# BOR_DOCS_TOKEN=` (commented — optional, house style of the block). CATALOG: `docs_user` plain, `docs_token` **`secret=True`** (inserted directly after `docs_repo` L303, before `docs_branch` L304). The `SECRETS` derivation + the generic overlay + the phase-168/171 panel make `docs_token` a write-only, masked row with zero extra code (C1). No frontend code changes anywhere in this phase.
|
||||
- **D2 — the PUT guard (task 02).** In `app/api/admin_settings.py`, the `PUT` route refuses `docs_repo` values that embed credentials — a `://<user>:<pass>@` segment (regex `://[^/@:]+:[^@/]+@`) — with a 422 whose detail names the replacement (use the Docs user / Docs token fields, keep the URL clean). Everything else about the row is unchanged (it stays overridable, plain). `docs_token`'s API behavior is the EXISTING generic secret contract (GET `is_set`/`value=null`, blank PUT 422, DELETE resets) — task 02 pins it with integration assertions, it needs no new code.
|
||||
- **D3 — credentials on the wire (task 03).** `run_git` gains an optional `env: dict[str, str] | None = None` passed straight to `subprocess.run` — **the subprocess env-replacement trap**: when set, the dict REPLACES the parent environment, so `docs_push` must build it as `{**os.environ, …}` (a test asserts `PATH` survives). `push_document` gains `user: str = ""` / `token: str = ""`; when `token.strip()` is non-empty, EVERY git step of the sequence (clone, both fetches, checkout, add, commit, push) gets `env = {**os.environ, "GIT_CURL_USERNAME": user, "GIT_CURL_PASSWORD": token, "GIT_TERMINAL_PROMPT": "0"}` (git's documented HTTP credential env; `GIT_TERMINAL_PROMPT=0` fails fast instead of hanging a headless server on a prompt); when the token is empty, `env` stays `None` and the invocations are byte-identical to today (C5). The clean URL stays in the argv. `app/api/doc_drafts.py` passes `settings.docs_user` / `settings.docs_token`. `GIT_CURL_*` only takes effect for `http(s)` remotes; for `ssh://` / local-path remotes it is inert (accepted). Phase 84's `sanitize_error` on the 502 response is untouched (defense-in-depth).
|
||||
- **D4 — the legacy migration (task 04).** New module `app/core/settings_migrate.py` with `migrate_legacy_docs_creds() -> bool`: read the `app_settings` overrides; take the EFFECTIVE `docs_repo` (DB override if present, else env — i.e. `resolve_settings().docs_repo`); if it matches `://<user>:<pass>@` (both non-empty) AND there is no `docs_token` override yet, write three DB rows — `docs_repo` = the credential-stripped URL, `docs_user` = user, `docs_token` = pass — and log one INFO line (`settings: migrated embedded docs-push credentials out of docs_repo into docs_user/docs_token`); any other shape → no-op. Idempotent (a `docs_token` row already present → never clobber owner-set values). Called ONCE from `create_app()` (the composition root) after the DB is available. `.env` is never read for writing or rewritten — the DB rows shadow the env (the settings model's standing rule). Unit-tested against the overlay module's state + `reset_state_for_tests()`.
|
||||
- **D5 — the E2E (task 05).** `tests/e2e/test_docs_push_creds.py` — the phase's dedicated Playwright suite (AGENTS rule 4): the Docs push group shows a `Docs repo` text row, a `Docs user` text row, and a `Docs token` write-only row rendered masked (`(stored)`, not pre-filled, `type=password`); typing a token, saving, and reloading keeps the row masked and the typed token text is **absent from the entire page DOM** (the core assertion — the leak class this phase kills); resetting the token unsets it. The regression set (`test_admin_settings_panel.py`, `test_admin_settings_layout.py`, `test_response_to_docs.py`) stays green.
|
||||
- **NOT touched:** `frontend/` (no file), the `app_settings` table shape, `settings_overlay.py` internals, `docs_configured` / the 409 inert-switch, the branch validator, `git_sync.clone_or_pull` (sync path — no credential settings exist for it; out of scope), `sanitize_error` (phase 84), every other setting.
|
||||
|
||||
## Dependencies
|
||||
— (none: the app is fully functional at the head of `complete/`; this phase builds on the committed phase-59 docs-push, phase-168/171 settings machinery, and phase-84 error sanitization)
|
||||
|
||||
## Tasks
|
||||
1. `01_config_and_catalog.md` — `Settings.docs_user` / `docs_token` (default `""`) + the two CATALOG rows (`docs_token` `secret=True`) + `.env.example`; the config/catalog unit tests.
|
||||
2. `02_api_masking_and_guards.md` — integration pin of the `docs_token` write-only contract + the new `docs_repo` embedded-credentials PUT guard (422); the guard's unit/integration tests.
|
||||
3. `03_run_git_env_and_push.md` — `run_git(env=None)` + `push_document(user, token)` credential env on every git step (merged over `os.environ`; `None` when the token is empty) + the `doc_drafts.py` call site; the mocked-git unit tests.
|
||||
4. `04_legacy_creds_migration.md` — `app/core/settings_migrate.py` (idempotent, DB-shadowing, one log line) wired into `create_app()`; unit + integration tests.
|
||||
5. `05_e2e_docs_creds_masking.md` — `tests/e2e/test_docs_push_creds.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_config.py` — `docs_user` / `docs_token` default `""`, env overrides, empty env stays `""` (no coercion). `tests/unit/test_settings_catalog.py` — the completeness pin holds with the two new fields; `docs_token` is in `SECRETS`, `docs_user` is in none of the derived sets. `tests/unit/test_git_sync.py` (or the existing `run_git` test home) — `env=None` leaves the subprocess call byte-identical; a passed env reaches `subprocess.run` verbatim. `tests/unit/test_docs_push.py` — token empty → every `run_git` call gets `env=None`; token set → every git step gets the 3-var env merged over `os.environ` (`PATH` present); `DocsPushError` propagation unchanged. `tests/unit/test_settings_migrate.py` — the 4 migration shapes (D4).
|
||||
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — `docs_token` in `GET /api/admin/settings` (group `Docs push`, `secret=true`, `value=null`, `is_set` flips on PUT); PUT stores / DELETE resets / blank PUT 422 (the generic contract pinned for the new key); `docs_repo` PUT with `://user:pass@` → 422 naming the user/token fields, clean URLs pass. `tests/integration/test_docs_push_migrate.py` — seeded env `BOR_DOCS_REPO=https://u:pass@host/r.git`, no DB rows → after the migration runs, `resolve_settings().docs_repo` is `https://host/r.git`, `docs_user`/`docs_token` resolve from the DB; a second run is a no-op.
|
||||
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_docs_push_creds.py`, run **in isolation** — per D5.
|
||||
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/admin/settings` (admin) lists `docs_user` (plain) and `docs_token` (secret, `value=null`) in the `Docs push` group; the token value is NEVER in any response body.
|
||||
- [ ] `PUT /api/admin/settings` for `docs_repo` with a `://user:pass@` value → 422 naming the Docs user / Docs token fields; a clean URL passes.
|
||||
- [ ] With `docs_token` set, a docs push invokes git with `GIT_CURL_USERNAME` / `GIT_CURL_PASSWORD` / `GIT_TERMINAL_PROMPT=0` in the environment and a clean URL in the argv (unit-asserted via mocked `run_git`); with it empty, the `run_git` calls are byte-identical to today (`env=None`).
|
||||
- [ ] A legacy `://user:pass@` env value is split once at startup into `app_settings` rows (clean repo + user + token), `.env` untouched, a second run is a no-op; the INFO migration line is logged exactly once.
|
||||
- [ ] `uv run pytest` green (100% — every test in the project, pre-existing included), `app/` coverage >90%, ruff + pyright clean.
|
||||
- [ ] `uv run pytest tests/e2e/test_docs_push_creds.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` + `test_response_to_docs.py` green in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(settings): split docs-push credentials (BOR_DOCS_USER/BOR_DOCS_TOKEN) out of the repo URL — masked, env-wired, legacy auto-migration`); the phase dir moves to `.agents/phases/complete/`.
|
||||
- [ ] `README.md` and `AGENTS.md` are accurate and consistent with the project as it will stand after the phase (the final pass reviews both in full and fixes every inconsistency found, including pre-existing ones).
|
||||
|
||||
## Documentation & Consistency (mandatory)
|
||||
The final pass reviews `README.md` and `AGENTS.md` in their entirety against the project as it will stand after the phase and fixes every inconsistency found — including pre-existing ones not caused by the phase. Tasks update both files whenever their own changes are user-visible (the `.env.example` block is the user-visible docs surface for this phase; the README's docs-push section gains the two new vars + a line that credentials in the repo URL are no longer supported).
|
||||
|
||||
## Locked decisions (owner-confirmed 2026-10-10, chat)
|
||||
- **C1 — the split shape:** `docs_repo` (clean URL) + `docs_user` (plain `str`) + `docs_token` (`secret=True` in the CATALOG — reuses the existing write-only/masked mechanism end-to-end, `llm_api_key`-style). No new UI code: the panel renders both rows from the catalogue.
|
||||
- **C2 — the credential transport:** `GIT_CURL_USERNAME` / `GIT_CURL_PASSWORD` (+ `GIT_TERMINAL_PROMPT=0`) environment variables on the push sequence — never in the git argv / process list; applied only when `docs_token` is non-empty.
|
||||
- **C3 — the migration:** one-time at startup (`create_app()`), effective-value based, DB rows shadow the env, `.env` never rewritten, idempotent, one INFO log line.
|
||||
- **C4 — the guard:** `PUT docs_repo` with embedded credentials → 422 (the leak cannot be re-created via the UI).
|
||||
- **C5 — no behavior change for anonymous pushes:** empty `docs_token` → the git invocations are byte-identical to today (`env=None`).
|
||||
@@ -0,0 +1,55 @@
|
||||
# Task 01 — The `docs_user` / `docs_token` settings + the catalogue rows + `.env.example`
|
||||
|
||||
**Phase:** `173_docs_push_creds_split` · **Source:** owner audit of the production admin Settings panel (2026-10-10): `BOR_DOCS_REPO` leaks its embedded Gitea token in plain text; the credentials get their own settings.
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The two new settings exist in `app/config.py` (default `""`), in the admin catalogue (so the build's completeness pin stays green — a field without a `SettingMeta` fails the build, and the per-task gate runs the full suite), and in `.env.example`, so `BOR_DOCS_USER` / `BOR_DOCS_TOKEN` are first-class, documented knobs before anything consumes them. `docs_token` is `secret=True` — write-only, masked end-to-end via the EXISTING mechanism (C1); nothing else in the phase touches `frontend/`.
|
||||
|
||||
## Work
|
||||
1. `app/config.py` — in the Docs push section, **directly after** the `docs_repo` block (L539) and **before** `docs_branch` (L545), add TWO fields, house docstring style (env var named, default, consumption point, off rule):
|
||||
```python
|
||||
#: Docs-push remote username (phase 173, ``BOR_DOCS_USER``) — paired
|
||||
#: with ``docs_token``; travels to git as the ``GIT_CURL_USERNAME``
|
||||
#: environment variable (``app/core/docs_push.py``, task 03) — never
|
||||
#: in the command line. Empty (default) = no credentials: the push
|
||||
#: is anonymous/URL-credential-free, exactly as before this phase.
|
||||
docs_user: str = ""
|
||||
#: Docs-push remote token (phase 173, ``BOR_DOCS_TOKEN``) — write-only
|
||||
#: (catalogue ``secret=True``): the API reports ``is_set`` and never
|
||||
#: the value; the admin panel renders it masked. Travels to git as
|
||||
#: ``GIT_CURL_PASSWORD`` (``app/core/docs_push.py``, task 03).
|
||||
#: Empty (default) = the credential env is not set at all.
|
||||
docs_token: str = ""
|
||||
```
|
||||
No validator, no coercion — plain `str`; an empty value is a valid state (C1/C5). The L831–846 branch validator (branches required only while `docs_repo` is set) and `docs_configured` (L902–906) stay byte-identical.
|
||||
2. `.env.example` — in the Docs push block (L229–234), directly under `# BOR_DOCS_REPO=/path/to/docs-repo` (L231), add:
|
||||
```
|
||||
# BOR_DOCS_USER=
|
||||
# BOR_DOCS_TOKEN=
|
||||
```
|
||||
(commented — optional, the block's house style; add one line to the block's comment: credentials in the repo URL are no longer supported — use BOR_DOCS_USER/BOR_DOCS_TOKEN).
|
||||
3. `app/core/settings_catalog.py` — in the `Docs push` group, **directly after** the `docs_repo` entry (L303) and **before** `docs_branch` (L304), add TWO entries:
|
||||
```python
|
||||
"docs_user": SettingMeta("docs_user", "Docs push", "Docs user"),
|
||||
"docs_token": SettingMeta(
|
||||
"docs_token", "Docs push", "Docs token", secret=True
|
||||
),
|
||||
```
|
||||
Nothing else — `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` are DERIVED from `CATALOG` (module docstring): `docs_token` lands in `SECRETS` automatically, `docs_user` in none. Both are plain `str` → the overlay is generic (live, no restart). The panel row falls out of the catalogue with zero `settings.js` code (C1 — the phase-168/171 machinery renders a `secret=True` row masked as `(stored)`, `llm_api_key`-style).
|
||||
4. `tests/unit/test_settings_catalog.py` — the completeness pin (`set(CATALOG) == set(Settings.model_fields)`) must stay green with the two new fields; add the house-style assertions: `CATALOG["docs_user"]` sits in group `Docs push` with label `Docs user` and is in **none** of the derived sets; `CATALOG["docs_token"]` has label `Docs token`, `secret=True`, and **is** in `SECRETS`.
|
||||
5. `tests/unit/test_config.py` — add the house-style cases: (a) the DEFAULTS are `""` / `""` (no env set); (b) the env override — `BOR_DOCS_USER=alice` / `BOR_DOCS_TOKEN=tok` resolve to `"alice"` / `"tok"`; (c) the empty env values stay `""` (no coercion, no normalization away from empty — C5's anonymous state stays expressible).
|
||||
|
||||
- ASSUMPTIONS: C1 (phase-locked) — the split shape; C5 (phase-locked) — empty token = no credential env, so the settings must NOT strip/coerce away the empty value at load.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_config.py` + `tests/unit/test_settings_catalog.py` green (`uv run pytest tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov`).
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest -k "docs" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` green: defaults `""`, env overrides, empty stays `""`, the completeness pin holds, `docs_token` in `SECRETS`, `docs_user` in none.
|
||||
- [ ] `grep -n "BOR_DOCS_USER" .env.example` shows the line under `# BOR_DOCS_REPO=…` (commented).
|
||||
- [ ] `GET /api/admin/settings` (admin) lists `docs_user` + `docs_token` in the `Docs push` group — `docs_token` as `secret=true` with `value=null` (a quick curl, or the task-02 integration test).
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (no other `Settings` field touched; `frontend/` untouched; `docs_configured` + the branch validator byte-identical).
|
||||
@@ -0,0 +1,45 @@
|
||||
# Task 02 — The `docs_token` write-only contract (pinned) + the `docs_repo` embedded-credentials guard
|
||||
|
||||
**Phase:** `173_docs_push_creds_split` · **Source:** C2/C4 (phase-locked) — the token is write-only end-to-end via the existing mechanism; a credentials-bearing repo URL is refused on the way in so the leak cannot be re-created from the UI.
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The `docs_token` row's API behavior is the EXISTING generic secret contract — task 02 pins it with integration assertions for the new key (no new code expected). The ONE piece of new logic: the `PUT` route refuses `docs_repo` values with embedded credentials (C4), pointing the operator at the Docs user / Docs token fields.
|
||||
|
||||
## Work
|
||||
1. `app/api/admin_settings.py` — in the `PUT` route, **before** the existing blank-secret 422 check (L273) or alongside it, add the `docs_repo` guard (a small module-level helper keeps the route readable — house style):
|
||||
```python
|
||||
#: A git URL with embedded credentials (``://<user>:<pass>@``) — the
|
||||
#: phase-173 leak class. Refused for ``docs_repo`` on the way in:
|
||||
#: the credentials belong in ``docs_user`` / ``docs_token`` (C4).
|
||||
_EMBEDDED_CREDS = re.compile(r"://[^/@:]+:[^@/]+@")
|
||||
|
||||
def _refuse_embedded_credentials(key: str, value: str) -> None:
|
||||
if key == "docs_repo" and _EMBEDDED_CREDS.search(value):
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
"docs_repo must not contain embedded credentials — "
|
||||
"keep the URL clean and set them in the Docs user / "
|
||||
"Docs token fields instead"
|
||||
),
|
||||
)
|
||||
```
|
||||
Call it at the top of the per-key validation in the PUT route (after the payload is parsed, before store). Only `docs_repo` is guarded — `ssh://` URLs, local paths, and clean `https://` URLs pass untouched. (A legacy `.env` value with embedded credentials is NOT a PUT — the task-04 startup migration handles that path; the guard covers only new writes.)
|
||||
2. `tests/integration/test_admin_settings_api.py` (or `test_admin_settings_live.py` — match the file that owns the secret-row contract; the phase-172 pin is the template) — add the house-style cases for the new key:
|
||||
- `GET /api/admin/settings` (admin): `docs_token` in group `Docs push`, `secret=true`, `value=null`, `is_set=false`; `docs_user` plain, `value=""`.
|
||||
- `PUT docs_token` with a value → 200; `GET` → `is_set=true`, `value=null` (the value is **never** in the response body — assert the token text is absent from the raw response JSON).
|
||||
- `PUT docs_token` with a blank value → 422 (the existing self-lockout guard — pinned for the new key).
|
||||
- `DELETE docs_token` → `is_set=false` (reset to env — `""` by default).
|
||||
3. `tests/integration/test_admin_settings_api.py` — the guard's integration cases: `PUT docs_repo` `https://u:pass@gitea.example.com/o/r.git` → 422, detail mentions the Docs user / Docs token fields; `PUT docs_repo` `https://gitea.example.com/o/r.git` → 200 (clean passes); `PUT docs_repo` a local path (`/srv/docs-repo`) → 200 (untouched); a `user:` segment that is NOT `://user:pass@` (e.g. `https://sub.domain.example.com/r.git`) → 200 (the regex must not false-positive on subdomains — the `@` requirement does that job).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the cases above green (`uv run pytest tests/integration/test_admin_settings_api.py -v --no-cov`).
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green — the guard helper must be covered by the 422 + pass cases).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/integration/test_admin_settings_api.py -v --no-cov` green, including: `docs_token` GET `is_set`/`value=null` (token text absent from the raw JSON), PUT store, blank PUT 422, DELETE reset; `docs_repo` embedded-creds 422 + clean/local/subdomain 200s.
|
||||
- [ ] The 422 detail names the Docs user / Docs token fields (asserted in the test, not just present).
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (no other key's PUT path touched; the blank-secret guard at L273 byte-identical).
|
||||
@@ -0,0 +1,47 @@
|
||||
# Task 03 — Credentials on the wire: `run_git(env=…)` + `push_document(user, token)`
|
||||
|
||||
**Phase:** `173_docs_push_creds_split` · **Source:** C2/C5 (phase-locked) — credentials travel as `GIT_CURL_USERNAME` / `GIT_CURL_PASSWORD` env (never in the argv / process list); an empty token leaves the invocations byte-identical to today.
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The docs-push git sequence authenticates with the settings' user/token via git's documented HTTP credential environment, without the credential ever appearing in the command line, and without changing a single invocation when no token is set (C5).
|
||||
|
||||
## Work
|
||||
1. `scripts/git_sync.py` — `run_git` (L320) gains one optional parameter:
|
||||
```python
|
||||
def run_git(argv: list[str], cwd: Path, env: dict[str, str] | None = None) -> str:
|
||||
…
|
||||
proc = subprocess.run(
|
||||
argv, cwd=cwd, capture_output=True, text=True, env=env
|
||||
)
|
||||
```
|
||||
`None` (the default, and the sync path's only usage) is byte-identical to today. **The subprocess env-replacement trap:** when `env` is a dict, it REPLACES the parent environment — the caller must build a complete environment (task 03's `docs_push` does the merge; a unit test pins `PATH` survival). Update the docstring: the `env` parameter is passed through verbatim; `None` = inherit.
|
||||
2. `app/core/docs_push.py` — `push_document` gains `user: str = ""` / `token: str = ""` (positional order: appended AFTER `commit_message`, so every existing call site keeps working; update the docstring with the C2 transport + the C5 no-token rule). At the top of the sequence:
|
||||
```python
|
||||
git_env: dict[str, str] | None = None
|
||||
if token.strip():
|
||||
# C2: git's documented HTTP credential env — never in the argv.
|
||||
# GIT_TERMINAL_PROMPT=0 fails fast instead of hanging the headless
|
||||
# server on a credential prompt.
|
||||
git_env = {
|
||||
**os.environ,
|
||||
"GIT_CURL_USERNAME": user,
|
||||
"GIT_CURL_PASSWORD": token,
|
||||
"GIT_TERMINAL_PROMPT": "0",
|
||||
}
|
||||
```
|
||||
Pass `env=git_env` to **every** `run_git` call in the sequence (clone, both fetches, `rev-parse` ×2, `branch --list`, checkout, add, commit, push — `rev-parse`/`branch` may take it too; uniformity over cleverness). The clean URL stays in the argv untouched. `DocsPushError` propagation is unchanged (the error still carries the full git stderr for logs; phase 84's `sanitize_error` on the API response stays as defense-in-depth). `GIT_CURL_*` is inert for `ssh://` / local-path remotes (accepted — the remote decides).
|
||||
3. `app/api/doc_drafts.py` — the push call site (L306–314) passes `user=settings.docs_user, token=settings.docs_token`. Nothing else changes (the 409 inert-switch on `docs_configured` stays repo-based — an empty token is a valid anonymous-push state, C5).
|
||||
4. `tests/unit/test_git_sync.py` (or the existing `run_git` test home) — (a) default: `run_git(["git", "version"], cwd)` → `subprocess.run` called with `env=None` (byte-identical to today); (b) an explicit `env={"A": "b"}` reaches `subprocess.run` verbatim.
|
||||
5. `tests/unit/test_docs_push.py` — (a) **no token** (default args) → every mocked `run_git` call gets `env=None` (the C5 byte-identity pin); (b) **token set** → every git step of a successful push gets an env with `GIT_CURL_USERNAME=user`, `GIT_CURL_PASSWORD=token`, `GIT_TERMINAL_PROMPT="0"`, **and** `PATH` present (the merge-over-`os.environ` pin — run with a controlled `os.environ` in the test); (c) whitespace-only token (`" "`) → treated as empty (the `.strip()` rule) → `env=None`; (d) a failing step still raises `DocsPushError` with git's stderr — with and without credentials (the error path is credential-agnostic).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the cases above green (`uv run pytest tests/unit/test_git_sync.py tests/unit/test_docs_push.py -v --no-cov` — adjust file names to the existing homes if they differ).
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green — the `token.strip()` branches are covered by 5a/5b/5c).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_docs_push.py -v --no-cov` green: no-token `env=None` (all steps), token-set 3-var env merged over `os.environ` (`PATH` present, all steps), whitespace-only token = empty, `DocsPushError` propagation unchanged.
|
||||
- [ ] The sync path (`clone_or_pull`) is byte-identical (it never passes `env` — verify by reading: no `env=` at its `run_git` call sites).
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (the git argv of an anonymous push is byte-identical to pre-phase; `sanitize_error` untouched).
|
||||
@@ -0,0 +1,45 @@
|
||||
# Task 04 — The one-time startup migration of a legacy `://user:pass@` repo URL
|
||||
|
||||
**Phase:** `173_docs_push_creds_split` · **Source:** C3 (phase-locked) — a legacy credentials-bearing `docs_repo` value (env or DB) is split once at startup into `app_settings` rows; the DB rows shadow the env; `.env` is never rewritten; idempotent; one INFO line.
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Production's existing `BOR_DOCS_REPO=https://ducoterra:<token>@gitea.…` keeps working (the push still authenticates) with zero operator action, while the effective value the settings page shows becomes the clean URL and the credentials live in `docs_user` / `docs_token` (masked). A second run is a no-op; owner-set values are never clobbered.
|
||||
|
||||
## Work
|
||||
1. New module `app/core/settings_migrate.py`:
|
||||
```python
|
||||
"""One-time settings migrations (phase 173).
|
||||
|
||||
``migrate_legacy_docs_creds`` splits a legacy ``://user:pass@``
|
||||
``docs_repo`` value into ``docs_repo`` (clean) + ``docs_user`` +
|
||||
``docs_token`` — DB rows that SHADOW the env (the settings model's
|
||||
standing rule: the database wins over the environment). ``.env`` is
|
||||
never read for writing or rewritten. Idempotent: it runs only while
|
||||
no ``docs_token`` override exists, so owner-set values are never
|
||||
clobbered. One INFO line when it acts (AGENTS rule 10).
|
||||
"""
|
||||
```
|
||||
- `migrate_legacy_docs_creds() -> bool` — (1) read the `app_settings` overrides (via the overlay module's state — `load_overrides()` L93 or its backing read; use `reset_state_for_tests()` in tests); (2) the target value is the **effective** `docs_repo` (an override if present, else the env value — i.e. what `resolve_settings().docs_repo` resolves to, L239); (3) match `://([^/@:]+):([^@/]+)@` with BOTH groups non-empty — otherwise return `False` (no-op); (4) if a `docs_token` override already exists → `False` (no-op — never clobber); (5) otherwise write three rows (the overlay's existing write path — the same one the `PUT` route uses, or a direct `app_settings` insert if the overlay exposes none): `docs_repo` = the value with the `user:pass@` segment removed (e.g. `https://u:pass@host/r.git` → `https://host/r.git`), `docs_user` = user, `docs_token` = pass; log ONE INFO line: `settings: migrated embedded docs-push credentials out of docs_repo into docs_user/docs_token`; return `True`. A malformed `user:pass@` (empty user or pass) → `False`, no rows, no log (the legacy value stays as-is — the pre-phase behavior; nothing breaks, nothing half-migrates).
|
||||
2. `app/main.py` — `create_app()` (L98, the composition root): after the DB is available and BEFORE `return app`, call `settings_migrate.migrate_legacy_docs_creds()` ONCE (wrap in `try/except` so a migration hiccup logs a WARNING and never blocks startup — the app boots with the pre-phase behavior in that case; the migration is a convenience, not a dependency).
|
||||
3. `tests/unit/test_settings_migrate.py` — the four shapes (with the overlay state controlled via `reset_state_for_tests()` + a fake/monkeypatched override store — house style):
|
||||
- (a) clean env URL, no overrides → `False`, no rows written;
|
||||
- (b) legacy env `https://u:pass@host/r.git`, no overrides → `True`, three rows written (clean repo / `u` / `pass`), `resolve_settings()` resolves the clean URL from the DB;
|
||||
- (c) legacy env + a `docs_token` override already present → `False`, the pre-existing rows are untouched (no clobber);
|
||||
- (d) malformed `https://:@host/r.git` (empty user) → `False`, no rows, no log.
|
||||
Plus: (e) the INFO line is emitted exactly once in (b) (caplog / log-capture, house style).
|
||||
4. `tests/integration/test_docs_push_migrate.py` — the end-to-end shape: seed env `BOR_DOCS_REPO=https://u:pass@host/r.git` (no DB rows) → run the migration (direct call, the `create_app()` wiring is covered by the unit seam) → `resolve_settings().docs_repo == "https://host/r.git"`, `docs_user == "u"`, `docs_token == "pass"` (all from DB overrides); a second call → `False` (idempotent); then `reset_state_for_tests()` for the next test.
|
||||
5. `.env.example` — the block comment added in task 01 (already covers this: "credentials in the repo URL are no longer supported — use BOR_DOCS_USER/BOR_DOCS_TOKEN"; the migration note: a legacy value is auto-split on startup into the new settings). No other file touched.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_settings_migrate.py` green (the 5 cases above).
|
||||
- Integration: `tests/integration/test_docs_push_migrate.py` green.
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green — the no-op/act/clobber/malformed branches are all covered by 3a–3d; the `create_app()` try/except seam is covered by a unit test that monkeypatches the migration to raise).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_settings_migrate.py tests/integration/test_docs_push_migrate.py -v --no-cov` green: the four shapes + the idempotency + the one-INFO-line pin.
|
||||
- [ ] `create_app()` calls the migration once, wrapped so a failure logs a WARNING and the app still boots (unit-asserted via the monkeypatched-raise seam).
|
||||
- [ ] After a (b)-shape migration, the settings API reports the clean `docs_repo` + `is_set=true` for `docs_token` with `value=null` (a quick curl or an integration assertion) — the leak is gone from the page.
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (no pre-migration behavior touched; the sync path untouched).
|
||||
@@ -0,0 +1,44 @@
|
||||
# Task 05 — The phase E2E suite + the full gates + the atomic commit
|
||||
|
||||
**Phase:** `173_docs_push_creds_split` · **Source:** AGENTS rule 4 (one dedicated Playwright E2E suite per phase, run in isolation) + the test-gate protocol (AGENTS rules 8/9).
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
`tests/e2e/test_docs_push_creds.py` proves the leak class is dead in the real UI — the `Docs token` row is write-only, masked, never pre-filled, and a saved token never appears anywhere in the page DOM — and the phase's full gate passes: 100% of the project suite, >90% `app/` coverage, ruff + pyright, the regression E2E set, one atomic `--no-gpg-sign` Conventional Commit, and the phase dir moves to `complete/`.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_docs_push_creds.py` — the phase's dedicated Playwright suite (the phase-168 `test_admin_settings_panel.py` is the template for login + the settings panel; the mock-LLM default per A16 — the suite needs no LLM at all). Tests, in order:
|
||||
- **rows render:** sign in as admin → Settings → the `Docs push` group shows a `Docs repo` text row (the clean URL, or the test-seed value), a `Docs user` text row, and a `Docs token` row that is write-only — its input is `type=password`, **not pre-filled** (value `""`, rendered `(stored)`-style per the phase-168 secret-row contract), and the token text is absent from the initial DOM.
|
||||
- **save stays masked:** type a unique sentinel token (e.g. `e2e-sentinel-<uuid>` from the test) into the `Docs token` row → Save → the row re-renders masked; reload the page → the row is still masked, `is_set`-state visible, and the **sentinel text is absent from the entire page DOM** (the core assertion — `page.content()` must not contain the sentinel).
|
||||
- **reset unsets:** Reset the `Docs token` row → the row returns to the not-set state (no `(stored)` marker).
|
||||
- **the guard, UI-side:** type a credentials-bearing URL (`https://u:pass@e2e.invalid/o/r.git`) into `Docs repo` → Save → the 422 surfaces (the phase-168 error-display contract), the row keeps the previous value, and the page DOM never shows the typed URL's `pass` segment as a *saved* value (the input echoes what was typed before save — that is fine; the assertion is on the saved/persisted state after the failed save + reload).
|
||||
- Seed/cleanup: the test manages its own `app_settings` rows via the admin API (or the test DB fixture the other settings E2Es use) — no state leaks between runs.
|
||||
2. **Regression set (run in isolation, house order):** `uv run pytest tests/e2e/test_admin_settings_panel.py tests/e2e/test_admin_settings_layout.py tests/e2e/test_response_to_docs.py -v --no-cov` — phase 168/171 (the settings panel) and phase 59 (the docs push) stay green.
|
||||
3. **Full gates:** `uv run pytest` (100% — every test in the project, pre-existing included) · `uv run pytest --cov=app --cov-report=term-missing` (>90% on `app/`) · `uv run ruff check . && uv run pyright`.
|
||||
4. **Docs & consistency pass (mandatory — AGENTS.md rule 5 / phase 00):** review `README.md` and `AGENTS.md` in their entirety against the project as it now stands; fix every inconsistency found — including pre-existing ones. At minimum: the README's docs-push section documents `BOR_DOCS_USER` / `BOR_DOCS_TOKEN`, states that credentials in the repo URL are no longer supported (the PUT 422 + the startup migration), and notes the token is masked (write-only) in the settings page/API.
|
||||
5. **Commit (AGENTS rule 8):** ONE atomic Conventional Commit, `--no-gpg-sign`, covering `app/`, `scripts/`, `.env.example`, `tests/`, the README fix, and this phase dir's move to `.agents/phases/complete/173_docs_push_creds_split/`. Message shape:
|
||||
```
|
||||
feat(settings): split docs-push credentials (BOR_DOCS_USER/BOR_DOCS_TOKEN) out of the repo URL
|
||||
|
||||
- docs_token is write-only (masked in the page + API, llm_api_key-style); the
|
||||
docs_repo PUT guard 422s on embedded credentials
|
||||
- the push authenticates via GIT_CURL_USERNAME/GIT_CURL_PASSWORD env (never in
|
||||
the argv); anonymous pushes are byte-identical to before
|
||||
- a one-time startup migration splits a legacy ://user:pass@ env value into the
|
||||
new settings (DB rows shadow the env; .env untouched; idempotent)
|
||||
|
||||
LLM-Generated: true
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_docs_push_creds.py -v --no-cov` green **in isolation** (the phase's dedicated suite, AGENTS rule 4).
|
||||
- Full suite: `uv run pytest` 100% green; `app/` coverage **>90%**; ruff + pyright clean.
|
||||
- Regression: the phase 168/171/59 E2E set green in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_docs_push_creds.py -v --no-cov` green in isolation: rows render, the token row is `type=password` + not pre-filled, save→reload keeps it masked with the sentinel **absent from the page DOM**, reset unsets, the 422 guard surfaces.
|
||||
- [ ] `uv run pytest tests/e2e/test_admin_settings_panel.py tests/e2e/test_admin_settings_layout.py tests/e2e/test_response_to_docs.py -v --no-cov` green in isolation.
|
||||
- [ ] `uv run pytest` 100% green; `uv run pytest --cov=app` >90% on `app/`; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `README.md` + `AGENTS.md` reviewed in full and consistent (the docs-push section documents the new vars + the guard + the migration).
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit; the phase dir moved to `.agents/phases/complete/173_docs_push_creds_split/`.
|
||||
- [ ] **Owner follow-up (noted, not code):** the exposed production Gitea token (seen in the 2026-10-10 audit) is rotated out-of-band; the migration makes its replacement the `docs_token` value.
|
||||
Reference in New Issue
Block a user