Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 84_docs_push_error_sanitization

Browse Source 
**Phase 84 — final verification pass: all green, no defects found**

- Verified implementation: `app/core/errors.py` (verbatim lift of sync masker), `app/api/sync.py` alias import, docs-push 502 `detail=sanitize_error(str(exc))`, all five `llm.py` error sites sanitized; new/extended test pins in place
- Tests: `uv run pytest` → **1714 passed, 0 failed**; targeted pins (new unit ×2 + integration ×1, existing 502 pin) → 13 passed; sync/git-sources regression → 67 passed
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **99%** (`app/core/errors.py` 100%, `app/rag/llm.py` 100%) — >90% met
- E2E isolation: `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → **3 passed**
- Lint/types: `uv run ruff check .` → clean; `uv run pyright` → **0 errors**
- Criteria: 502 masks `*****@`/never token + row untouched ✅; LLM base-URL masked, credential-free strings byte-identical ✅; `_CREDS_RE` only in `app/core/errors.py` (working-tree grep) ✅; full gate green ✅; `git diff --stat` limited to the 4 app files + 2 modified test files + 3 phase task files (untracked: new module, new unit test, complete/ dir, reports, audit plan) ✅
- Commit/phase move left to the harness per instructions (task files already in `complete/`)
- No deviations; nothing to fix
- Next pending phase: **85_mobile_menu_gate_overlap**
This commit is contained in:
ducoterra
2026-09-08 01:11:45 -04:00
parent fa189dede7
commit f4150421bb
23 changed files with 732 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/84_docs_push_error_sanitization/00_phase.md → .agents/phases/complete/84_docs_push_error_sanitization/00_phase.md
View File
.agents/phases/todo/84_docs_push_error_sanitization/01_shared_sanitizer.md → .agents/phases/complete/84_docs_push_error_sanitization/01_shared_sanitizer.md
View File
.agents/phases/todo/84_docs_push_error_sanitization/02_apply_docs_push_and_llm.md → .agents/phases/complete/84_docs_push_error_sanitization/02_apply_docs_push_and_llm.md
View File
.agents/phases/todo/84_docs_push_error_sanitization/03_verify_and_commit.md → .agents/phases/complete/84_docs_push_error_sanitization/03_verify_and_commit.md
View File
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__00_phase.a1.err
View File
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__00_phase.a1.md
+11
View File
@@ -0,0 +1,11 @@
**Phase 84 — final verification pass: all green, no defects found**
- Verified implementation: `app/core/errors.py` (verbatim lift of sync masker), `app/api/sync.py` alias import, docs-push 502 `detail=sanitize_error(str(exc))`, all five `llm.py` error sites sanitized; new/extended test pins in place
- Tests: `uv run pytest` → **1714 passed, 0 failed**; targeted pins (new unit ×2 + integration ×1, existing 502 pin) → 13 passed; sync/git-sources regression → 67 passed
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **99%** (`app/core/errors.py` 100%, `app/rag/llm.py` 100%) — >90% met
- E2E isolation: `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → **3 passed**
- Lint/types: `uv run ruff check .` → clean; `uv run pyright` → **0 errors**
- Criteria: 502 masks `*****@`/never token + row untouched ✅; LLM base-URL masked, credential-free strings byte-identical ✅; `_CREDS_RE` only in `app/core/errors.py` (working-tree grep) ✅; full gate green ✅; `git diff --stat` limited to the 4 app files + 2 modified test files + 3 phase task files (untracked: new module, new unit test, complete/ dir, reports, audit plan) ✅
- Commit/phase move left to the harness per instructions (task files already in `complete/`)
- No deviations; nothing to fix
- Next pending phase: **85_mobile_menu_gate_overlap**
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__00_phase.a1.validate
+85
View File
@@ -0,0 +1,85 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 21%]
........................................................................ [ 25%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 46%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 58%]
........................................................................ [ 63%]
........................................................................ [ 67%]
........................................................................ [ 71%]
........................................................................ [ 75%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 88%]
........................................................................ [ 92%]
........................................................................ [ 96%]
.......................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 98 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 19 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 64 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 217 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 232 0 100%
--------------------------------------------------
TOTAL 3237 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__01_shared_sanitizer.a1.err
View File
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__01_shared_sanitizer.a1.md
+18
View File
@@ -0,0 +1,18 @@
All criteria verified. Final report:
**Task 01 — shared sanitizer module + sync alias refactor: complete**
- `app/core/errors.py` (new): owns `_CREDS_RE` (exact phase-32 pattern) + `sanitize_error()` (`sub("*****@")`), with module docstring covering SEC-08/SEC-13 audit basis, narrow-userinfo byte-identity contract, and the `*****@` replacement shape.
- `app/api/sync.py`: local `_CREDS_RE`/`_sanitize_error` deleted; now `from app.core.errors import sanitize_error as _sanitize_error`; one-line module-docstring note added; now-unused `import re` removed. `_run_sync` failure path byte-identical; `git_sources.py`/`docs.py` alias imports untouched.
- `tests/unit/test_error_sanitization.py` (new, 9 tests): single/multiple userinfo masking, SEC-08 git push-failure line (token masked, `fatal:`/`exit 128`/host intact), byte-identity pins (bare https URL, colon+space prose, email `owner@example.com` — pattern doesn't match it, pinned as unchanged, empty string), idempotence on all masked forms, and `sync_api._sanitize_error is sanitize_error` alias identity.
Results:
- `uv run pytest tests/unit/test_error_sanitization.py -v` → 9 passed
- `uv run pytest tests/integration/test_git_sources_upload.py tests/integration/test_sync_api.py` → 44 passed (no test edits)
- `uv run pytest --cov=app --cov-report=term-missing` → 1711 passed; `app/core/errors.py` 100%, app/ total 99%
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- `grep -rn _CREDS_RE app/` → exactly one definition, in `app/core/errors.py`
Decisions: none — verbatim lift per design block. No pre-existing defects found. No git commits made (left for harness).
Next pending task: `.agents/phases/todo/84_docs_push_error_sanitization/02_apply_docs_push_and_llm.md`
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__01_shared_sanitizer.a1.validate
+85
View File
@@ -0,0 +1,85 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 21%]
........................................................................ [ 25%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 46%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 58%]
........................................................................ [ 63%]
........................................................................ [ 67%]
........................................................................ [ 71%]
........................................................................ [ 75%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 88%]
........................................................................ [ 92%]
........................................................................ [ 96%]
....................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 98 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 19 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 64 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 232 0 100%
--------------------------------------------------
TOTAL 3235 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__02_apply_docs_push_and_llm.a1.err
View File
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__02_apply_docs_push_and_llm.a1.md
+18
View File
@@ -0,0 +1,18 @@
All green. Final report:
**Phase 84, Task 02 — docs-push 502 + LLM error sanitization**
Implemented:
- `app/api/doc_drafts.py`: `push_doc_draft` 502 detail now `sanitize_error(str(exc))` (SEC-08); route + module docstrings updated to name the SANITIZED stderr
- `app/rag/llm.py`: all five error f-strings (embed, chat×3, chat_stream) wrap `self.settings.llm_base_url` in `sanitize_error(...)` (SEC-13) — surrounding copy byte-identical
- `tests/integration/test_doc_drafts_api.py`: new pin — faked `DocsPushError` with `bot:ghp_LEAKTOKEN@github.com` → 502 detail has `*****@github.com`, `exit 128`, `fatal: Authentication failed`, never `ghp_LEAKTOKEN`; row untouched
- `tests/unit/test_llm_client.py`: new pins — embed + chat transport failures with `https://svc:topsecret@llm.local/v1` → masked `https://*****@llm.local/v1`, no `topsecret`; plus a byte-identical credential-free message assertion
Results:
- `uv run pytest tests/integration/test_doc_drafts_api.py -v` — 28 passed; `uv run pytest tests/unit/test_llm_client.py -v` — 57 passed (pre-existing error-string pins green unchanged)
- `uv run pytest --cov=app --cov-report=term-missing` — full suite green, TOTAL 99% (>90%); doc_drafts.py / llm.py / errors.py 100%
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` — 3 passed in isolation
- `uv run ruff check .` — clean; `uv run pyright` — 0 errors
- No deviations; diff limited to the four task files (task 01's files remain in the tree for the harness's atomic phase commit)
Next pending task: `.agents/phases/todo/84_docs_push_error_sanitization/03_verify_and_commit.md`
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__02_apply_docs_push_and_llm.a1.validate
+85
View File
@@ -0,0 +1,85 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 21%]
........................................................................ [ 25%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 46%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 58%]
........................................................................ [ 63%]
........................................................................ [ 67%]
........................................................................ [ 71%]
........................................................................ [ 75%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 88%]
........................................................................ [ 92%]
........................................................................ [ 96%]
.......................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 98 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 19 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 64 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 217 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 232 0 100%
--------------------------------------------------
TOTAL 3237 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__03_verify_and_commit.a1.err
View File
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__03_verify_and_commit.a1.md
+21
View File
@@ -0,0 +1,21 @@
All gates verified. Every task-03 completion criterion is met in the working tree; no defects found in prior tasks' work.
**Phase 84 gate report — task 03 (verify) complete**
Implemented (by tasks 01–02, verified by me):
- `app/core/errors.py` (new): `sanitize_error` — verbatim lift of the sync `user:pass@` masker
- `app/api/sync.py`: local regex+function deleted, private alias import (callers `git_sources.py`/`docs.py` untouched)
- `app/api/doc_drafts.py`: docs-push 502 detail runs through `sanitize_error` (SEC-08)
- `app/rag/llm.py`: all five LLM error f-strings sanitize `llm_base_url` (SEC-13)
- New/extended pins: `tests/unit/test_error_sanitization.py`, `test_llm_client.py` (mask + byte-identical), `test_doc_drafts_api.py` (502 masks `ghp_LEAKTOKEN`, row untouched)
Gate results (all green):
- `uv run pytest` → 1714 passed, exit 0 (regression anchors incl. sync/upload, doc-drafts, LLM)
- `uv run pytest --cov=app --cov-report=term-missing` → `app/` TOTAL **99%** (>90% required)
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed in isolation
- `uv run ruff check . && uv run pyright` → clean (0 errors/warnings)
- `grep -rn _CREDS_RE app/ scripts/` → only `app/core/errors.py`; diff scope limited to the 7 code/test files + phase files; NO `scripts/`, `docs_push.py`, `frontend/`, `pyproject.toml`/`uv.lock`
Deviations (per harness executor rules, overriding task step 2–3): no `git add`/`commit` and no phase-dir move by me — all changes left in the working tree for the harness's one atomic `fix(security)` commit + todo→complete move.
Next pending task: none in phase 84 (03 was last); next phase: `85_mobile_menu_gate_overlap`.
.agents/reports/84_docs_push_error_sanitization/84_docs_push_error_sanitization__03_verify_and_commit.a1.validate
+85
View File
@@ -0,0 +1,85 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 21%]
........................................................................ [ 25%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 46%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 58%]
........................................................................ [ 63%]
........................................................................ [ 67%]
........................................................................ [ 71%]
........................................................................ [ 75%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 88%]
........................................................................ [ 92%]
........................................................................ [ 96%]
.......................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 98 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 19 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 64 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 217 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 232 0 100%
--------------------------------------------------
TOTAL 3237 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
app/api/doc_drafts.py
+20 -8
View File
@@ -26,8 +26,10 @@ triggers — commit + ``git push`` the draft's file to the
:func:`app.core.docs_push.push_document`; success records
``status`` / ``branch`` / ``commit_sha`` on the row and returns
``DocDraftPushed``; 409 while unconfigured, 422 on a path that no
longer passes the guard-rails, 502 on git failure with git's stderr
in the detail — the row untouched).
longer passes the guard-rails, 502 on git failure with the
SANITIZED git stderr in the detail (credential ``user:pass@``
userinfo masked by :func:`app.core.errors.sanitize_error` — phase
84, SEC-08) — the row untouched).
Every ``path`` (create, update **and** push) passes the shared
:func:`validate_draft_path` guard, so no draft can ever be created,
@@ -45,6 +47,7 @@ from sqlalchemy.orm import Session
from app.config import Settings, get_settings
from app.core.auth import require_admin
from app.core.docs_push import DocsPushError, push_document
from app.core.errors import sanitize_error
from app.db import get_db
from app.models import DocDraft
from app.schemas import DocDraft as DocDraftOut
@@ -251,11 +254,16 @@ def push_doc_draft(
3. the stored ``path`` re-runs :func:`validate_draft_path` → 422
on the first violated rule (a row must not be pushable into a
bad path, whatever wrote it);
4. :class:`DocsPushError` → 502 with ``detail=str(exc)`` — git's
stderr, the ``GitSyncError`` → ``detail`` mapping from
:mod:`app.api.git_sources`. Only a SUCCESS mutates the row:
the failed push leaves ``status`` / ``branch`` / ``commit_sha``
exactly as found (no partial commit).
4. :class:`DocsPushError` → 502 with the SANITIZED git stderr in
the detail (phase 84, SEC-08): ``str(exc)`` runs through
:func:`app.core.errors.sanitize_error`, which masks any
``user:pass@`` userinfo git's stderr may echo (a
``BOR_DOCS_REPO`` URL with embedded credentials) — the failing
repo and git's reason stay readable, and the
``GitSyncError`` → ``detail`` mapping from
:mod:`app.api.git_sources` is otherwise kept. Only a SUCCESS
mutates the row: the failed push leaves ``status`` / ``branch``
/ ``commit_sha`` exactly as found (no partial commit).
On success the row becomes ``status = "pushed"`` with the landed
``branch`` and ``commit_sha`` (must equal ``git rev-parse
@@ -283,7 +291,11 @@ def push_doc_draft(
commit_message=f"docs: {row.title}",
)
except DocsPushError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from None
# Phase 84 (SEC-08): git's stderr may echo the remote URL with
# embedded credentials — the RESPONSE is where the secret would
# leak, so it is sanitized here (the DocsPushError itself keeps
# carrying the full stderr for logs/inspection).
raise HTTPException(status_code=502, detail=sanitize_error(str(exc))) from None
row.status = "pushed"
row.branch = branch
row.commit_sha = sha
app/api/sync.py
+5 -15
View File
@@ -57,12 +57,15 @@ carries the phase-64 per-file progress — ``current_file`` (the
``files_done`` / ``files_total`` — null/0/0 before the import starts
(clone/pull reports no file yet) and in terminal states, which clear
``current_file`` but keep the run's final counts.
The ``failed`` state's ``error`` string is masked by the shared
sanitizer — the ``user:pass@`` masker now lives in :mod:`app.core.errors`
(imported here under the private name ``_sanitize_error``).
"""
from __future__ import annotations
import asyncio
import logging
import re
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
@@ -72,6 +75,7 @@ from fastapi import APIRouter, Depends, HTTPException
from app.config import get_settings
from app.core.auth import require_admin
from app.core.errors import sanitize_error as _sanitize_error
from app.db import SessionLocal
from app.rag.git_sources import effective_sources
from app.rag.importer import ImportSummary, import_sources
@@ -89,20 +93,6 @@ router = APIRouter(
dependencies=[Depends(require_admin)], # phase 16 pattern: admin-only surface
)
#: ``user:pass@`` inside any error text (git stderr, endpoint URLs) —
#: masked so a sync failure can never leak credentials into the UI.
_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")
def _sanitize_error(message: str) -> str:
"""Mask credentials embedded in an error string (no secrets in the UI).
Git's stderr is otherwise surfaced verbatim (phase locked decisions) —
it names the failing repo and git's reason, which is what the admin
needs to fix things.
"""
return _CREDS_RE.sub("*****@", message)
@dataclass
class SyncStatus:
app/core/errors.py
+57
View File
@@ -0,0 +1,57 @@
"""Shared error-string sanitizer (phase 84 — audit SEC-08 / SEC-13).
``sanitize_error`` masks ``user:pass@`` userinfo in an error string so
no error surface ever ships an embedded credential into the UI, a
response body, or an SSE frame. It is the verbatim lift of the phase-32
sync masker (same regex, same ``*****@`` replacement — byte-identical
behavior); ``app/api/sync.py`` keeps the private name ``_sanitize_error``
as an alias import, and the docs-push 502 detail (SEC-08) and the LLM
error messages (SEC-13) run through the same function.
Audit basis (``.agents/remediation_plan.md``, security audit 2026-09-07):
* SEC-08 (Medium) — the docs-push 502 surfaced git's stderr verbatim;
``BOR_DOCS_REPO`` is documented as "any remote (URL or local path)",
so an ``https://user:token@host/...`` URL is a normal config shape,
and a failed push (revoked token, network) echoes the remote URL in
git's stderr straight into the browser and the logs. The sync path
already solved exactly this problem — the docs-push and LLM surfaces
simply never got the treatment.
* SEC-13 (Low) — the LLM error f-strings interpolated
``settings.llm_base_url`` raw; a base URL configured with embedded
credentials would echo into SSE ``error`` frames and logs. Same class
of leak, same fix.
Contract (narrow userinfo-regex only — byte-identical for
credential-free text):
* Only the git/HTTP ``user:pass@`` userinfo shape is rewritten — a run
of userinfo characters, the separating colon, a second run, and the
``@``. Ordinary text survives character for character: plain hosts,
``https://host/...`` URLs without userinfo, prose with a colon +
space (``fatal: ...``), emails in prose (no colon userinfo run
before the ``@``). The existing sync / git-sources / LLM error-string
tests are the tripwire proving the move is behavior-identical.
* The replacement shape is ``*****@`` — the existing sync copy, so
every already-pinned masked error stays byte-identical.
* Idempotent for the masked forms the surfaces produce (sanitizing an
already-sanitized git/URL failure string is a no-op).
"""
from __future__ import annotations
import re
#: ``user:pass@`` inside any error text (git stderr, endpoint URLs) —
#: the narrow userinfo run (git/HTTP convention) that carries
#: credentials; everything else is left untouched.
_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")
def sanitize_error(message: str) -> str:
"""Mask credentials embedded in an error string (no secrets in the UI).
Git's stderr is otherwise surfaced verbatim (the admin needs the
failing repo and git's reason to fix things) — only the ``user:pass@``
userinfo shape is rewritten, to ``*****@``.
"""
return _CREDS_RE.sub("*****@", message)
app/rag/llm.py
+16 -6
View File
@@ -29,6 +29,7 @@ from openai import AsyncOpenAI, AsyncStream
from openai.types.chat import ChatCompletionChunk, ChatCompletionMessageParam
from app.config import Settings, get_settings
from app.core.errors import sanitize_error
if TYPE_CHECKING:
# Phase 71: the filter type is only needed for typing (the module
@@ -285,7 +286,11 @@ class LLMClient:
raise
except Exception as e: # noqa: BLE001 — wrap transport-level failures
raise EmbeddingError(
f"embeddings request to {self.settings.llm_base_url} failed: {e}"
# Phase 84 (SEC-13): a base URL configured with
# embedded ``user:pass@`` credentials must not reach
# the error string — sanitized at construction.
f"embeddings request to {sanitize_error(self.settings.llm_base_url)} "
f"failed: {e}"
) from e
self.embed_batches += 1
self._check_dims(vecs)
@@ -324,18 +329,20 @@ class LLMClient:
raise
except Exception as e: # noqa: BLE001 — wrap transport-level failures
raise LLMError(
f"chat completion from {self.settings.llm_base_url} failed: {e}"
# Phase 84 (SEC-13): sanitize the base URL (see embed).
f"chat completion from {sanitize_error(self.settings.llm_base_url)} "
f"failed: {e}"
) from e
if not resp.choices:
raise LLMError(
f"chat completion from {self.settings.llm_base_url} "
f"chat completion from {sanitize_error(self.settings.llm_base_url)} "
"returned no choices"
)
content = resp.choices[0].message.content
if content is None or not content.strip():
raise LLMError(
f"chat completion from {self.settings.llm_base_url} returned "
"empty content — refusing to store a silent summary"
f"chat completion from {sanitize_error(self.settings.llm_base_url)} "
"returned empty content — refusing to store a silent summary"
)
return content.strip()
@@ -483,7 +490,10 @@ class LLMClient:
except LLMError:
raise
except Exception as e: # noqa: BLE001 — wrap transport-level failures
raise LLMError(f"chat stream from {self.settings.llm_base_url} failed: {e}") from e
# Phase 84 (SEC-13): sanitize the base URL (see embed).
raise LLMError(
f"chat stream from {sanitize_error(self.settings.llm_base_url)} failed: {e}"
) from e
finally:
# Phase 48: deterministic teardown — whenever ``create()``
# succeeded, close the endpoint's stream on every subsequent
tests/integration/test_doc_drafts_api.py
+61
View File
@@ -23,7 +23,9 @@ import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select, text
import app.api.doc_drafts as doc_drafts
from app.config import Settings, get_settings
from app.core.docs_push import DocsPushError
from app.main import app as fastapi_app
from app.models import DocDraft
@@ -483,6 +485,65 @@ def test_push_non_repo_dir_returns_502_with_git_stderr(
assert body["commit_sha"] is None
def test_push_502_detail_masks_git_stderr_credentials(
admin_client: TestClient,
db,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 84 (SEC-08): a ``DocsPushError`` whose git stderr carries a
``user:pass@`` remote URL (a ``BOR_DOCS_REPO`` configured with
embedded credentials) → the 502 detail is the SANITIZED stderr — the
token never reaches the browser or the logs, while the failing repo
and git's reason stay readable; the row is untouched (only a success
mutates). ``push_document`` is monkeypatched, so no real git is
needed."""
def failing_push(**kwargs: Any) -> tuple[str, str]:
raise DocsPushError(
"git push origin bor-docs failed (exit 128): "
"fatal: Authentication failed for "
"'https://bot:ghp_LEAKTOKEN@github.com/owner/docs.git/'"
)
monkeypatch.setattr(doc_drafts, "push_document", failing_push)
fastapi_app.dependency_overrides[
get_settings
] = lambda: _settings(
docs_repo="/nonexistent/docs.git", # configured — push_document is faked
docs_branch=DOCS_BRANCH,
docs_base_branch=BASE_BRANCH,
docs_work_dir=str(tmp_path / "docs-workdir"),
)
created = _create(admin_client)
try:
r = admin_client.post(f"/api/doc-drafts/{created['token']}/push")
finally:
fastapi_app.dependency_overrides.pop(get_settings, None)
assert r.status_code == 502
detail = r.json()["detail"]
# The credential is masked to the sync masker's shape ...
assert "*****@github.com" in detail
assert "ghp_LEAKTOKEN" not in detail
# ... but the failure context stays readable (the GitSyncError →
# detail mapping, otherwise kept).
assert "exit 128" in detail
assert "fatal: Authentication failed" in detail
assert "owner/docs.git" in detail
# The row is untouched (status/branch/commit_sha as found).
body = admin_client.get(f"/api/doc-drafts/{created['token']}").json()
assert body["status"] == "draft"
assert body["branch"] is None
assert body["commit_sha"] is None
row = db.execute(
select(DocDraft).where(DocDraft.token == uuid.UUID(created["token"]))
).scalars().one()
assert row.status == "draft"
assert row.branch is None
assert row.commit_sha is None
def test_push_unknown_token_returns_404(
admin_client: TestClient, docs_push_settings: Settings
) -> None:
tests/unit/test_error_sanitization.py
+119
View File
@@ -0,0 +1,119 @@
"""Unit tests: the shared error-string sanitizer (phase 84, task 01).
``app.core.errors.sanitize_error`` is the verbatim lift of the phase-32
sync masker — a NARROW ``user:pass@`` userinfo regex (git/HTTP
convention) with the ``*****@`` replacement. These pins fix both sides
of the contract:
* masking — every pinned ``user:pass@`` occurrence is rewritten to
``*****@`` (a single userinfo, two in one string, and one inside a
realistic git push-failure line — where ``fatal:`` / ``exit 128`` /
the host survive);
* byte-identity — credential-free text (a bare https URL without
userinfo, prose with a colon + space, a plain email, the empty
string) comes back unchanged, character for character;
* idempotence — sanitizing an already-sanitized (masked) string is a
no-op.
Plus the refactor pin: ``app.api.sync._sanitize_error`` is the shared
function under the private alias name (the phase-32 callers —
``app/api/git_sources.py``, ``app/api/docs.py`` — see the identical
function through the import).
"""
from __future__ import annotations
import app.api.sync as sync_api
from app.core.errors import sanitize_error
TWO_USERINFO = "https://alice:secret1@repo.example.com/a.git and https://bob:secret2@repo.example.com/b.git"
GIT_PUSH_FAILURE = (
"git push failed (exit 128): fatal: Authentication failed for "
"'https://bot:tok@github.com/o/r.git/'"
)
# --- The masked forms (the regex must fire) --------------------------------
def test_masks_single_userinfo_in_url() -> None:
"""The canonical shape: ``user:pass@`` → ``*****@``, host intact."""
assert sanitize_error("https://user:pass@host/repo.git") == "https://*****@host/repo.git"
def test_masks_every_userinfo_occurrence() -> None:
"""Two userinfo runs in one string → both masked, nothing else touched."""
masked = sanitize_error(TWO_USERINFO)
assert masked == "https://*****@repo.example.com/a.git and https://*****@repo.example.com/b.git"
assert "secret1" not in masked
assert "secret2" not in masked
def test_masks_token_in_git_push_failure_line() -> None:
"""The SEC-08 shape: git's auth-failure stderr with a token-bearing
remote URL → the token is masked, the actionable context survives."""
masked = sanitize_error(GIT_PUSH_FAILURE)
assert masked == (
"git push failed (exit 128): fatal: Authentication failed for "
"'https://*****@github.com/o/r.git/'"
)
assert "tok" not in masked
assert "exit 128" in masked # the reason survives
assert "fatal:" in masked
assert "github.com/o/r.git" in masked # host + repo path survive
# --- The byte-identity contract (credential-free text) ---------------------
def test_plain_url_without_userinfo_is_unchanged() -> None:
"""A bare https URL (no userinfo) plus a realistic git failure line
→ byte-identical."""
message = (
"git push failed (exit 128): fatal: repository "
"'https://github.com/owner/repo.git/' not found"
)
assert sanitize_error(message) == message
def test_prose_with_colon_and_space_is_unchanged() -> None:
"""Colons in prose are followed by spaces, not by userinfo runs —
the narrow regex must not fire (the sync tests pin the same copy)."""
model_message = (
"The embedding model ('embed') is not available — check the model endpoint and retry."
)
assert sanitize_error(model_message) == model_message
sentence = "note: the sync failed: see the log for details"
assert sanitize_error(sentence) == sentence
def test_email_in_prose_is_unchanged() -> None:
"""``owner@example.com`` carries no colon userinfo run before the
``@`` — the pattern does not match it, so it stays byte-identical
(pinned for determinism: the masker only rewrites ``user:pass@``)."""
message = "please contact owner@example.com about the failing push"
assert sanitize_error(message) == message
def test_empty_string_is_unchanged() -> None:
assert sanitize_error("") == ""
# --- Idempotence (masked forms) ---------------------------------------------
def test_idempotent_on_masked_forms() -> None:
"""Sanitizing an already-sanitized string is a no-op — the ``*****@``
replacement never forms a new ``user:pass@`` run in the pinned cases."""
for message in ("https://user:pass@host/repo.git", TWO_USERINFO, GIT_PUSH_FAILURE):
once = sanitize_error(message)
assert sanitize_error(once) == once
# --- The sync alias refactor (phase 84, task 01) ----------------------------
def test_sync_keeps_the_private_alias_of_the_shared_function() -> None:
"""``app.api.sync._sanitize_error`` IS the shared sanitizer under the
private name — the phase-32 callers (``git_sources.py``, ``docs.py``)
import it through ``app.api.sync`` and see the identical function."""
assert sync_api._sanitize_error is sanitize_error
assert sync_api._sanitize_error(GIT_PUSH_FAILURE) == sanitize_error(GIT_PUSH_FAILURE)
tests/unit/test_llm_client.py
+46
View File
@@ -189,6 +189,36 @@ def test_http_error_surfaces_status() -> None:
assert llm.embed_batches == 0
def test_embed_transport_failure_masks_credentials_in_base_url() -> None:
"""Phase 84 (SEC-13): a base URL configured with embedded
``user:pass@`` credentials never reaches the error message — the URL
in the raised :class:`EmbeddingError` is sanitized (``*****@``),
while the failure context stays readable."""
llm, _ = _make_client(
_FakeEmbeddingsService(fail=RuntimeError("connection refused")),
llm_base_url="https://svc:topsecret@llm.local/v1",
)
with pytest.raises(EmbeddingError) as exc:
asyncio.run(llm.embed(["hello"]))
msg = str(exc.value)
assert "https://*****@llm.local/v1" in msg
assert "topsecret" not in msg
assert "connection refused" in msg
assert llm.embed_batches == 0
# The credential-free pin stays byte-identical (sanitize is a no-op
# without userinfo): the default base URL appears verbatim.
llm_plain, _ = _make_client(
_FakeEmbeddingsService(fail=RuntimeError("connection refused"))
)
with pytest.raises(EmbeddingError) as exc_plain:
asyncio.run(llm_plain.embed(["hello"]))
assert (
str(exc_plain.value)
== "embeddings request to https://aipi.reeseapps.com/v1 failed: "
"connection refused"
)
def test_missing_vector_row_is_rejected() -> None:
llm, _ = _make_client(_FakeEmbeddingsService(drop_index=1))
with pytest.raises(EmbeddingError, match="returned 1 vectors for 2 inputs"):
@@ -925,6 +955,22 @@ def test_chat_transport_failure_wrapped_as_llm_error_with_base_url() -> None:
assert "aipi.reeseapps.com" in str(exc.value)
def test_chat_transport_failure_masks_credentials_in_base_url() -> None:
"""Phase 84 (SEC-13): the chat failure f-string sanitizes the base
URL the same way the embed path does (the five sites share the
construction; this is the chat representative)."""
llm, _ = _make_chat_client(
fail=RuntimeError("HTTP 502 Bad Gateway"),
llm_base_url="https://svc:topsecret@llm.local/v1",
)
with pytest.raises(LLMError) as exc:
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
msg = str(exc.value)
assert "https://*****@llm.local/v1" in msg
assert "topsecret" not in msg
assert "HTTP 502 Bad Gateway" in msg
def test_chat_llm_error_passes_through_unwrapped() -> None:
llm, _ = _make_chat_client(fail=LLMError("already wrapped"))
with pytest.raises(LLMError, match="already wrapped"):