phase: 81_login_rate_limit

All gates verified. The phase is complete — everything was already implemented by prior task runs; this pass verified each completion criterion end-to-end and found no defects.

**Phase 81 final verification report** (tasks 01–03 all present in `complete/`; no code changes needed this pass)

- Verified: `app/core/rate_limit.py` (stdlib sliding window, fail-open) + both login routes' 429 pre-check/record/reset wiring + all unit/integration pins present (11th-429, blocked-success-still-429, clean-counter reset, shared counter, autouse `clean_rate_limit` fixture documented).
- Live check (task 03): dev server + 11 rapid wrong logins → `401 ×10, 429`; 429 body carries `retry-after: 900` + generic detail; server restarted (per-process counter cleared by design).
- `uv run pytest tests/unit/test_rate_limit.py -v --no-cov` → 10 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed, **TOTAL 99%** (>90%; rate_limit.py 100%, auth.py 100%)
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` (isolation) → 3 passed
- `uv run ruff check . && uv run pyright` → All checks passed / 0 errors, 0 warnings
- Completion criteria: all met, except commit + phase-dir move — per harness rules I left all changes uncommitted in the working tree (harness commits atomically and moves the phase).
- Diff scope: exactly `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py` + phase files; `pyproject.toml` / `uv.lock` / `frontend/` untouched.
- Deviations: none in code; commit/move deferred to harness as instructed.
- Next pending phase: `82_security_headers`.
This commit is contained in:
2026-09-07 23:08:46 -04:00
parent 894637108c
commit 42a4222949
32 changed files with 1439 additions and 2 deletions
@@ -0,0 +1,69 @@
# Phase 81 — Rate-limit failed sign-ins: 429 + Retry-After after repeated failures
**Source:** `.agents/remediation_plan.md` SEC-03 (security audit 2026-09-07, severity Medium — "No rate limiting / lockout on login + token-auth")
**Story:** n/a (security hardening — audit-derived, no user story)
**Context:** `app/api/auth.py` (`POST /api/login` — `check_password` then `sign_in`; `POST /api/token-auth` — `find_active_by_token` then session keys — both return one generic 401 on every failure shape, no throttling of any kind), `app/core/auth.py` (the constant-time `check_password`, `sign_in`/`sign_out` — untouched by this phase), `app/main.py` (middleware registration order — this phase adds no middleware), `frontend/assets/login.js` (the login form's error path: non-204 → `Sign-in failed (HTTP ${r.status}) — try again.` via `showError` — a 429 already renders there as a generic message, so no frontend change is required), `tests/unit/test_auth.py` + `tests/integration/test_auth_api.py` (the existing auth contract pins — every 401 shape, no enumeration).
## Objective
A client IP that fails sign-in repeatedly (admin password or token) gets **429 + `Retry-After`** instead of unlimited line-rate guesses: a stdlib-only in-memory sliding-window limiter shared by both login routes, counting only failed attempts, reset on success, failing open (a limiter bug must never lock the owner out).
## Audit basis (read this, not the chat)
- Neither `POST /api/login` nor `POST /api/token-auth` throttles: the password is compared with `compare_digest` (fast), so an attacker who can reach `:8000` can try thousands of guesses per second (audit PoC: 10 000 sequential logins all answered at line rate).
- The fix is deliberately small and stateless-per-process (A10: no new service, no DB table, no Redis): a per-IP deque of failure timestamps in module memory, lost on restart (an acceptable reset — it is a friction bump, not the boundary; the homelab HTTP posture is the documented owner decision).
- Threshold (locked in this phase): **10 failed attempts per client IP per 15-minute sliding window** → subsequent attempts get **429** `{"detail": "too many failed sign-in attempts — try again later"}` + a `Retry-After: <seconds>` header until the window clears.
- The counter is SHARED by both routes (any auth failure from the IP counts — a token-spraying attack must not reset by alternating routes), and a SUCCESS on either route clears that IP's counter (a legitimate owner who fat-fingers twice is not poisoned).
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
- **A1 — in-memory, stdlib-only:** no new dependency, no DB table, per-process state (A10/A12 spirit). A restart clears the counter — accepted.
- **A2 — 10 / 15 min, module constants:** `MAX_FAILURES = 10`, `WINDOW_SECONDS = 900` as documented module constants in the limiter module — NOT new `BOR_` settings (a security control with a fixed, tested default; no env surface to mistype).
- **A3 — 429 contract:** one generic detail (no enumeration between login/token failures), `Retry-After` header carrying the whole-window seconds; the login page's existing `Sign-in failed (HTTP 429) — try again.` line renders it — no frontend change in this phase.
- **A4 — client IP = `request.client.host`:** the direct peer address. ASSUMPTION (locked here): the app is served directly (homelab), no reverse proxy — if a proxy is ever put in front, the limiter needs `X-Forwarded-For` handling (noted in the module docstring, out of scope now).
## Design (shared by all tasks — the executor reads this, not the chat)
- **`app/core/rate_limit.py` (new)** — the limiter, pure stdlib (`collections.deque`, `time.monotonic`, `threading.Lock` for the dict — the ASGI event loop is single-threaded but the lock costs nothing and keeps the unit tests honest under pytest-xdist if ever added):
```python
MAX_FAILURES = 10
WINDOW_SECONDS = 900
_failures: dict[str, deque[float]] = {}
_lock = threading.Lock()
def record_failure(client_ip: str) -> None: ... # append now(); prune expired; never raises
def remaining_wait(client_ip: str) -> int: ... # 0 = allowed; else whole-window seconds (ceil) until the oldest counted failure expires
def reset(client_ip: str) -> None: ... # drop the IP's entry (on success)
```
Semantics (all unit-pinned): a failure is counted when it is recorded; the window slides — only failures within the last `WINDOW_SECONDS` count; the IP is blocked while the count of in-window failures `>= MAX_FAILURES` (the 10th failure already blocked? NO — the 10th failure is recorded and the **11th** attempt is the first 429: the check happens before the attempt); `remaining_wait` returns `int(ceil(WINDOW_SECONDS - (now - oldest)))` while blocked, `0` otherwise; **fail-open**: any internal error (corrupt deque, clock skew) → `remaining_wait` returns `0` and `record_failure` no-ops — a limiter bug never denies the owner.
- **`app/api/auth.py`** — the two route changes (identical shape):
- `login`: first line of the handler — `ip = request.client.host; if (wait := _wait(ip)) > 0: raise HTTPException(429, detail=TOO_MANY_DETAIL, headers={"Retry-After": str(wait)})`; on password failure → `record_failure(ip)` before the 401; on success → `reset(ip)` before the 204.
- `token_auth`: same three points (pre-check, record on the 401 path, reset on the 204 path).
- `TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"` (one string shared by both routes — no enumeration).
- Module docstrings updated: the 429 case joins the documented status set of each route.
- **Not touched:** `app/core/auth.py` (password check + session helpers), the 401 contract of both routes (byte-identical on the non-throttled path), everything else in the app, all frontend files.
## Dependencies
— (none; extends the completed phase-16/79 auth surface additively — the 401/204 contract is unchanged for non-throttled callers)
## Tasks
1. `01_rate_limiter_core.md` — `app/core/rate_limit.py` + the unit suite for the window semantics.
2. `02_auth_routes_wiring.md` — the 429 pre-check + failure/success bookkeeping on both login routes + integration tests.
3. `03_verify_and_commit.md` — full gate (suite + coverage + smoke E2E in isolation) + atomic commit.
## Testing & Quality
- Unit — `tests/unit/test_rate_limit.py` (new): 9 failures → allowed; the 10th failure recorded; the next attempt → `remaining_wait > 0`; two independent IPs independent; a failure older than the window expires (monkeypatch a fake clock or pre-seed timestamps) and unblocks; `reset` clears (a success unblocks immediately); `remaining_wait` is the whole-window seconds, not the per-failure age; fail-open: a pre-corrupted state (e.g. a non-deque entry injected) → `remaining_wait` returns 0, `record_failure` doesn't raise.
- Integration — `tests/integration/test_auth_api.py` (extend): 10 wrong-password logins from the TestClient → 11th → **429** with the generic detail + a `Retry-After` header (int > 0); a correct password after 9 failures → 204 AND the counter is reset (the next 10 failures needed to block again — assert by 9 more failures still 401); token-auth failures share the counter (5 login + 5 token failures → the 11th of either → 429); the non-throttled 401 contract (existing pins) byte-identical.
- E2E (isolation gate per AGENTS.md rule 9): `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green — no UI change; the login form's existing 429 rendering is covered by the integration contract (the generic `HTTP ${status}` line).
- Coverage: **>90%** on `app/` — `app/core/rate_limit.py` is small and fully unit-exercised; the `app/api/auth.py` branches (pre-check hit/miss, record, reset) integration-covered.
## Completion Criteria
- [ ] 11 rapid failed logins from one client: the first 10 → 401 (unchanged detail), the 11th → 429 + `Retry-After`; the token-auth route participates in the same counter (integration pins).
- [ ] A successful login/token-auth resets the IP's counter (integration pin).
- [ ] `uv run pytest tests/unit/test_rate_limit.py -v` green; full `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
- [ ] `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green in isolation.
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] No dependency change (`pyproject.toml` / `uv.lock` untouched); `git diff --stat` limited to `app/core/rate_limit.py`, `app/api/auth.py`, the two test files, phase files.
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(auth): rate-limit failed logins and token-auth attempts with 429 + Retry-After`); phase dir moved to `.agents/phases/complete/`.
## Locked decisions
- **A10 intact** — no new service/table/middleware; one module + two route pre-checks.
- **401 contract unchanged** — the no-enumeration single-401 discipline of phase 16/79 is byte-identical for every non-throttled attempt; 429 is a new status, not a rewording.
- **No frontend change** — `login.js` already renders any non-204 as `Sign-in failed (HTTP ${status}) — try again.` (the 429 reads correctly there).
@@ -0,0 +1,33 @@
# Task 01 — The sliding-window limiter module + unit suite
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
## Objective
`app/core/rate_limit.py` exists with the exact semantics from the phase design block, and `tests/unit/test_rate_limit.py` pins every one of them.
## Work
1. `app/core/rate_limit.py` (new) — implement per the phase overview's design block:
- constants `MAX_FAILURES = 10`, `WINDOW_SECONDS = 900`;
- module state: `_failures: dict[str, deque[float]]` + a `threading.Lock`;
- `record_failure(client_ip: str)` — under the lock: get/create the deque, append `time.monotonic()`, prune entries older than `WINDOW_SECONDS`; **never raises** (wrap the body, `except Exception: return` — the fail-open contract);
- `remaining_wait(client_ip: str) -> int` — under the lock: prune; count in-window entries; if `< MAX_FAILURES` return `0`; else `max(1, int(-oldest - now + WINDOW_SECONDS))` as a ceiling — i.e. `int(math.ceil(WINDOW_SECONDS - (now - oldest)))` clamped to >= 1; **never raises** (fail-open → 0);
- `reset(client_ip: str)` — under the lock: `_failures.pop(client_ip, None)`;
- module docstring: the audit basis (SEC-03), the 10/15-min threshold, the shared-counter intent (both login routes), fail-open rationale (a limiter bug must never lock the owner out), and the `request.client.host` assumption — direct peer, no reverse proxy in the current homelab deployment (proxy deployments would need `X-Forwarded-For` handling — out of scope).
2. `tests/unit/test_rate_limit.py` (new) — pin, in this order:
- 9 × `record_failure("ip")` → `remaining_wait("ip") == 0`;
- a 10th `record_failure` → `remaining_wait("ip") > 0` (and `<= WINDOW_SECONDS`);
- a second IP is unaffected (`remaining_wait("ip2") == 0`);
- expiry: pre-seed one IP's deque with a timestamp `WINDOW_SECONDS + 1` in the past (reach into the module state, or a small `for` loop calling `record_failure` after monkeypatching `time.monotonic` to walk the clock forward) → `remaining_wait` back to 0;
- `reset` after 10 failures → `remaining_wait == 0`;
- `remaining_wait` for an unknown IP → 0;
- fail-open: inject a corrupt entry (`_failures["bad"] = object()` — reach into module state) → `remaining_wait("bad") == 0` and `record_failure("bad")` returns without raising;
- `reset` on an unknown IP → no error.
## Testing & Quality
- `uv run pytest tests/unit/test_rate_limit.py -v` green.
- Coverage: **>90%** on `app/core/rate_limit.py` (the fail-open branches included — they are the point).
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_rate_limit.py -v` — all tests green.
- [ ] `uv run ruff check app/core/rate_limit.py tests/unit/test_rate_limit.py && uv run pyright` clean.
- [ ] `git diff --stat` for this task: exactly the two new files.
@@ -0,0 +1,33 @@
# Task 02 — Wire the limiter into both login routes + integration tests
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
## Objective
`POST /api/login` and `POST /api/token-auth` enforce the 429 pre-check and record/reset the counter, and the integration suite pins the HTTP contract (429 + `Retry-After`, shared counter, success reset, unchanged 401s).
## Work
1. `app/api/auth.py` — add the module constant `TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"` and import the limiter (`from app.core import rate_limit`).
2. `app/api/auth.py::login` — insert at the TOP of the handler (before `check_password`):
- `ip = request.client.host or "unknown"` (the `or` guards a scope with no client — TestClient edge; fail-open spirit);
- `if (wait := rate_limit.remaining_wait(ip)) > 0: raise HTTPException(status_code=429, detail=TOO_MANY_DETAIL, headers={"Retry-After": str(wait)})`;
- in the failure branch: `rate_limit.record_failure(ip)` immediately before the 401;
- in the success branch: `rate_limit.reset(ip)` immediately before the 204.
3. `app/api/auth.py::token_auth` — the identical three insertions (same `ip` derivation, same `TOO_MANY_DETAIL`, same pre-check/record/reset points — the counter is shared by design).
4. `app/api/auth.py` module docstring — extend the two route bullets with the 429 case (one line each: "while the per-IP failure window is exhausted → 429 + `Retry-After`, one generic detail — audit SEC-03").
5. `tests/integration/test_auth_api.py` — extend (keep every existing pin intact; the TestClient's `request.client.host` is a fixed "testclient" value, so these tests share one counter — **add a fixture or helper that calls `rate_limit.reset("testclient")` between tests** so existing tests can't trip the limiter, and document why in the test module docstring):
- 10 × wrong-password `POST /api/login` → 11th → **429**, `detail == TOO_MANY_DETAIL`, `Retry-After` header present and `int(...) > 0`;
- after the 429, a CORRECT password → still 429 (the window is not bypassed by success — success only RESETS a clean counter; while blocked, the pre-check fires first; pin this: it is the intended semantics — an exhausted window stays exhausted until it slides);
- reset: with a clean counter, 9 failures + 1 success (204) → the next 9 failures are all 401 (the counter is back to 0 after the success);
- shared counter: 5 login failures + 5 token failures → the next token-auth attempt → 429;
- the existing non-throttled 401 pins (generic `invalid password` / `invalid token`) still green with the clean-counter fixture.
## Testing & Quality
- `uv run pytest tests/integration/test_auth_api.py -v` green (new + existing pins).
- `uv run pytest tests/unit/test_rate_limit.py -v` still green (no regressions to task 01).
- Coverage: **>90%** on the modified `app/api/auth.py` branches (429 hit/miss, record, reset paths all hit by the integration tests).
## Completion Criteria
- [ ] The 11th rapid failed login from one client is a 429 with `Retry-After` (integration pin); the token route shares the counter (integration pin).
- [ ] A success on a clean counter resets it (integration pin); a success while blocked does not unblock (integration pin).
- [ ] All pre-existing `test_auth_api.py` pins green (the clean-counter fixture is documented).
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,28 @@
# Task 03 — Full gate + atomic commit
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
## Objective
Run the complete phase gate, land the phase as one atomic commit, and move the phase directory to `complete/`.
## Work
1. **Full regression gate** (AGENTS.md rule 9):
- `uv run pytest` — unit + integration green.
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**; confirm `app/core/rate_limit.py` and `app/api/auth.py` show no meaningful uncovered lines in the new branches.
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` — green **in isolation** (this phase's E2E contract: no UI change — `login.js`'s existing non-204 rendering covers the 429).
- `uv run ruff check . && uv run pyright` — clean.
2. **Manual live check** (keep the output in the session log): start the dev server (`uv run uvicorn app.main:app`) and, from the shell, 11 rapid wrong-password logins: `for i in $(seq 11); do curl -s -o /dev/null -w "%{http_code} " -X POST localhost:8000/api/login -H 'Content-Type: application/json' -d '{"password":"wrong"}'; done` → expect `401 ×10, 429` (and the 429 body carries `Retry-After`). Restart the server to clear the counter (the per-process state is by design).
3. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py`, and the phase files:
`fix(auth): rate-limit failed logins and token-auth attempts with 429 + Retry-After`
— body: security audit SEC-03 (2026-09-07) — neither login route throttled, so a reachable deployment took unlimited line-rate guesses; a stdlib-only sliding-window limiter (10 failures / IP / 15 min, shared by both routes, success resets, fail-open) now answers the 11th attempt with 429 + `Retry-After` while every non-throttled 401/204 is byte-identical. No dependency or frontend change.
4. Move the phase directory: `mv .agents/phases/todo/81_login_rate_limit .agents/phases/complete/` and include the move in the same commit.
## Testing & Quality
- This task IS the phase-level gate — the commands above are the completion evidence.
- Coverage: >90% held.
## Completion Criteria
- [ ] The live check shows `401 ×10, 429` (output kept in the session log); the 429 response carries `Retry-After`.
- [ ] `uv run pytest` green; coverage >90%; `tests/e2e/test_smoke.py` green in isolation; ruff + pyright clean.
- [ ] Exactly one new commit; `git show --stat HEAD` lists the five files above + the phase files (todo → complete move) — nothing else (in particular `pyproject.toml` / `uv.lock` / `frontend/` untouched).
- [ ] `.agents/phases/complete/81_login_rate_limit/` exists; `todo/` no longer contains it.
@@ -0,0 +1,71 @@
# Phase 82 — Security headers on every response: strict CSP, no-framing, nosniff
**Source:** `.agents/remediation_plan.md` SEC-04 (security audit 2026-09-07, severity Medium — "No security headers (CSP, X-Frame-Options / frame-ancestors)")
**Story:** n/a (security hardening — audit-derived, no user story)
**Context:** no response in the app carries `Content-Security-Policy`, `X-Frame-Options`, or `X-Content-Type-Options` (checked `app/main.py`, `app/core/caching.py`, all templates — none do). The frontend is No-CDN and was verified for this phase: **no inline `<style>` blocks, no `style="…"` attributes, no `onclick`/`on*` handlers, no `javascript:` URLs, no external hosts** in any `frontend/*.html` or the JS-injected markup (inline `<svg>` elements are CSP-legal; the two `el.style.…` CSSOM writes in `app.js` are not CSP-blocked). `app/main.py` is the single app-assembly point (middleware added in order: `SessionMiddleware` → `configure_caching`'s `CachingMiddleware` → static mount last). `app/core/caching.py` is the existing transport-layer middleware precedent (but it BUFFERS page bodies — this phase's middleware must not).
## Objective
Every response — pages, static assets, API JSON, the SSE chat stream, even 404s — carries a strict same-origin CSP (`frame-ancestors 'none'` included → clickjacking closed), `X-Frame-Options: DENY`, and `X-Content-Type-Options: nosniff`, added by a header-only pure-ASGI middleware that never touches a body (the SSE stream passes through byte-identical).
## Audit basis (read this, not the chat)
- Every page (including the admin sign-in and the admin-only views) can be embedded in a third-party page's iframe: a malicious LAN page can overlay the admin UI and trick the signed-in owner into clicking Sync / Revoke / Delete-source actions (audit PoC: `<iframe src="http://<server>:8000/git-sources.html">` with a benign-looking overlay). `frame-ancestors 'none'` (in the CSP) + `X-Frame-Options: DENY` (legacy fallback) close it.
- The app serves no CSP today: the codebase is XSS-clean (escape-first renderer, `textContent` for data — audit-verified at every `innerHTML` site), but the CSP is the standard second line of defense against any future regression. It is cheap here precisely because of the No-CDN + no-inline-markup verification above: `default-src 'self'` is sufficient without any `'unsafe-inline'` — a genuinely strict policy.
- `nosniff` is the one-line belt-and-braces against MIME-confusion on the static mount.
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
- **A1 — the exact policy:** `Content-Security-Policy: default-src 'self'; base-uri 'none'; frame-ancestors 'none'` — no `script-src`/`style-src`/`connect-src` entries (they inherit `default-src 'self'`); no `'unsafe-inline'` anywhere (verified unnecessary); no `report-uri`/`report-to` (no collector in the homelab — a report would just vanish).
- **A2 — header-only, pure ASGI:** a small pure-ASGI middleware (not `BaseHTTPMiddleware`) — it wraps `send` and writes headers on `http.response.start` only; no body drain, no buffering, so the SSE chat stream and every other response pass through byte-identical (the explicit contrast with `CachingMiddleware`, which does buffer page bodies).
- **A3 — outermost position:** registered in `app/main.py` AFTER `configure_caching(app)` (last `add_middleware` = outermost), so even responses built by the static catch-all / 404 handler carry the headers.
- **A4 — the three headers only:** CSP + `X-Frame-Options: DENY` + `X-Content-Type-Options: nosniff`. No `Referrer-Policy`, no `Permissions-Policy`, no HSTS (the app serves plain homelab HTTP — HSTS would be wrong here; TLS is the documented non-goal).
## Design (shared by all tasks — the executor reads this, not the chat)
- **`app/core/security_headers.py` (new)**:
```python
CSP = "default-src 'self'; base-uri 'none'; frame-ancestors 'none'"
class SecurityHeadersMiddleware:
"""Header-only, pure-ASGI — never reads or buffers a body (SSE-safe)."""
def __init__(self, app): self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] != "http":
await self.app(scope, receive, send)
return
async def send_wrapper(message):
if message["type"] == "http.response.start":
headers = MutableHeaders(scope=message)
headers["Content-Security-Policy"] = CSP
headers["X-Frame-Options"] = "DENY"
headers["X-Content-Type-Options"] = "nosniff"
await send(message)
await self.app(scope, receive, send_wrapper)
```
Module docstring: the audit basis (SEC-04), the No-CDN verification that makes `'unsafe-inline'` unnecessary (re-verify if inline scripts/styles are ever introduced — the policy would break the page loudly, which is the feature), the pure-ASGI rationale (body untouched — SSE stream byte-identical, unlike `CachingMiddleware`'s page buffering).
- **`app/main.py`** — import + `app.add_middleware(SecurityHeadersMiddleware)` placed immediately AFTER the `configure_caching(app)` call (with a one-line comment: outermost on purpose — headers on every response incl. static 404s). Nothing else in `main.py` moves.
- **Not touched:** `app/core/caching.py`, any template, any JS, the API layer, the No-CDN surface.
## Dependencies
— (none; standalone transport hardening — wraps the whole app, no route or template change)
## Tasks
1. `01_headers_middleware.md` — `app/core/security_headers.py` + unit tests (incl. the SSE-stream passthrough pin).
2. `02_registration_and_integration.md` — `app/main.py` registration + integration tests on real app responses.
3. `03_e2e_and_commit.md` — the dedicated Playwright suite + full gate + atomic commit.
## Testing & Quality
- Unit — `tests/unit/test_security_headers.py` (new): a minimal ASGI test-app wrapped by the middleware → a plain response carries all three headers with the exact values; a 404-shaped response carries them too; a **streaming** response (a generator yielding two chunks, `media_type="text/event-stream"`) — the body is consumed byte-identical (both chunks, in order) AND the headers are present (the SSE passthrough pin); a non-http scope (e.g. `websocket`) passes through with the app called and no `send` wrapping error.
- Integration — `tests/integration/test_security_headers.py` (new): against the real app (the `conftest` client): `GET /` (a page) → all three headers, CSP exactly the A1 string; `GET /api/health` → all three; `GET /assets/styles.css` → all three; `GET /nope` (404) → all three. (If the existing `tests/integration/test_caching_revalidation.py` fixture pattern for the app client exists, reuse it — same setup, no new fixtures.)
- E2E (mandatory, dedicated suite — this phase changes what the browser receives): `tests/e2e/test_security_headers.py` — (1) navigate to the chat page: the navigation `response.headers` carry all three (CSP exact); (2) the page actually works under the CSP — no CSP violations: collect `console` messages during load + one chat turn is NOT required (mock LLM is not needed for a header test) — assert zero console messages matching `/Content Security Policy/` and that the stylesheet applied (computed `background-color` of `body` is not `rgba(0, 0, 0, 0)` — the page painted, not a CSP-broken white page); (3) navigate to `/sources.html` (a second page) → headers present, no CSP violations. Run in isolation per AGENTS.md rule 9.
- Coverage: **>90%** on `app/` — the new module is small and unit-exercised; `app/main.py`'s registration is covered by every integration/E2E boot.
## Completion Criteria
- [ ] `curl -sI localhost:8000/ | grep -iE "content-security-policy|x-frame-options|x-content-type"` shows all three with the exact A1 CSP value (manual check, output kept in the session log); same for `/api/health`, `/assets/styles.css`, and a 404 path.
- [ ] The SSE chat stream is byte-identical through the middleware: the unit streaming pin + the existing chat E2E suites (e.g. `tests/e2e/test_chat_rag.py`) green — a body-touching regression would break them.
- [ ] `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` green **in isolation** — headers on real page loads + zero CSP violations + the page painted.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] No template/JS/`pyproject.toml`/`uv.lock` diff; `git diff --stat` limited to the two new files, `app/main.py`, the three test files, phase files.
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(security): add CSP, X-Frame-Options and nosniff headers to every response`); phase dir moved to `.agents/phases/complete/`.
## Locked decisions
- **No-CDN contract intact** — the CSP is `default-src 'self'` because the audit verified zero external hosts and zero inline scripts/styles; introducing either later breaks pages loudly (the policy does the enforcing).
- **No HSTS / Referrer-Policy / Permissions-Policy** (owner A4) — plain homelab HTTP is the documented posture; the three headers above are the scope.
- **SSE byte-identity** — the middleware wraps `send` for headers only; any body change is a phase failure (the chat E2E suites are the tripwire).
@@ -0,0 +1,23 @@
# Task 01 — The pure-ASGI header middleware + unit suite
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
## Objective
`app/core/security_headers.py` exists exactly per the phase design block, and `tests/unit/test_security_headers.py` pins the header values, the 404 path, the SSE streaming passthrough (byte-identical body + headers present), and the non-http scope passthrough.
## Work
1. `app/core/security_headers.py` (new) — implement per the phase overview's design block: the `CSP` constant (the exact A1 string), `SecurityHeadersMiddleware` as a pure-ASGI class wrapping `send` (headers written on `http.response.start` via `starlette.datastructures.MutableHeaders(scope=message)`), the `scope["type"] != "http"` fast path, and the module docstring content listed in the design block (audit basis, the No-CDN verification, the pure-ASGI/SSE rationale).
2. `tests/unit/test_security_headers.py` (new):
- a tiny ASGI `test_app` (a plain 200 `Response`-like dict sequence, or starlette's `Response` wrapped in an ASGI callable) + the middleware: assert all three headers with exact values;
- a 404-shaped response (status 404) → same three headers;
- a streaming response (an async generator yielding `data: a\n\n` then `data: b\n\n`, `media_type="text/event-stream"`) → consume the full body via the ASGI `receive`/`send` protocol (drive it with a small async client loop, or `httpx.AsyncClient(transport=ASGITransport(...))` if already used in the unit suite) → body is exactly the two frames in order AND the three headers are present (the SSE pin);
- a `websocket` scope → the app is called, no crash, no header injection attempt.
## Testing & Quality
- `uv run pytest tests/unit/test_security_headers.py -v` green.
- Coverage: **>90%** on `app/core/security_headers.py` (every branch: http start, http other messages, non-http scope).
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_security_headers.py -v` — all tests green, including the SSE byte-identity pin.
- [ ] `uv run ruff check app/core/security_headers.py tests/unit/test_security_headers.py && uv run pyright` clean.
- [ ] `git diff --stat` for this task: exactly the two new files.
@@ -0,0 +1,26 @@
# Task 02 — Register the middleware + integration tests on real app responses
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
## Objective
The middleware wraps the whole app (outermost), and the integration suite pins the three headers on a page, an API response, a static asset, and a 404 — against the real app.
## Work
1. `app/main.py` — import `SecurityHeadersMiddleware` and add `app.add_middleware(SecurityHeadersMiddleware)` immediately AFTER the `configure_caching(app)` call, with the one-line comment from the design block (outermost on purpose: headers on every response, including static catch-all 404s). Do not reorder or re-add any existing middleware.
2. `tests/integration/test_security_headers.py` (new) — use the same app/client fixture pattern as `tests/integration/test_caching_revalidation.py` (reuse its setup; do not duplicate the client factory if a shared helper exists):
- `GET /` → 200 + all three headers; `Content-Security-Policy` exactly `default-src 'self'; base-uri 'none'; frame-ancestors 'none'`;
- `GET /api/health` → 200 + all three;
- `GET /assets/styles.css` → 200 + all three;
- `GET /definitely-not-a-page` → 404 + all three (the static catch-all's 404 still carries them — the outermost-position proof);
- a regression pin: the page body is still the rewritten HTML with `?v=` asset refs (assert one `?v=` occurrence in the body — the two middlewares coexist and the caching rewrite still runs, i.e. header middleware did not swallow/alter the CachingMiddleware's work).
3. Run the existing caching integration suite (`tests/integration/test_caching_revalidation.py`) to prove no interaction regression (it must stay green untouched).
## Testing & Quality
- `uv run pytest tests/integration/test_security_headers.py tests/integration/test_caching_revalidation.py -v` green.
- Coverage: **>90%** on `app/` (the registration line is hit by every integration/E2E boot; no new uncovered branches).
## Completion Criteria
- [ ] All four real-response pins pass (page / API / asset / 404) with the exact CSP string.
- [ ] The `?v=` coexistence pin passes (caching rewrite intact under the new outermost middleware).
- [ ] `tests/integration/test_caching_revalidation.py` green untouched.
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,37 @@
# Task 03 — Dedicated Playwright suite + full gate + atomic commit
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
## Objective
Prove the headers reach a real browser AND the strict CSP does not break the page (no violations, page painted), run the full gate, and land the phase as one atomic commit.
## Work
1. `tests/e2e/test_security_headers.py` (new) — follow the house E2E conventions (read `tests/e2e/conftest.py` + one existing suite, e.g. `test_header_consistency.py`, for the fixtures and the app-server pattern):
- **Headers on a real navigation:** `page.goto` the chat page (the shell at `/`) → the navigation response's headers carry all three, CSP exactly the A1 string; `page.goto` `/sources.html` → same (a second page).
- **No CSP violations:** attach a `page.on("console")` listener before navigation; after both loads assert zero console messages whose text matches `/Content Security Policy/i` (Chromium reports CSP denials to the console).
- **The page painted (CSP not breaking assets):** after the chat-page load, `document.querySelector('body')`'s computed `background-color` is not `rgba(0, 0, 0, 0)` (the dark-tech palette applied — styles loaded under the CSP) and the shell's root element is present (e.g. `#main` — the page booted, not a white-broken document).
- No LLM interaction needed (the page load is the contract) — the suite must not depend on the aipi endpoint.
2. **Manual live check** (keep the output in the session log): `curl -sI localhost:8000/ | grep -iE "content-security-policy|x-frame-options|x-content-type"` + the same for `/api/health`, `/assets/styles.css`, `/nope` (404).
3. **Full regression gate** (AGENTS.md rule 9):
- `uv run pytest` — unit + integration green.
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
- `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` — green **in isolation** (this phase's mandatory E2E).
- ONE chat-stream suite to prove SSE byte-identity end-to-end: `uv run pytest tests/e2e/test_chat_rag.py -v --no-cov` green (the mock-LLM chat suite — a body-touching regression in the middleware would break the stream).
- `uv run ruff check . && uv run pyright` — clean.
4. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/security_headers.py`, `app/main.py`, the three test files, and the phase files:
`fix(security): add CSP, X-Frame-Options and nosniff headers to every response`
— body: security audit SEC-04 (2026-09-07) — no response carried framing/XSS-mitigation headers: any page (incl. the admin UI) could be iframe-embedded for clickjacking. A header-only pure-ASGI middleware (outermost) now sets `default-src 'self'; base-uri 'none'; frame-ancestors 'none'` (no 'unsafe-inline' — the No-CDN + no-inline-markup audit made the strict policy possible), `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff` on every response including 404s; the SSE stream passes byte-identical (unit streaming pin + chat E2E regression). No template/JS/dependency change.
5. Move the phase directory: `mv .agents/phases/todo/82_security_headers .agents/phases/complete/` and include the move in the same commit.
## Testing & Quality
- E2E: the dedicated suite green in isolation (headers + zero CSP violations + painted page).
- Regression: `test_chat_rag.py` green (SSE byte-identity through the real middleware).
- Coverage: >90% held.
## Completion Criteria
- [ ] `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` green in isolation (both pages, exact CSP, no violations, page painted).
- [ ] `uv run pytest tests/e2e/test_chat_rag.py -v --no-cov` green (the stream is untouched).
- [ ] The `curl` header check output (4 URLs) kept in the session log.
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean.
- [ ] Exactly one new commit; `git show --stat HEAD` lists the two app files, the three test files, and the phase files (todo → complete move) — nothing else.
- [ ] `.agents/phases/complete/82_security_headers/` exists; `todo/` no longer contains it.
@@ -0,0 +1,65 @@
# Phase 83 — Bound the anonymous saved-chat payload sizes at the schema boundary
**Source:** `.agents/remediation_plan.md` SEC-05 (security audit 2026-09-07, severity Medium — "Unbounded payload sizes on the anonymous public chat-save endpoints")
**Story:** n/a (security hardening — audit-derived, no user story)
**Context:** `app/schemas.py` — `ChatMessage` (`who`/`text: Field(min_length=1)` **no max**/`sources: list[SourceRef] | None`/`suggestions: list[str] | None`/`thinking: str | None` **no max**/`tools: list[ToolCall] | None`/`stopped`, `extra="forbid"`), `SourceRef` (`source`/`path`/`title` — all bare `str`, **no max** — shared with the server-built `ChatDoneEvent.sources` SSE shape), `ToolCall` (`name: str`/`argument: str | None` — bare, **no max**), `SavedChatCreate` (`title: max_length=500`/`messages: Field(min_length=1)` **no max_items**/`share`), `SavedChatUpdate` (same message list) — consumed by the **public** (no-session) routes `POST /api/chats` and `PUT /api/chats/{chat_id}` in `app/api/chats.py` (phase 55 A1: the write surface is public — the row id is the credential). `ChatMessage` is shared with `HistoryTurn` (`text: max_length=32_000`, `thinking: max_length=32_000` — the existing caps to mirror) and with the server-built `ChatDoneEvent`/SSE shapes. The DB side: `documents.source String(120)`, `documents.path String(1000)`, `documents.title String(500)` (the column lengths the caps mirror), `saved_chats.messages` is JSONB (unbounded at the DB level — the pydantic boundary is the only gate). `tests/unit/` has no existing `test_schemas.py`; `tests/integration/test_chats_api.py` exists (the phase-50/51/53/55 contract pins — the 201/404/409/share matrix).
## Objective
An anonymous `POST/PUT /api/chats` can no longer carry a single arbitrarily large string, an unbounded message list, or unbounded nested lists: every `ChatMessage`/`SourceRef`/`ToolCall`/messages-list field gets a pydantic cap (422 at the boundary, house style), sized to the realistic `bor.chat.v1` record the UI produces — while a normal save (the E2E suites' payloads) still lands 201 byte-for-byte.
## Audit basis (read this, not the chat)
- `ChatMessage.text` has `min_length=1` and **no `max_length`** (unlike its sibling `HistoryTurn.text` at 32 000), and NO list field anywhere in the saved-chat shape has `max_items` — an anonymous caller can POST one 200 MB JSON string (`{"messages":[{"who":"user","text":"aaa…"}]}`) and create unlimited such rows: an unauthenticated memory + storage DoS on the app's only anonymous write surface (audit PoC in the plan).
- The fix is boundary-only (the phase-56 house style: fail loud at the schema with a 422 — no route code changes at all; FastAPI's request validation rejects before the handler runs, nothing is stored).
- The caps mirror the EXISTING `HistoryTurn` caps for the text fields (32 000 — a single chat message longer than that is already rejected on the chat path, so a saved chat can never legitimately carry more) and the DB column lengths for the source-ref fields (a `SourceRef` is built from `documents` rows server-side — `source ≤120`, `path ≤1000`, `title ≤500` — so the SSE `done` event can never trip the new caps: the server's own values always fit).
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
- **A1 — the exact caps** (pydantic boundary, 422 on overflow):
- `ChatMessage.text`: `max_length=32_000` (mirror `HistoryTurn.text`);
- `ChatMessage.thinking`: `max_length=32_000` (mirror `HistoryTurn.thinking`);
- `ChatMessage.sources`: `max_items=20` (top-N docs + agent reads — the UI shows a handful; 20 is 10× the realistic max);
- `ChatMessage.suggestions`: `max_items=50`, each item `max_length=200` (chips are short deterministic strings — `derive_suggestions` produces ≤ ~80 chars);
- `ChatMessage.tools`: `max_items=50` (one entry per tool call; the round cap is 10 and even generous multi-call turns stay far below 50);
- `SourceRef`: `source max_length=120`, `path max_length=1000`, `title max_length=500` (mirror the `documents` column lengths — server-built SSE values always fit);
- `ToolCall`: `name max_length=100`, `argument max_length=2000 | None` (the combined `source/path` identity is ≤ 120 + 1 + 1000; 2 000 is 2× headroom for a grep pattern);
- `SavedChatCreate.messages` / `SavedChatUpdate.messages`: `max_items=200` (well past any realistic conversation — the chat history budget itself is 40 turns — and far below a DoS-sized list).
- **A2 — boundary-only:** NO route/handler changes in `app/api/chats.py` (FastAPI validates the pydantic model before the handler — the 422 is the framework's standard validation response); the stored-row contract (JSONB round-trip, `extra="forbid"`, the null-safe restore path) is untouched.
- **A3 — `SourceRef` is shared:** the caps on `SourceRef` also constrain the client-side `sources` inside saved chats AND are satisfied by every server-built `ChatDoneEvent.sources` (column-length mirror) — the SSE path is provably unaffected (pinned by an integration test).
## Design (shared by all tasks — the executor reads this, not the chat)
- **`app/schemas.py`** — the ONLY file changed in `app/`:
- `SourceRef`: `source: str = Field(max_length=120)`, `path: str = Field(max_length=1000)`, `title: str = Field(max_length=500)` (docstring: caps mirror the `documents` column lengths — server-built SSE refs always fit; client-saved refs are bounded at the boundary).
- `ToolCall`: `name: str = Field(max_length=100)`, `argument: str | None = Field(default=None, max_length=2000)`.
- `ChatMessage`: `text: str = Field(min_length=1, max_length=32_000)`, `thinking: str | None = Field(default=None, max_length=32_000)`, `sources: list[SourceRef] | None = Field(default=None, max_items=20)`, `suggestions: list[str] | None = Field(default=None, max_items=50)` with item length enforced by `Field(max_length=200)` on the list item type (pydantic v2: annotate `list[Annotated[str, Field(max_length=200)]]` or a small `_Chip = Annotated[str, Field(max_length=200)]` alias — keep the JSON shape identical), `tools: list[ToolCall] | None = Field(default=None, max_items=50)`.
- `SavedChatCreate.messages` / `SavedChatUpdate.messages`: `Field(min_length=1, max_items=200)`.
- Update the affected docstrings: each cap's one-line rationale (the A1 mirror sources) — the file's dense-docstring house style.
- **Not touched:** `app/api/chats.py` (validation happens before the handlers — zero route diff), the frontend (the UI's real payloads are far inside every cap — the E2E save suites prove it), the DB (no migration — JSONB stays unbounded at rest; the boundary is the gate, A2).
- **422 shape:** FastAPI's standard validation error body (the house boundary response — same shape the existing `min_length` violations already produce; no custom error copy).
## Dependencies
— (none; boundary-only hardening on the completed phase-50/51/55 saved-chat surface — the 201/404/409/share contract is unchanged for in-cap payloads)
## Tasks
1. `01_schema_caps.md` — the `app/schemas.py` cap changes + the unit suite (one test per cap, boundary values included).
2. `02_integration_and_e2e.md` — the oversized-422 integration pins + the dedicated Playwright suite + the SSE `done`-event unaffected pin.
3. `03_verify_and_commit.md` — full gate (incl. the existing chat-save E2E suites) + atomic commit.
## Testing & Quality
- Unit — `tests/unit/test_schemas.py` (new; the first schema-boundary suite): for EACH cap — a value exactly at the cap validates; one past it raises a pydantic `ValidationError` naming the field (`text` 32_000/32_001, `thinking` same, `sources` 20/21 items, `suggestions` 50/51 items + a 200/201-char item, `tools` 50/51, `SourceRef` source/path/title 120/1000/500 boundaries, `ToolCall` name 100/101 + argument 2000/2001, `SavedChatCreate.messages` 200/201 items, `SavedChatUpdate.messages` 200/201) — plus a regression pin: a realistic `bor.chat.v1` payload (a few messages, sources, tools, thinking) validates cleanly and round-trips `model_dump()` (the stored-shape contract).
- Integration — `tests/integration/test_chats_api.py` (extend, existing pins intact): anonymous `POST /api/chats` with a 32_001-char text → **422** AND no row created (list stays the same length); `messages` with 201 items → 422; a 21-item `sources` list → 422; the existing 201 create / share / unshare / stale pins green; an SSE regression pin: a `ChatDoneEvent` built from a full-length `Document` row (source 120 / path 1000 / title 500 — construct the row values at the column maxima) still serializes (A3: the server-built refs fit the new caps — build the event and `model_dump()` it in an integration test next to the existing chat pins).
- E2E (dedicated suite — this phase's contract is reachable from a browser's network layer): `tests/e2e/test_chat_save_payload_limits.py` — using Playwright's API request context against the running app (the house pattern — `page.request` or the context's request API): anonymous `POST /api/chats` with an oversized text (e.g. 40_000 chars) → **422** (no session needed — the surface is public, exactly the audit vector); a normal small save → 201 + `id` in the body (the happy path still works end-to-end). Run in isolation per AGENTS.md rule 9.
- Regression E2E: the existing save flows stay green — `tests/e2e/test_chat_history.py` (save/restore through the real UI — the in-cap proof).
- Coverage: **>90%** on `app/` (the changed file is `app/schemas.py` — declarative, exercised by every unit/integration test).
## Completion Criteria
- [ ] Every A1 cap is pinned at both boundaries (at-cap passes, over-cap 422s) in `tests/unit/test_schemas.py`.
- [ ] Anonymous oversized `POST /api/chats` → 422 with **no row stored** (integration pin); the same vector from the Playwright suite → 422 (E2E pin).
- [ ] The SSE `done`-event pin passes at the column-maximum source-ref lengths (A3).
- [ ] `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` green in isolation; `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` green (real UI saves unaffected).
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `git diff --stat` limited to `app/schemas.py`, the three test files, phase files (NO `app/api/chats.py` diff, no migration, no `pyproject.toml`/`uv.lock`/`frontend/` diff).
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(api): bound anonymous saved-chat payload sizes at the schema boundary`); phase dir moved to `.agents/phases/complete/`.
## Locked decisions
- **A2 boundary-only** — no route/handler/migration change; the pydantic 422 IS the control (house style, phase 56 precedent).
- **Caps mirror existing invariants** (A1) — `HistoryTurn`'s 32 000 text cap and the `documents` column lengths are the sizing sources, so no legitimate payload (chat-path history OR server-built SSE refs) can ever trip a cap; only oversized anonymous input does.
- **JSONB stored shape untouched** — `extra="forbid"` and the null-safe round-trip contract (the phase-14/50 restore path) are byte-identical for in-cap payloads (the unit round-trip pin).
@@ -0,0 +1,32 @@
# Task 01 — The schema caps + unit boundary suite
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
## Objective
`app/schemas.py` carries every A1 cap from the phase overview, and `tests/unit/test_schemas.py` pins each cap at both boundaries plus the realistic-payload round-trip regression.
## Work
1. `app/schemas.py` — apply the caps exactly as the phase overview's design block lists them:
- `SourceRef` — `source`/`path`/`title` → `Field(max_length=120/1000/500)`;
- `ToolCall` — `name` → `Field(max_length=100)`, `argument` → `Field(default=None, max_length=2000)`;
- `ChatMessage` — `text` → `Field(min_length=1, max_length=32_000)`, `thinking` → `Field(default=None, max_length=32_000)`, `sources` → `Field(default=None, max_items=20)`, `suggestions` → a list of `_Chip` (`Annotated[str, Field(max_length=200)]`) with `Field(default=None, max_items=50)`, `tools` → `Field(default=None, max_items=50)`;
- `SavedChatCreate.messages` + `SavedChatUpdate.messages` → `Field(min_length=1, max_items=200)`;
- docstrings: one rationale line per cap group (the A1 mirror sources — `HistoryTurn` caps / `documents` column lengths), matching the file's dense style.
- Verify no JSON-shape change: the models still accept/reject exactly the same KEYS (`extra="forbid"` untouched) — only value bounds are added.
2. `tests/unit/test_schemas.py` (new) — one test per cap at BOTH boundaries (at-cap validates; one-over raises `ValidationError` — assert the failing field name via `e.errors()[0]["loc"]`):
- `ChatMessage.text` 32_000 / 32_001; `thinking` 32_000 / 32_001 (and `None` still valid);
- `sources` 20 / 21 items; `suggestions` 50 / 51 items + a single 200 / 201-char item; `tools` 50 / 51;
- `SourceRef.source` 120 / 121, `.path` 1000 / 1001, `.title` 500 / 501;
- `ToolCall.name` 100 / 101, `.argument` 2000 / 2001 (and `None` still valid);
- `SavedChatCreate.messages` 200 / 201 items, `SavedChatUpdate.messages` 200 / 201;
- the realistic-payload regression: a full `bor.chat.v1`-shaped `SavedChatCreate` (4–8 messages mixing user/brain, one brain message with `thinking` + `tools` + `sources`, one with `suggestions` + `stopped`) → validates, and `model_dump()` of the messages equals the input dict (the stored-shape round-trip contract, `None`-keys preserved).
## Testing & Quality
- `uv run pytest tests/unit/test_schemas.py -v` green.
- Coverage: **>90%** on `app/schemas.py` (declarative — exercised by every test in the new file).
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_schemas.py -v` — every boundary test green (at-cap passes, over-cap 422-shaped `ValidationError` with the right `loc`).
- [ ] The round-trip regression test green (stored shape unchanged for in-cap payloads).
- [ ] `uv run pytest tests/unit/ -q` green (no unit regression — in particular any test that constructs `ChatMessage`/`SourceRef` values still passes).
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,31 @@
# Task 02 — Integration pins + the dedicated Playwright suite
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
## Objective
The HTTP contract is pinned: anonymous oversized saves 422 and store nothing, the SSE `done`-event shape is provably unaffected at the column maxima, and the dedicated E2E suite drives the exact audit vector through a real browser's network layer.
## Work
1. `tests/integration/test_chats_api.py` — extend (every existing pin stays green; if the file's fixtures reset the DB between tests, the new tests follow the same pattern):
- anonymous `POST /api/chats` with one message whose `text` is 32_001 chars → **422** AND the subsequent admin `GET /api/chats` list length is unchanged (nothing stored);
- `POST /api/chats` with 201 messages (minimal valid each) → 422;
- `POST /api/chats` with one message carrying a 21-item `sources` list (valid `SourceRef` shapes) → 422;
- `PUT /api/chats/{id}` (an existing row) with an oversized message → 422 and the row content unchanged (a GET shows the original text);
- the happy path regression: a normal small save → 201 (the existing pins already cover this — confirm green).
2. **The A3 SSE pin** — in the same file (or the existing chat-API integration file where the `ChatDoneEvent` shape is already exercised — `tests/integration/test_chat_api.py`): build the event from maximum-length values — `SourceRef(source="s"*120, path="p"*1000, title="t"*500)` inside a `ChatDoneEvent(deflected=False, sources=[…], suggestions=[])` → `model_dump()` succeeds (the server-built refs fit the new caps; a failure here would mean the caps broke the SSE contract).
3. `tests/e2e/test_chat_save_payload_limits.py` (new) — house E2E conventions (read `tests/e2e/conftest.py` + `test_chat_history.py` for the server/client fixtures):
- the audit vector, end-to-end and anonymous (NO login): via the Playwright request API (`page.request.post("/api/chats", data={...})` on a fresh page) POST one message with a 40_000-char `text` → expect **422**;
- the happy path in the same suite: a small valid save → **201** with an `id` field (proves the boundary didn't break the real flow from the browser layer);
- no LLM dependency (the endpoints are DB-only).
## Testing & Quality
- `uv run pytest tests/integration/test_chats_api.py -v` green (new + existing).
- The A3 pin green wherever it lands.
- `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` green **in isolation** (this phase's mandatory E2E).
- Coverage: **>90%** on `app/` (no new `app/` code — the gate is regression + boundary proof).
## Completion Criteria
- [ ] All four oversized-422 integration pins pass (text / message-count / sources / PUT), each with the "nothing stored / row unchanged" assertion.
- [ ] The A3 SSE pin passes at the column-maximum lengths.
- [ ] The dedicated E2E suite green in isolation (422 anonymous + 201 small save).
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,28 @@
# Task 03 — Full gate + atomic commit
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
## Objective
Run the complete phase gate (including the real-UI save regression), land the phase as one atomic commit, and move the phase directory to `complete/`.
## Work
1. **Full regression gate** (AGENTS.md rule 9):
- `uv run pytest` — unit + integration green.
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
- `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` — green **in isolation** (this phase's E2E).
- `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` — green (the real UI save/restore flow with in-cap payloads — the no-regression proof for the surface the caps sit on).
- `uv run ruff check . && uv run pyright` — clean.
2. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/schemas.py`, `tests/unit/test_schemas.py`, `tests/integration/test_chats_api.py` (+ the chat-API file if the A3 pin landed there), `tests/e2e/test_chat_save_payload_limits.py`, and the phase files:
`fix(api): bound anonymous saved-chat payload sizes at the schema boundary`
— body: security audit SEC-05 (2026-09-07) — `POST/PUT /api/chats` are public (phase 55) and every `ChatMessage`/list field was unbounded, so an anonymous caller could store arbitrarily large JSONB rows (memory + storage DoS). Pydantic caps at the boundary (text/thinking 32 000 mirroring `HistoryTurn`, `SourceRef` mirroring the `documents` column lengths, list `max_items`, `messages` ≤ 200) → 422 on overflow, nothing stored; the 201/404/share contract and the stored JSONB shape are unchanged for in-cap payloads (round-trip pin + real-UI E2E regression). Boundary-only: no route, migration, or frontend change.
3. Move the phase directory: `mv .agents/phases/todo/83_chat_save_payload_limits .agents/phases/complete/` and include the move in the same commit.
## Testing & Quality
- This task IS the phase-level gate — the commands above are the completion evidence.
- Coverage: >90% held.
## Completion Criteria
- [ ] `uv run pytest` green; coverage >90%; both E2E suites green (the dedicated one in isolation, `test_chat_history.py` as the regression).
- [ ] `uv run ruff check . && uv run pyright` clean.
- [ ] Exactly one new commit; `git show --stat HEAD` lists the files above + the phase files (todo → complete move) — in particular NO `app/api/chats.py`, NO `alembic/`, NO `frontend/`, NO `pyproject.toml`/`uv.lock`.
- [ ] `.agents/phases/complete/83_chat_save_payload_limits/` exists; `todo/` no longer contains it.
@@ -0,0 +1,63 @@
# Phase 84 — Mask credentials in docs-push and LLM error surfaces
**Source:** `.agents/remediation_plan.md` SEC-08 (security audit 2026-09-07, severity Medium — "Docs-push 502 surfaces git stderr verbatim (possible credential echo)"), with the SEC-13 fold-in (Low — "LLM error strings echo `llm_base_url`")
**Story:** n/a (security hardening — audit-derived, no user story)
**Context:** `app/api/sync.py` — the existing masker: `_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")` + `_sanitize_error(message)` (masks `user:pass@` in any error text; the sync failure detail runs through it — "no secrets in the UI"). `app/api/git_sources.py` — imports it (`from app.api.sync import _sanitize_error`) for upload-failure details. `app/api/doc_drafts.py::push_doc_draft` — the GAP: `except DocsPushError as exc: raise HTTPException(502, detail=str(exc))` — git's stderr verbatim (a `BOR_DOCS_REPO` URL with embedded credentials would reach the browser + server log on a failed push). `app/core/docs_push.py` — `push_document` raises `DocsPushError(str(err))` carrying `GitSyncError`'s "git … failed (exit N): <stderr>" text. `scripts/git_sync.py::run_git` — the single git invocation point (list args, no shell — the error text is git's own stderr). `app/rag/llm.py` — SEC-13: five error f-strings embed `self.settings.llm_base_url` raw (lines 288, 327, 331, 337, 486 — embed + chat + stream failure paths) — a base URL configured with embedded credentials would echo into SSE `error` frames and logs. `tests/integration/test_doc_drafts_api.py` (the existing 502-contract pins — git stderr in the detail) and `tests/unit/test_llm_client.py` (the error-string pins) are the regression anchors.
## Objective
No error surface ever ships a `user:pass@` credential: the sync masker moves to a shared core module, the docs-push 502 detail runs through it (SEC-08), and the LLM error construction sanitizes the base URL (SEC-13) — while every non-credential error string (the sync/git/LLM tests' pinned copies) stays byte-identical.
## Audit basis (read this, not the chat)
- SEC-08: `BOR_DOCS_REPO` is documented as "any remote (URL or local path)" — an `https://user:token@host/…` URL is a normal config shape. On a failed push (revoked token, network), git's stderr echoes the remote URL; `push_doc_draft` returns it verbatim as the 502 `detail` → the credential lands in the browser (admin screen) and in the server log (`logger` writes the 502? the detail is the response body — and `run_git`'s stderr also surfaces into any traceback logging). The sync path already solved this exact problem (`_sanitize_error`) — the docs-push path simply never got the treatment.
- SEC-13 (fold-in): the five `llm.py` error f-strings interpolate `llm_base_url` raw — same class of leak on the LLM side (the real deployment uses a bare URL + header key, so today it is latent).
- The masker is deliberately a NARROW regex (only the `user:pass@` userinfo shape — git/HTTP convention): it must not rewrite ordinary text (`a: b @ c` without the userinfo run, plain hosts, emails in prose stay untouched as long as they don't match the userinfo pattern — the existing sync tests pin the behavior; the move must be byte-identical).
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
- **A1 — shared core module:** `app/core/errors.py` (new) owns `sanitize_error(message: str) -> str` — the regex + logic move VERBATIM from `app/api/sync.py` (same `_CREDS_RE` pattern, same `sub("*****@", …)` replacement — byte-identical behavior). `app/api/sync.py` keeps the private name as an alias (`from app.core.errors import sanitize_error as _sanitize_error`) so `app/api/git_sources.py`'s existing import and every sync test stay untouched (zero caller diff outside the two target surfaces).
- **A2 — SEC-08 application point:** `app/api/doc_drafts.py::push_doc_draft` — the `except DocsPushError` clause becomes `raise HTTPException(502, detail=sanitize_error(str(exc))) from None` (the ONLY line that changes there).
- **A3 — SEC-13 application point:** `app/rag/llm.py` — the five error f-strings sanitize the URL at construction: a module-level `self._base = sanitize_error(settings.llm_base_url)` is NOT introduced (the client is constructed with settings; simpler and more local: each f-string uses `sanitize_error(self.settings.llm_base_url)`). With a credential-free URL (every real deployment) `sanitize_error` is a no-op → the existing error-string pins in `tests/unit/test_llm_client.py` stay byte-identical green.
- **A4 — no other surfaces in scope:** the sync/git-sources upload details already sanitize; the chat SSE error frames carry fixed operator copy (not git/URL text) — untouched.
## Design (shared by all tasks — the executor reads this, not the chat)
- **`app/core/errors.py` (new):**
```python
_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")
def sanitize_error(message: str) -> str:
"""Mask user:pass@ userinfo in an error string (no secrets in the UI/logs)."""
return _CREDS_RE.sub("*****@", message)
```
Module docstring: the audit basis (SEC-08/SEC-13), the "narrow userinfo-regex only" contract (byte-identical for credential-free text — the sync tests are the proof), the replacement shape (`*****@` — the existing sync copy).
- **`app/api/sync.py`** — delete the local `_CREDS_RE` + `_sanitize_error` definitions; add `from app.core.errors import sanitize_error as _sanitize_error` (the `_run_sync` failure path and its tests see the identical function under the identical private name).
- **`app/api/doc_drafts.py`** — import `sanitize_error` from `app.core.errors`; the `push_doc_draft` except clause per A2; the route docstring's outcome-4 line updated (`502 with the SANITIZED git stderr in the detail — credential userinfo masked, the phase-59 `GitSyncError → detail` mapping kept`).
- **`app/rag/llm.py`** — import `sanitize_error`; the five f-strings (lines 288/327/331/337/486 today) wrap the URL: `f"embeddings request to {sanitize_error(self.settings.llm_base_url)} failed: {e}"` etc. (the message copy around it unchanged).
- **Not touched:** `scripts/git_sync.py` (the stderr source — unchanged), `app/core/docs_push.py` (raises the raw text — the SANITIZE point is the API boundary, where the response is built), the sync/upload failure paths' behavior (byte-identical via the alias).
## Dependencies
— (none; standalone error-surface hardening — builds on the completed phase-32 sync sanitizer and phase-59 docs-push; no behavior change for credential-free errors)
## Tasks
1. `01_shared_sanitizer.md` — `app/core/errors.py` + the sync alias refactor + unit suite.
2. `02_apply_docs_push_and_llm.md` — the docs-push 502 + the five LLM error sites + the integration/unit pins.
3. `03_verify_and_commit.md` — full gate + atomic commit.
## Testing & Quality
- Unit — `tests/unit/test_error_sanitization.py` (new): the exact existing sync behavior is pinned — `https://user:pass@host/x` → `https://*****@host/x`; multiple userinfo occurrences all masked; an ssh-style `git:token@host` userinfo masked; credential-free text (a git failure line with a bare `https://github.com/owner/repo.git` URL, a plain error sentence, an email-shaped `a@b.c` — no userinfo run) **byte-identical**; idempotent (sanitize(sanitize(x)) == sanitize(x)).
- Unit — `tests/unit/test_llm_client.py` (extend): with `llm_base_url="https://svc:topsecret@llm.local/v1"`, a forced transport failure on the embed path → the `EmbeddingError` message contains `https://*****@llm.local/v1` and NOT `topsecret` (one representative path — the embed failure; the other four sites share the same construction, pinned by the same test style if cheap); the credential-free base-URL error pins stay byte-identical (existing tests green).
- Integration — `tests/integration/test_doc_drafts_api.py` (extend): monkeypatch `app.core.docs_push.push_document` (or the doc_drafts module's reference) to raise `DocsPushError("git push failed (exit 128): fatal: Authentication failed for 'https://bot:ghp_LEAK@github.com/owner/docs.git/'")` → `POST /api/doc-drafts/{token}/push` → **502** whose `detail` contains `*****@github.com` and NOT `ghp_LEAK` (and the repo/exit/`fatal:` context still readable); the existing 502 pins (plain stderr detail) stay byte-identical green.
- Regression: `tests/integration/test_git_sources_upload.py` + the sync-status tests (the alias refactor's proof — the `_sanitize_error` import in `git_sources.py` still works; the sync `failed`-state error is masked exactly as before).
- E2E (isolation gate per AGENTS.md rule 9): `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green — no UI change (the doc-edit screen renders whatever detail string the API returns; the shape is unchanged).
- Coverage: **>90%** on `app/` — the new module is two lines + fully unit-pinned; `app/rag/llm.py`'s changed lines hit by the new unit tests + the existing error-path tests.
## Completion Criteria
- [ ] A docs-push failure with userinfo in git's stderr → 502 detail shows `*****@` and never the token (integration pin); the row is untouched (status/branch/sha as found — the existing pin).
- [ ] A LLM transport failure with a userinfo-bearing `llm_base_url` → the error message is masked (unit pin); credential-free error strings byte-identical (existing pins green).
- [ ] `app/api/sync.py` no longer defines its own `_CREDS_RE`/`_sanitize_error` (the alias is the import); `git grep "_CREDS_RE" app/` shows only `app/core/errors.py`; the sync + git-sources upload suites green untouched.
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green in isolation; `uv run ruff check . && uv run pyright` clean.
- [ ] `git diff --stat` limited to `app/core/errors.py`, `app/api/sync.py`, `app/api/doc_drafts.py`, `app/rag/llm.py`, the three test files, phase files.
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(security): mask credentials in docs-push and LLM error surfaces`); phase dir moved to `.agents/phases/complete/`.
## Locked decisions
- **Sanitize at the API boundary** (A2) — `DocsPushError` keeps carrying the full stderr (logs/inspection value); the RESPONSE is where the secret would leak, so that is where it is masked.
- **Byte-identical for credential-free text** (A1/A3) — the narrow-regex contract; the existing sync/LLM error pins are the tripwire, and they must pass unchanged.
- **No logging changes** — this phase changes what responses carry; the server-log side of SEC-08/SEC-13 is improved as a consequence (the 502 detail is the logged-adjacent surface) but no log-format change is in scope.
@@ -0,0 +1,29 @@
# Task 01 — The shared sanitizer module + sync alias refactor + unit suite
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08)
## Objective
`app/core/errors.py` owns `sanitize_error` (verbatim behavior of the sync masker), `app/api/sync.py` becomes a thin alias caller (zero behavior change), and `tests/unit/test_error_sanitization.py` pins the masking + the byte-identical-for-plain-text contract.
## Work
1. `app/core/errors.py` (new) — per the phase overview's design block: `_CREDS_RE` (the exact pattern from `app/api/sync.py`) + `sanitize_error(message: str) -> str` (`_CREDS_RE.sub("*****@", message)`) + the module docstring content (audit basis SEC-08/SEC-13, the narrow-userinfo contract, the `*****@` replacement shape — the sync copy).
2. `app/api/sync.py` — remove the local `_CREDS_RE` definition and the `_sanitize_error` function body; add `from app.core.errors import sanitize_error as _sanitize_error`; the `_run_sync` failure path (`_status.error = _sanitize_error(str(e))`) and everything else in the file stays byte-identical (the private name keeps working — `app/api/git_sources.py`'s `from app.api.sync import _sanitize_error` continues to import the same function through the alias).
- Update the `app/api/sync.py` module docstring's mention of the masker (one line: it now lives in `app/core/errors.py`, imported under the private name).
3. `tests/unit/test_error_sanitization.py` (new):
- `https://user:pass@host/repo.git` → `https://*****@host/repo.git`;
- two userinfo occurrences in one string → both masked;
- `git push failed (exit 128): fatal: Authentication failed for 'https://bot:tok@github.com/o/r.git/'` → the token masked, `fatal:`/`exit 128`/host intact;
- byte-identical cases (the contract): a plain git error with a bare `https://github.com/owner/repo.git` (no userinfo), a sentence with a colon + space, an email `owner@example.com` (no userinfo run before the `@`… assert exactly what the regex does — if the pattern masks it, pin THAT and note it; the point is determinism, not guessing), the empty string;
- idempotence: `sanitize_error(sanitize_error(x)) == sanitize_error(x)` for the masked cases.
4. Run the sync-related suites to prove the alias refactor: `uv run pytest tests/integration/test_git_sources_upload.py -q` + any existing sync-status integration test file (`tests/integration/` — locate the sync suite, e.g. via `rg -l "sync" tests/integration/`) → all green untouched.
## Testing & Quality
- `uv run pytest tests/unit/test_error_sanitization.py -v` green.
- The sync/upload regression suites green (the alias is behavior-identical).
- Coverage: **>90%** on `app/core/errors.py` (trivial — both branches of the regex hit).
## Completion Criteria
- [ ] `git grep -n "_CREDS_RE" app/` shows exactly one definition — `app/core/errors.py` — and `app/api/sync.py` imports under the private alias.
- [ ] `uv run pytest tests/unit/test_error_sanitization.py -v` green.
- [ ] The sync/upload integration suites green with NO test edits.
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,29 @@
# Task 02 — Apply the sanitizer to the docs-push 502 and the LLM error sites
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08 + SEC-13)
## Objective
The two leaking surfaces are closed: `push_doc_draft`'s 502 detail is sanitized (SEC-08), and the five `llm.py` error f-strings sanitize the base URL (SEC-13) — with integration/unit pins for both and the credential-free error strings byte-identical.
## Work
1. `app/api/doc_drafts.py` — import `from app.core.errors import sanitize_error`; in `push_doc_draft`'s `except DocsPushError as exc:` clause change the raise to `raise HTTPException(status_code=502, detail=sanitize_error(str(exc))) from None`; update the route docstring's outcome-4 line (502 detail = the SANITIZED git stderr — userinfo masked, the `GitSyncError → detail` mapping otherwise kept) and the module docstring's one-line error-contract sentence if it names the raw stderr.
2. `tests/integration/test_doc_drafts_api.py` — extend (the existing 502 pins stay green):
- a draft with `status="draft"` + `docs_configured` (the file's existing configured-settings pattern — monkeypatch `get_settings` or use the fixture's env);
- monkeypatch the module-level `push_document` reference in `app.api.doc_drafts` to `raise DocsPushError("git push origin bor-docs failed (exit 128): fatal: Authentication failed for 'https://bot:ghp_LEAKTOKEN@github.com/owner/docs.git/'")`;
- `POST /api/doc-drafts/{token}/push` → **502**; the `detail` contains `*****@github.com` and `exit 128` and `fatal: Authentication failed` but NOT `ghp_LEAKTOKEN`;
- the row is untouched (status/branch/commit_sha as found — the existing pin style).
3. `app/rag/llm.py` — import `sanitize_error`; wrap the base URL in the five error f-strings (the lines building `EmbeddingError`/`LLMError` with `…to/from {self.settings.llm_base_url}…`): each becomes `{sanitize_error(self.settings.llm_base_url)}`; the surrounding copy is byte-identical.
4. `tests/unit/test_llm_client.py` — extend:
- a client built with `llm_base_url="https://svc:topsecret@llm.local/v1"` (the file's existing fake-settings pattern) + a forced transport failure on the EMBED path → the raised `EmbeddingError`'s message contains `https://*****@llm.local/v1` and NOT `topsecret`;
- if the file's structure makes it cheap, the same for one of the chat paths (the `chat_stream`/`chat` failure f-string) — otherwise the embed path alone plus a comment (the five sites share the construction; the byte-identical regression is the existing pins);
- the existing credential-free error-string pins (bare `https://aipi.example/v1`-style base URLs) stay byte-identical green.
## Testing & Quality
- `uv run pytest tests/integration/test_doc_drafts_api.py -v` green (new + existing).
- `uv run pytest tests/unit/test_llm_client.py -v` green (new + existing — the byte-identical proof).
- Coverage: **>90%** on `app/` — the changed lines in `doc_drafts.py`/`llm.py` are hit by the new pins; the other four LLM sites share the identical construction (the existing error-path tests still exercise them).
## Completion Criteria
- [ ] The 502 pin passes: `*****@github.com` present, `ghp_LEAKTOKEN` absent, context readable; the row untouched.
- [ ] The LLM unit pin passes: masked URL in the error message, `topsecret` absent; the pre-existing error-string tests green unchanged.
- [ ] `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,26 @@
# Task 03 — Full gate + atomic commit
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08 + SEC-13)
## Objective
Run the complete phase gate, land the phase as one atomic commit, and move the phase directory to `complete/`.
## Work
1. **Full regression gate** (AGENTS.md rule 9):
- `uv run pytest` — unit + integration green (the sync/upload + doc-drafts + LLM suites are the regression anchors for the alias refactor and the byte-identical contract).
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` — green **in isolation** (this phase's E2E contract: no UI change — the doc-edit screen renders the detail string it always rendered; the shape is unchanged, only the userinfo run is masked).
- `uv run ruff check . && uv run pyright` — clean.
2. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/errors.py`, `app/api/sync.py`, `app/api/doc_drafts.py`, `app/rag/llm.py`, `tests/unit/test_error_sanitization.py`, `tests/unit/test_llm_client.py`, `tests/integration/test_doc_drafts_api.py`, and the phase files:
`fix(security): mask credentials in docs-push and LLM error surfaces`
— body: security audit SEC-08 + SEC-13 (2026-09-07) — a docs-push failure returned git's stderr verbatim as the 502 detail (a `BOR_DOCS_REPO` URL with embedded credentials would reach the browser + logs), and the LLM error f-strings echoed `llm_base_url` raw; the sync-phase userinfo masker (`user:pass@` → `*****@`) now lives in `app/core/errors.py`, the docs-push 502 and the five LLM error sites run through it, and the sync import keeps working via a private alias. Credential-free error strings are byte-identical (existing pins green).
3. Move the phase directory: `mv .agents/phases/todo/84_docs_push_error_sanitization .agents/phases/complete/` and include the move in the same commit.
## Testing & Quality
- This task IS the phase-level gate — the commands above are the completion evidence.
- Coverage: >90% held.
## Completion Criteria
- [ ] `uv run pytest` green (all regression anchors included); coverage >90%; `tests/e2e/test_smoke.py` green in isolation; ruff + pyright clean.
- [ ] Exactly one new commit; `git show --stat HEAD` lists the files above + the phase files (todo → complete move) — in particular NO `scripts/`, NO `app/core/docs_push.py`, NO `frontend/`, NO `pyproject.toml`/`uv.lock`.
- [ ] `.agents/phases/complete/84_docs_push_error_sanitization/` exists; `todo/` no longer contains it.
@@ -0,0 +1,14 @@
All gates verified. The phase is complete — everything was already implemented by prior task runs; this pass verified each completion criterion end-to-end and found no defects.
**Phase 81 final verification report** (tasks 01–03 all present in `complete/`; no code changes needed this pass)
- Verified: `app/core/rate_limit.py` (stdlib sliding window, fail-open) + both login routes' 429 pre-check/record/reset wiring + all unit/integration pins present (11th-429, blocked-success-still-429, clean-counter reset, shared counter, autouse `clean_rate_limit` fixture documented).
- Live check (task 03): dev server + 11 rapid wrong logins → `401 ×10, 429`; 429 body carries `retry-after: 900` + generic detail; server restarted (per-process counter cleared by design).
- `uv run pytest tests/unit/test_rate_limit.py -v --no-cov` → 10 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed, **TOTAL 99%** (>90%; rate_limit.py 100%, auth.py 100%)
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` (isolation) → 3 passed
- `uv run ruff check . && uv run pyright` → All checks passed / 0 errors, 0 warnings
- Completion criteria: all met, except commit + phase-dir move — per harness rules I left all changes uncommitted in the working tree (harness commits atomically and moves the phase).
- Diff scope: exactly `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py` + phase files; `pyproject.toml` / `uv.lock` / `frontend/` untouched.
- Deviations: none in code; commit/move deferred to harness as instructed.
- Next pending phase: `82_security_headers`.
@@ -0,0 +1,82 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 13%]
........................................................................ [ 17%]
........................................................................ [ 21%]
........................................................................ [ 26%]
........................................................................ [ 30%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 43%]
........................................................................ [ 47%]
........................................................................ [ 52%]
........................................................................ [ 56%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
........................................................................ [ 74%]
........................................................................ [ 78%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 95%]
.................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 101 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 62 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 231 0 100%
-----------------------------------------------
TOTAL 3211 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
@@ -0,0 +1,18 @@
Task 01 complete — all gates green.
**Implemented**
- `app/core/rate_limit.py` (new): stdlib-only sliding-window limiter — `MAX_FAILURES = 10` / `WINDOW_SECONDS = 900` module constants, per-IP `deque[float]` of `time.monotonic()` timestamps under a `threading.Lock`; `record_failure` (append + prune, never raises, drops the count silently on corrupt state), `remaining_wait` (0 when < 10 in-window; else `max(1, ceil(900 − (now − oldest)))`, never raises), `reset` (pop, no-op on unknown). Docstring carries the SEC-03 audit basis, shared-counter intent, fail-open rationale, and the `request.client.host` / no-proxy assumption.
- `tests/unit/test_rate_limit.py` (new): 10 tests pinning, in task order — 9 failures allowed → 10th blocks (`0 < wait ≤ 900`, int) → second IP independent → expiry via pre-seeded `WINDOW_SECONDS + 1` state AND a fake monotonic clock (whole-window wait from the oldest: 891 → 300 → clamped 1 → 0 the moment the oldest slides out) → `reset` unblocks → unknown IP 0 → fail-open on non-deque state and on a raising clock → `reset` no-op on unknown IP.
**Results**
- `uv run pytest tests/unit/test_rate_limit.py -v` → 10 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1648 passed; `app/core/rate_limit.py` **100%**, TOTAL **99%** (>90%)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- `git status`: only the two new files are code changes (pre-existing untracked phase/report files untouched); no tracked file modified
**Notable decisions**
- Corrupt state entry → `record_failure` is a literal no-op (entry left in place, treated as not-blocked) per the phase's "record_failure no-ops" wording; over-denial is the only forbidden outcome.
- Removed one unreachable defensive branch after the first coverage run to reach 100% on the module.
- `.agents/PLAN.md` is absent from the repo (AGENTS.md references it); the phase overview's design block was followed as the binding spec — no deviation.
**Next pending task:** `02_auth_routes_wiring.md` (429 pre-check + bookkeeping on both login routes + integration tests).
@@ -0,0 +1,82 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 13%]
........................................................................ [ 17%]
........................................................................ [ 21%]
........................................................................ [ 26%]
........................................................................ [ 30%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 52%]
........................................................................ [ 56%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
........................................................................ [ 74%]
........................................................................ [ 78%]
........................................................................ [ 83%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 40 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 101 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 62 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 231 0 100%
-----------------------------------------------
TOTAL 3199 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
@@ -0,0 +1,16 @@
**Task 02 complete — report**
**Implemented**
- `app/api/auth.py`: `TOO_MANY_DETAIL` constant + `rate_limit` import; on both `login` and `token_auth` — 429 pre-check (`Retry-After`, generic detail) at handler top, `record_failure` before the 401, `reset` before the 204 (shared per-IP counter); module + handler docstrings extended with the 429 case.
- `tests/integration/test_auth_api.py`: autouse `clean_rate_limit` fixture (`rate_limit.reset("testclient")` around every test — documented: TestClient's host is a fixed shared IP, counter is process state) + 4 new pins: 11th failed login → 429 + detail + `Retry-After` (≤ window); correct password while blocked → still 429; 9 failures + success → counter reset (next 9 all 401); 5 login + 5 token failures → 11th (valid token) → 429.
**Results**
- `uv run pytest tests/integration/test_auth_api.py tests/unit/test_rate_limit.py` → 27 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed; **99%** total (`app/api/auth.py` 100%, `app/core/rate_limit.py` 100%)
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (isolation)
- `uv run ruff check . && uv run pyright` → clean (0 errors)
- `pyproject.toml`/`uv.lock` untouched; diff limited to `app/api/auth.py` + the two test files (plus task-01's two new files, untracked for the harness commit)
**Deviation**: used `request.client.host if request.client is not None else "unknown"` instead of the task's literal `or "unknown"` — starlette types `client` as optional; pyright (non-negotiable gate) requires the explicit check. Same fail-open semantics.
**Next pending task**: `.agents/phases/todo/81_login_rate_limit/03_verify_and_commit.md`
@@ -0,0 +1,82 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 13%]
........................................................................ [ 17%]
........................................................................ [ 21%]
........................................................................ [ 26%]
........................................................................ [ 30%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 43%]
........................................................................ [ 47%]
........................................................................ [ 52%]
........................................................................ [ 56%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
........................................................................ [ 74%]
........................................................................ [ 78%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 95%]
.................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 101 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 62 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 231 0 100%
-----------------------------------------------
TOTAL 3211 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
@@ -0,0 +1,15 @@
**Task 03 (phase-level gate) complete — report**
**Gate results (all run in this session):**
- `uv run pytest` → 1652 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 99% total; `app/api/auth.py` 100%, `app/core/rate_limit.py` 100%
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (in isolation, server on :8123)
- `uv run ruff check . && uv run pyright` → all checks passed; 0 errors
- Live check (fresh `uv run uvicorn app.main:app` on :8000): 11 rapid wrong logins → `401 ×10, 429`; the 429 carries `retry-after: 900` + generic detail; correct password while blocked → 429; kill + restart → counter cleared (wrong login → 401). Server stopped, port free.
- `git diff` scope: only `app/api/auth.py`, `tests/integration/test_auth_api.py` (modified) + `app/core/rate_limit.py`, `tests/unit/test_rate_limit.py` (new); `pyproject.toml`/`uv.lock`/`frontend/` untouched (0 diff lines).
**Notable decisions:**
- Per harness rules (which override the task file), I did **not** run `git add`/`git commit` nor move the phase dir to `complete/` — the harness makes the atomic commit and the move. All changes left in the working tree.
- No defects found in tasks 01/02; one operational slip of my own (first restart check killed the wrong pid, so a stale blocked server answered) — redone cleanly with a full process kill; final results above are from a verified-fresh server.
**Next pending task:** `.agents/phases/todo/82_security_headers/01_headers_middleware.md` (phase 81 is fully gated — harness commit + move remains).
@@ -0,0 +1,82 @@
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 13%]
........................................................................ [ 17%]
........................................................................ [ 21%]
........................................................................ [ 26%]
........................................................................ [ 30%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 43%]
........................................................................ [ 47%]
........................................................................ [ 52%]
........................................................................ [ 56%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
........................................................................ [ 74%]
........................................................................ [ 78%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 95%]
.................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 178 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 29 0 100%
app/api/sync.py 101 0 100%
app/api/tokens.py 28 0 100%
app/config.py 141 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/tokens.py 33 0 100%
app/db.py 21 0 100%
app/main.py 62 0 100%
app/models.py 94 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 231 0 100%
-----------------------------------------------
TOTAL 3211 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
+41 -2
View File
@@ -2,11 +2,16 @@
* ``POST /api/login`` — 204 + signed session cookie on success; 401
``invalid password`` on any mismatch (constant-time, one generic
message, no session set).
message, no session set); while the per-IP failure window is exhausted
→ 429 + ``Retry-After``, one generic detail — audit SEC-03.
* ``POST /api/token-auth`` — 204 + signed session cookie for a valid,
unrevoked API token (PUBLIC — it is the token holders' login route);
every failure shape (malformed / unknown / revoked / empty) is ONE
generic 401 ``invalid token`` (no enumeration, the phase-16 pattern).
generic 401 ``invalid token`` (no enumeration, the phase-16 pattern);
while the per-IP failure window is exhausted → 429 + ``Retry-After``,
one generic detail — audit SEC-03 (the 429 counter is SHARED with
``/api/login``: any auth failure from an IP counts, a success on
either route resets it).
* ``POST /api/logout`` — 204; clears the session and expires the
cookie (idempotent for anonymous callers — one logout wipes BOTH
roles, the session is one dict).
@@ -26,6 +31,7 @@ from fastapi import APIRouter, Depends, HTTPException, Request, Response
from sqlalchemy.orm import Session
from app.config import get_settings
from app.core import rate_limit
from app.core import tokens as token_service
from app.core.auth import (
ADMIN_SESSION_KEY,
@@ -38,6 +44,10 @@ from app.core.auth import (
from app.db import get_db
from app.schemas import LoginRequest, TokenAuthRequest, WhoamiResponse
#: One generic 429 detail shared by BOTH throttled login routes (audit
#: SEC-03): no enumeration between a login failure and a token failure.
TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"
router = APIRouter(tags=["auth"])
@@ -49,11 +59,27 @@ def login(payload: LoginRequest, request: Request) -> Response:
12 h default lifetime). Failure: one generic 401 — the admin count is
one, so there is nothing else to leak, and a wrong password must not
set any session state.
Rate limiting (audit SEC-03): while the per-IP failure window is
exhausted (10 misses in 15 min, shared with ``/api/token-auth``),
the pre-check fires BEFORE the password is compared — 429 +
``Retry-After``, one generic detail; even a correct password stays
429 until the window slides. A success on a CLEAN counter resets it;
a failure is recorded on the 401 path.
"""
ip = request.client.host if request.client is not None else "unknown"
if (wait := rate_limit.remaining_wait(ip)) > 0:
raise HTTPException(
status_code=429,
detail=TOO_MANY_DETAIL,
headers={"Retry-After": str(wait)},
)
settings = get_settings()
if check_password(payload.password, settings.admin_password):
sign_in(request.session)
rate_limit.reset(ip)
return Response(status_code=204)
rate_limit.record_failure(ip)
raise HTTPException(status_code=401, detail="invalid password")
@@ -77,15 +103,28 @@ def token_auth(
generic 401 ``invalid token``: the lookup is by hash (the service
returns ``None`` for anything that is not an active row), so there
is no enumeration surface (the phase-16 pattern).
Rate limiting (audit SEC-03): identical bookkeeping to ``/api/login``
on the SAME per-IP counter — pre-check → 429 + ``Retry-After`` while
the window is exhausted; failures recorded, a success resets.
"""
ip = request.client.host if request.client is not None else "unknown"
if (wait := rate_limit.remaining_wait(ip)) > 0:
raise HTTPException(
status_code=429,
detail=TOO_MANY_DETAIL,
headers={"Retry-After": str(wait)},
)
token = payload.token.strip()
row = token_service.find_active_by_token(db, token) if token else None
if row is None:
rate_limit.record_failure(ip)
raise HTTPException(status_code=401, detail="invalid token")
token_service.mark_used(row)
db.commit()
request.session[USER_SESSION_KEY] = True
request.session[USER_TOKEN_ID_KEY] = str(row.id)
rate_limit.reset(ip)
return Response(status_code=204)
+139
View File
@@ -0,0 +1,139 @@
"""Sliding-window rate limiting for failed sign-ins (phase 81, audit SEC-03).
Audit basis
-----------
SEC-03 (security audit 2026-09-07, severity Medium): neither ``POST
/api/login`` nor ``POST /api/token-auth`` throttled failed attempts —
the password is compared in constant time (``secrets.compare_digest``),
so an attacker who can reach ``:8000`` could guess at line rate (audit
PoC: 10 000 sequential logins, all answered at line rate). This module
is the fix: a per-client-IP sliding window over FAILED sign-in
attempts only.
Threshold (owner decision A2 — module constants, NOT ``BOR_`` settings:
a security control with a fixed, tested default and no env surface to
mistype): ``MAX_FAILURES = 10`` failed attempts per client IP within
``WINDOW_SECONDS = 900`` (15 min). The window slides — only failures
within the last ``WINDOW_SECONDS`` count. The IP is blocked while its
in-window failure count is >= ``MAX_FAILURES``: the 10th failure is
recorded, and the 11th attempt is the first 429 (the routes' pre-check
runs BEFORE the attempt).
Shared counter
--------------
The counter is SHARED by both login routes (``/api/login`` and
``/api/token-auth``): any auth failure from an IP counts, so a
token-spraying attack cannot dodge the window by alternating routes,
and a SUCCESS on either route clears that IP's counter (a legitimate
owner who fat-fingers twice is not poisoned). The routes do the
bookkeeping (pre-check / record / reset); this module only knows about
failure timestamps.
Client IP
---------
Callers pass ``request.client.host`` — the DIRECT peer address (owner
decision A4). Assumption locked in the phase: the app is served
directly in the homelab, with NO reverse proxy in front. If a proxy is
ever put in front, this module would need ``X-Forwarded-For`` handling
(out of scope now).
Fail-open
---------
The limiter is a friction bump, not the security boundary (owner
decision A1 — in-memory, per-process, stdlib-only; the counter is lost
on restart, an accepted reset). A limiter bug must NEVER lock the
owner out: any internal error (a corrupted state entry, a clock
anomaly) makes ``remaining_wait`` return 0 (allow) and
``record_failure`` drop the count silently. Over-denial is the one
outcome this module is forbidden to produce.
"""
from __future__ import annotations
import math
import threading
import time
from collections import deque
#: An IP is blocked while it has at least this many failed sign-ins in
#: the window (owner decision A2: 10 attempts per 15 minutes).
MAX_FAILURES = 10
#: Sliding-window length in seconds (15 minutes).
WINDOW_SECONDS = 900
#: Failed-attempt timestamps per client IP, oldest first. Module-global
#: on purpose (A1: per-process state, no new service or table); the
#: lock keeps the dict honest if the event loop is ever shared.
_failures: dict[str, deque[float]] = {}
_lock = threading.Lock()
def _prune(dq: deque[float], now: float) -> None:
"""Drop failures that have slid out of the window (oldest first).
A timestamp exactly ``WINDOW_SECONDS`` old is expired: the window
holds the last ``WINDOW_SECONDS`` strictly.
"""
cutoff = now - WINDOW_SECONDS
while dq and dq[0] <= cutoff:
dq.popleft()
def record_failure(client_ip: str) -> None:
"""Count one failed sign-in attempt from ``client_ip``.
Appends ``time.monotonic()`` to the IP's deque and prunes expired
entries. NEVER raises (fail-open): any internal error — including a
corrupted, non-deque state entry — drops the count silently instead
of risk blocking the owner. A corrupt entry is left in place (and
treated as "not blocked" by ``remaining_wait``) until ``reset`` or a
process restart clears it.
"""
try:
with _lock:
now = time.monotonic()
dq = _failures.get(client_ip)
if dq is None:
dq = deque()
_failures[client_ip] = dq
elif not isinstance(dq, deque):
return # corrupt state → drop this count (fail-open)
dq.append(now)
_prune(dq, now)
except Exception:
return # a limiter bug must never break the login route
def remaining_wait(client_ip: str) -> int:
"""Seconds until ``client_ip`` may attempt again (0 = allowed now).
Prunes expired failures; if fewer than ``MAX_FAILURES`` remain in
the window the IP is allowed (0). Otherwise returns the WHOLE-window
wait — ``ceil(WINDOW_SECONDS - (now - oldest))``, clamped to >= 1 —
i.e. until the oldest counted failure expires (the value the routes
put in ``Retry-After``; the window, not any single failure's age,
governs). NEVER raises (fail-open → 0).
"""
try:
with _lock:
now = time.monotonic()
dq = _failures.get(client_ip)
if not isinstance(dq, deque) or not dq:
return 0 # unknown IP or corrupt state → allow (fail-open)
_prune(dq, now)
if len(dq) < MAX_FAILURES:
return 0 # includes the all-expired (empty) case
wait = WINDOW_SECONDS - (now - dq[0])
return max(1, math.ceil(wait))
except Exception:
return 0 # a limiter bug must never deny the owner
def reset(client_ip: str) -> None:
"""Clear ``client_ip``'s failure count (called on a SUCCESS).
A successful sign-in on either route unblocks the IP immediately —
a legitimate owner who fat-fingers twice is not poisoned by their
own earlier misses. Unknown IPs are a silent no-op.
"""
with _lock:
_failures.pop(client_ip, None)
+96
View File
@@ -14,6 +14,15 @@ the ONLY anonymous content is the shared chats (plus the login/infra
endpoints the gate itself needs) — anonymous chat / suggestions /
document content now 401 ``authentication required``.
Rate limiting (phase 81, audit SEC-03): the sign-in failure counter is
PROCESS state (in-memory per IP), and TestClient's
``request.client.host`` is the fixed value ``"testclient"`` — so every
client in this module (including ``_admin_client``'s separate
TestClient) shares ONE limiter entry. The ``clean_rate_limit`` autouse
fixture resets that entry around every test so the deliberate 429 pins
can't poison the other tests (and the table TRUNCATEs can't help: the
counter is not row state).
Requires: podman compose up -d db
"""
from __future__ import annotations
@@ -28,6 +37,9 @@ from sqlalchemy import text
from test_chat_api import FakeRagLLM, _stream_chat
from app.api import chat as chat_api
from app.api.auth import TOO_MANY_DETAIL
from app.core import rate_limit
from app.core.rate_limit import WINDOW_SECONDS
from app.main import app as fastapi_app
from app.models import Document
from app.rag.importer import import_sources
@@ -37,6 +49,24 @@ FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
QUESTION = "How is my Kubernetes cluster set up?"
@pytest.fixture(autouse=True)
def clean_rate_limit() -> Iterator[None]:
"""Reset the in-memory sign-in-failure counter around every test.
TestClient's ``request.client.host`` is the fixed value
``"testclient"``, so every client in this module — including the
``_admin_client`` helper's separate TestClient — shares ONE limiter
entry, and the counter is process state the table TRUNCATEs cannot
clear. Without this reset, the deliberate 429 tests below (which
block the shared IP on purpose) would make every later test in the
process hit the throttled 429 path instead of the 401/204 contract
it pins.
"""
rate_limit.reset("testclient")
yield
rate_limit.reset("testclient")
@pytest.fixture(autouse=True)
def clean_tables(db) -> Iterator[None]:
"""Docs + steering + query log + tokens are global state: reset
@@ -425,3 +455,69 @@ def test_admin_chat_still_streams(client: TestClient, db, seeded_kb: FakeRagLLM)
assert frames[-1]["type"] == "done"
assert frames[-1]["deflected"] is False
assert frames[-1]["sources"][0]["path"] == "homelab/kubernetes.md"
# ---------- phase 81: rate-limited failed sign-ins (audit SEC-03) ----------
def test_eleventh_failed_login_429_with_retry_after(client: TestClient) -> None:
"""10 wrong passwords → 401 (the phase-16 contract, unchanged
detail); the 11th attempt is the FIRST 429 — one generic detail + a
``Retry-After`` header carrying whole-window seconds."""
for _ in range(10):
r = client.post("/api/login", json={"password": "nope"})
assert r.status_code == 401
assert r.json() == {"detail": "invalid password"}
r = client.post("/api/login", json={"password": "nope"})
assert r.status_code == 429
assert r.json()["detail"] == TOO_MANY_DETAIL
retry_after = int(r.headers["Retry-After"])
assert retry_after > 0
assert retry_after <= WINDOW_SECONDS # the whole-window wait, never more
def test_correct_password_while_blocked_still_429(client: TestClient) -> None:
"""An exhausted window stays exhausted until it slides: the pre-check
fires BEFORE the password is compared, so even a CORRECT password
gets the 429 while blocked (success only resets a CLEAN counter —
this is the intended semantics, pinned)."""
for _ in range(10):
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 429
assert r.json()["detail"] == TOO_MANY_DETAIL
assert "bor_session" not in client.cookies # not signed in while throttled
def test_success_on_clean_counter_resets_it(client: TestClient) -> None:
"""9 failures + 1 success (204) → the counter is back to 0: the next
9 failures are all plain 401s (a fresh 10 would be needed to block
again). A fat-fingered owner is not poisoned by earlier misses."""
for _ in range(9):
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
assert client.post("/api/login", json={"password": ADMIN_PASSWORD}).status_code == 204
for _ in range(9):
r = client.post("/api/login", json={"password": "nope"})
assert r.status_code == 401
assert r.json() == {"detail": "invalid password"}
def test_login_and_token_failures_share_the_counter(client: TestClient) -> None:
"""The counter is SHARED by both routes (a token-spraying attack must
not dodge the window by alternating routes): 5 login failures + 5
token failures exhaust it, so the 11th attempt of EITHER shape — even
with a VALID token — is a 429."""
# Admin + token FIRST: the successful admin login resets the shared
# counter, so it must not happen after failures have accumulated.
admin = _admin_client()
_token_id, token = _create_token(admin)
for _ in range(5):
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
for _ in range(5):
r = client.post("/api/token-auth", json={"token": "bor_" + "0" * 32})
assert r.status_code == 401
assert r.json() == {"detail": "invalid token"}
r = client.post("/api/token-auth", json={"token": token})
assert r.status_code == 429
assert r.json()["detail"] == TOO_MANY_DETAIL
assert int(r.headers["Retry-After"]) > 0
+149
View File
@@ -0,0 +1,149 @@
"""Unit: the sliding-window failed sign-in limiter (phase 81, task 01).
Covers ``app.core.rate_limit`` — the per-client-IP sliding window that
both login routes share (the audit SEC-03 fix). Pins, in the task's
order:
* 9 failures → still allowed (``remaining_wait == 0``);
* the 10th failure → blocked (``remaining_wait > 0``, ``<= WINDOW_SECONDS``);
* independent IPs never share counters;
* window slide/expiry — a failure older than ``WINDOW_SECONDS`` stops
counting (both pre-seeded module state and a fake monotonic clock);
* ``remaining_wait`` is the WHOLE-window wait until the OLDEST in-window
failure expires (the ``Retry-After`` contract), not a per-failure age;
* ``reset`` (a success on either route) unblocks immediately;
* an unknown IP never blocks;
* FAIL-OPEN — a corrupted, non-deque state entry and a broken clock
both degrade to "allow": ``remaining_wait`` returns 0 and
``record_failure`` drops the count without raising. A limiter bug
must never lock the owner out.
"""
from __future__ import annotations
import time
from collections import deque
from collections.abc import Iterator
import pytest
from app.core import rate_limit as rl
@pytest.fixture(autouse=True)
def _clean_state() -> Iterator[None]:
"""The limiter is module-global: no counter may leak between tests."""
rl._failures.clear()
yield
rl._failures.clear()
def test_nine_failures_still_allowed() -> None:
"""Below the threshold: 9 in-window failures → no wait (the 10th
failure is the one that crosses the bar)."""
for _ in range(rl.MAX_FAILURES - 1):
rl.record_failure("ip")
assert rl.remaining_wait("ip") == 0
def test_tenth_failure_blocks_until_the_window_clears() -> None:
"""The 10th recorded failure crosses the threshold: the routes'
pre-check (the 11th attempt) sees a positive wait — bounded by the
whole window and integral for the ``Retry-After`` header."""
for _ in range(rl.MAX_FAILURES):
rl.record_failure("ip")
wait = rl.remaining_wait("ip")
assert isinstance(wait, int)
assert wait > 0
assert wait <= rl.WINDOW_SECONDS
def test_ips_are_independent() -> None:
"""One IP's failures never count toward another IP's window."""
for _ in range(rl.MAX_FAILURES):
rl.record_failure("ip")
assert rl.remaining_wait("ip") > 0
assert rl.remaining_wait("ip2") == 0
def test_failures_older_than_the_window_stop_counting_preseeded() -> None:
"""Reach into the module state: timestamps ``WINDOW_SECONDS + 1`` in
the past have slid out — 10 such 'failures' leave the IP unblocked."""
now = time.monotonic()
rl._failures["ip"] = deque([now - rl.WINDOW_SECONDS - 1.0] * rl.MAX_FAILURES)
assert rl.remaining_wait("ip") == 0
def test_window_slides_and_wait_is_measured_from_the_oldest(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Fake monotonic clock, 10 failures one second apart (t = 1000..1009):
* the wait is the WHOLE-window seconds until the OLDEST in-window
failure expires — ``ceil(900 - 9) == 891`` at t = 1009, not a
per-failure age;
* the wait counts down with the oldest (300 at t = 1600, clamped to
1 at t = 1899);
* the block lifts the moment the oldest failure slides out (t =
1900): 9 failures are then still in the window, yet the count is
back below the threshold.
"""
clock = {"t": 1000.0}
monkeypatch.setattr(time, "monotonic", lambda: clock["t"])
for i in range(rl.MAX_FAILURES):
clock["t"] = 1000.0 + i # failures at 1000, 1001, ..., 1009
rl.record_failure("ip")
assert rl.remaining_wait("ip") == 891 # ceil(900 - 9)
clock["t"] = 1600.0
assert rl.remaining_wait("ip") == 300 # oldest is 600 s old
clock["t"] = 1899.0
assert rl.remaining_wait("ip") == 1 # ceil(900 - 899), clamped >= 1
clock["t"] = 1900.0
assert rl.remaining_wait("ip") == 0 # oldest just expired → 9 left
def test_reset_unblocks_immediately() -> None:
"""A success on either route clears the IP's counter: the owner who
fat-fingered is not poisoned by their own earlier misses."""
for _ in range(rl.MAX_FAILURES):
rl.record_failure("ip")
assert rl.remaining_wait("ip") > 0
rl.reset("ip")
assert rl.remaining_wait("ip") == 0
def test_unknown_ip_never_blocks() -> None:
"""No failures recorded → no wait (the routes' pre-check fast path)."""
assert rl.remaining_wait("never-seen") == 0
def test_fail_open_on_corrupt_state_entry() -> None:
"""A non-deque entry in the module state (corruption):
``remaining_wait`` must return 0 (never block) and ``record_failure``
must drop the count WITHOUT raising (fail-open — a limiter bug never
locks the owner out). The IP stays unblocked afterwards."""
rl._failures["bad"] = object() # type: ignore[assignment]
assert rl.remaining_wait("bad") == 0
rl.record_failure("bad") # must not raise
assert rl.remaining_wait("bad") == 0
def test_fail_open_on_broken_clock(monkeypatch: pytest.MonkeyPatch) -> None:
"""A clock anomaly (monotonic raising) is equally fail-open: 0 wait
and a silent drop — the login route itself must never 500 because
of the limiter."""
def broken_clock() -> float:
raise ValueError("clock skew")
monkeypatch.setattr(time, "monotonic", broken_clock)
assert rl.remaining_wait("any") == 0
rl.record_failure("any") # must not raise
assert rl.remaining_wait("any") == 0
def test_reset_unknown_ip_is_a_noop() -> None:
"""No entry to drop: ``reset`` must not raise (the routes call it on
every success unconditionally)."""
rl.reset("ghost")
assert rl.remaining_wait("ghost") == 0