phase: 81_login_rate_limit
All gates verified. The phase is complete — everything was already implemented by prior task runs; this pass verified each completion criterion end-to-end and found no defects. **Phase 81 final verification report** (tasks 01–03 all present in `complete/`; no code changes needed this pass) - Verified: `app/core/rate_limit.py` (stdlib sliding window, fail-open) + both login routes' 429 pre-check/record/reset wiring + all unit/integration pins present (11th-429, blocked-success-still-429, clean-counter reset, shared counter, autouse `clean_rate_limit` fixture documented). - Live check (task 03): dev server + 11 rapid wrong logins → `401 ×10, 429`; 429 body carries `retry-after: 900` + generic detail; server restarted (per-process counter cleared by design). - `uv run pytest tests/unit/test_rate_limit.py -v --no-cov` → 10 passed - `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed, **TOTAL 99%** (>90%; rate_limit.py 100%, auth.py 100%) - `uv run pytest tests/e2e/test_smoke.py -v --no-cov` (isolation) → 3 passed - `uv run ruff check . && uv run pyright` → All checks passed / 0 errors, 0 warnings - Completion criteria: all met, except commit + phase-dir move — per harness rules I left all changes uncommitted in the working tree (harness commits atomically and moves the phase). - Diff scope: exactly `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py` + phase files; `pyproject.toml` / `uv.lock` / `frontend/` untouched. - Deviations: none in code; commit/move deferred to harness as instructed. - Next pending phase: `82_security_headers`.
This commit is contained in:
@@ -0,0 +1,69 @@
|
|||||||
|
# Phase 81 — Rate-limit failed sign-ins: 429 + Retry-After after repeated failures
|
||||||
|
|
||||||
|
**Source:** `.agents/remediation_plan.md` SEC-03 (security audit 2026-09-07, severity Medium — "No rate limiting / lockout on login + token-auth")
|
||||||
|
**Story:** n/a (security hardening — audit-derived, no user story)
|
||||||
|
**Context:** `app/api/auth.py` (`POST /api/login` — `check_password` then `sign_in`; `POST /api/token-auth` — `find_active_by_token` then session keys — both return one generic 401 on every failure shape, no throttling of any kind), `app/core/auth.py` (the constant-time `check_password`, `sign_in`/`sign_out` — untouched by this phase), `app/main.py` (middleware registration order — this phase adds no middleware), `frontend/assets/login.js` (the login form's error path: non-204 → `Sign-in failed (HTTP ${r.status}) — try again.` via `showError` — a 429 already renders there as a generic message, so no frontend change is required), `tests/unit/test_auth.py` + `tests/integration/test_auth_api.py` (the existing auth contract pins — every 401 shape, no enumeration).
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
A client IP that fails sign-in repeatedly (admin password or token) gets **429 + `Retry-After`** instead of unlimited line-rate guesses: a stdlib-only in-memory sliding-window limiter shared by both login routes, counting only failed attempts, reset on success, failing open (a limiter bug must never lock the owner out).
|
||||||
|
|
||||||
|
## Audit basis (read this, not the chat)
|
||||||
|
- Neither `POST /api/login` nor `POST /api/token-auth` throttles: the password is compared with `compare_digest` (fast), so an attacker who can reach `:8000` can try thousands of guesses per second (audit PoC: 10 000 sequential logins all answered at line rate).
|
||||||
|
- The fix is deliberately small and stateless-per-process (A10: no new service, no DB table, no Redis): a per-IP deque of failure timestamps in module memory, lost on restart (an acceptable reset — it is a friction bump, not the boundary; the homelab HTTP posture is the documented owner decision).
|
||||||
|
- Threshold (locked in this phase): **10 failed attempts per client IP per 15-minute sliding window** → subsequent attempts get **429** `{"detail": "too many failed sign-in attempts — try again later"}` + a `Retry-After: <seconds>` header until the window clears.
|
||||||
|
- The counter is SHARED by both routes (any auth failure from the IP counts — a token-spraying attack must not reset by alternating routes), and a SUCCESS on either route clears that IP's counter (a legitimate owner who fat-fingers twice is not poisoned).
|
||||||
|
|
||||||
|
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
|
||||||
|
- **A1 — in-memory, stdlib-only:** no new dependency, no DB table, per-process state (A10/A12 spirit). A restart clears the counter — accepted.
|
||||||
|
- **A2 — 10 / 15 min, module constants:** `MAX_FAILURES = 10`, `WINDOW_SECONDS = 900` as documented module constants in the limiter module — NOT new `BOR_` settings (a security control with a fixed, tested default; no env surface to mistype).
|
||||||
|
- **A3 — 429 contract:** one generic detail (no enumeration between login/token failures), `Retry-After` header carrying the whole-window seconds; the login page's existing `Sign-in failed (HTTP 429) — try again.` line renders it — no frontend change in this phase.
|
||||||
|
- **A4 — client IP = `request.client.host`:** the direct peer address. ASSUMPTION (locked here): the app is served directly (homelab), no reverse proxy — if a proxy is ever put in front, the limiter needs `X-Forwarded-For` handling (noted in the module docstring, out of scope now).
|
||||||
|
|
||||||
|
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||||
|
- **`app/core/rate_limit.py` (new)** — the limiter, pure stdlib (`collections.deque`, `time.monotonic`, `threading.Lock` for the dict — the ASGI event loop is single-threaded but the lock costs nothing and keeps the unit tests honest under pytest-xdist if ever added):
|
||||||
|
```python
|
||||||
|
MAX_FAILURES = 10
|
||||||
|
WINDOW_SECONDS = 900
|
||||||
|
|
||||||
|
_failures: dict[str, deque[float]] = {}
|
||||||
|
_lock = threading.Lock()
|
||||||
|
|
||||||
|
def record_failure(client_ip: str) -> None: ... # append now(); prune expired; never raises
|
||||||
|
def remaining_wait(client_ip: str) -> int: ... # 0 = allowed; else whole-window seconds (ceil) until the oldest counted failure expires
|
||||||
|
def reset(client_ip: str) -> None: ... # drop the IP's entry (on success)
|
||||||
|
```
|
||||||
|
Semantics (all unit-pinned): a failure is counted when it is recorded; the window slides — only failures within the last `WINDOW_SECONDS` count; the IP is blocked while the count of in-window failures `>= MAX_FAILURES` (the 10th failure already blocked? NO — the 10th failure is recorded and the **11th** attempt is the first 429: the check happens before the attempt); `remaining_wait` returns `int(ceil(WINDOW_SECONDS - (now - oldest)))` while blocked, `0` otherwise; **fail-open**: any internal error (corrupt deque, clock skew) → `remaining_wait` returns `0` and `record_failure` no-ops — a limiter bug never denies the owner.
|
||||||
|
- **`app/api/auth.py`** — the two route changes (identical shape):
|
||||||
|
- `login`: first line of the handler — `ip = request.client.host; if (wait := _wait(ip)) > 0: raise HTTPException(429, detail=TOO_MANY_DETAIL, headers={"Retry-After": str(wait)})`; on password failure → `record_failure(ip)` before the 401; on success → `reset(ip)` before the 204.
|
||||||
|
- `token_auth`: same three points (pre-check, record on the 401 path, reset on the 204 path).
|
||||||
|
- `TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"` (one string shared by both routes — no enumeration).
|
||||||
|
- Module docstrings updated: the 429 case joins the documented status set of each route.
|
||||||
|
- **Not touched:** `app/core/auth.py` (password check + session helpers), the 401 contract of both routes (byte-identical on the non-throttled path), everything else in the app, all frontend files.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
— (none; extends the completed phase-16/79 auth surface additively — the 401/204 contract is unchanged for non-throttled callers)
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
1. `01_rate_limiter_core.md` — `app/core/rate_limit.py` + the unit suite for the window semantics.
|
||||||
|
2. `02_auth_routes_wiring.md` — the 429 pre-check + failure/success bookkeeping on both login routes + integration tests.
|
||||||
|
3. `03_verify_and_commit.md` — full gate (suite + coverage + smoke E2E in isolation) + atomic commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- Unit — `tests/unit/test_rate_limit.py` (new): 9 failures → allowed; the 10th failure recorded; the next attempt → `remaining_wait > 0`; two independent IPs independent; a failure older than the window expires (monkeypatch a fake clock or pre-seed timestamps) and unblocks; `reset` clears (a success unblocks immediately); `remaining_wait` is the whole-window seconds, not the per-failure age; fail-open: a pre-corrupted state (e.g. a non-deque entry injected) → `remaining_wait` returns 0, `record_failure` doesn't raise.
|
||||||
|
- Integration — `tests/integration/test_auth_api.py` (extend): 10 wrong-password logins from the TestClient → 11th → **429** with the generic detail + a `Retry-After` header (int > 0); a correct password after 9 failures → 204 AND the counter is reset (the next 10 failures needed to block again — assert by 9 more failures still 401); token-auth failures share the counter (5 login + 5 token failures → the 11th of either → 429); the non-throttled 401 contract (existing pins) byte-identical.
|
||||||
|
- E2E (isolation gate per AGENTS.md rule 9): `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green — no UI change; the login form's existing 429 rendering is covered by the integration contract (the generic `HTTP ${status}` line).
|
||||||
|
- Coverage: **>90%** on `app/` — `app/core/rate_limit.py` is small and fully unit-exercised; the `app/api/auth.py` branches (pre-check hit/miss, record, reset) integration-covered.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] 11 rapid failed logins from one client: the first 10 → 401 (unchanged detail), the 11th → 429 + `Retry-After`; the token-auth route participates in the same counter (integration pins).
|
||||||
|
- [ ] A successful login/token-auth resets the IP's counter (integration pin).
|
||||||
|
- [ ] `uv run pytest tests/unit/test_rate_limit.py -v` green; full `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
|
||||||
|
- [ ] `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green in isolation.
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
|
- [ ] No dependency change (`pyproject.toml` / `uv.lock` untouched); `git diff --stat` limited to `app/core/rate_limit.py`, `app/api/auth.py`, the two test files, phase files.
|
||||||
|
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(auth): rate-limit failed logins and token-auth attempts with 429 + Retry-After`); phase dir moved to `.agents/phases/complete/`.
|
||||||
|
|
||||||
|
## Locked decisions
|
||||||
|
- **A10 intact** — no new service/table/middleware; one module + two route pre-checks.
|
||||||
|
- **401 contract unchanged** — the no-enumeration single-401 discipline of phase 16/79 is byte-identical for every non-throttled attempt; 429 is a new status, not a rewording.
|
||||||
|
- **No frontend change** — `login.js` already renders any non-204 as `Sign-in failed (HTTP ${status}) — try again.` (the 429 reads correctly there).
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Task 01 — The sliding-window limiter module + unit suite
|
||||||
|
|
||||||
|
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
`app/core/rate_limit.py` exists with the exact semantics from the phase design block, and `tests/unit/test_rate_limit.py` pins every one of them.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/core/rate_limit.py` (new) — implement per the phase overview's design block:
|
||||||
|
- constants `MAX_FAILURES = 10`, `WINDOW_SECONDS = 900`;
|
||||||
|
- module state: `_failures: dict[str, deque[float]]` + a `threading.Lock`;
|
||||||
|
- `record_failure(client_ip: str)` — under the lock: get/create the deque, append `time.monotonic()`, prune entries older than `WINDOW_SECONDS`; **never raises** (wrap the body, `except Exception: return` — the fail-open contract);
|
||||||
|
- `remaining_wait(client_ip: str) -> int` — under the lock: prune; count in-window entries; if `< MAX_FAILURES` return `0`; else `max(1, int(-oldest - now + WINDOW_SECONDS))` as a ceiling — i.e. `int(math.ceil(WINDOW_SECONDS - (now - oldest)))` clamped to >= 1; **never raises** (fail-open → 0);
|
||||||
|
- `reset(client_ip: str)` — under the lock: `_failures.pop(client_ip, None)`;
|
||||||
|
- module docstring: the audit basis (SEC-03), the 10/15-min threshold, the shared-counter intent (both login routes), fail-open rationale (a limiter bug must never lock the owner out), and the `request.client.host` assumption — direct peer, no reverse proxy in the current homelab deployment (proxy deployments would need `X-Forwarded-For` handling — out of scope).
|
||||||
|
2. `tests/unit/test_rate_limit.py` (new) — pin, in this order:
|
||||||
|
- 9 × `record_failure("ip")` → `remaining_wait("ip") == 0`;
|
||||||
|
- a 10th `record_failure` → `remaining_wait("ip") > 0` (and `<= WINDOW_SECONDS`);
|
||||||
|
- a second IP is unaffected (`remaining_wait("ip2") == 0`);
|
||||||
|
- expiry: pre-seed one IP's deque with a timestamp `WINDOW_SECONDS + 1` in the past (reach into the module state, or a small `for` loop calling `record_failure` after monkeypatching `time.monotonic` to walk the clock forward) → `remaining_wait` back to 0;
|
||||||
|
- `reset` after 10 failures → `remaining_wait == 0`;
|
||||||
|
- `remaining_wait` for an unknown IP → 0;
|
||||||
|
- fail-open: inject a corrupt entry (`_failures["bad"] = object()` — reach into module state) → `remaining_wait("bad") == 0` and `record_failure("bad")` returns without raising;
|
||||||
|
- `reset` on an unknown IP → no error.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/unit/test_rate_limit.py -v` green.
|
||||||
|
- Coverage: **>90%** on `app/core/rate_limit.py` (the fail-open branches included — they are the point).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest tests/unit/test_rate_limit.py -v` — all tests green.
|
||||||
|
- [ ] `uv run ruff check app/core/rate_limit.py tests/unit/test_rate_limit.py && uv run pyright` clean.
|
||||||
|
- [ ] `git diff --stat` for this task: exactly the two new files.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Task 02 — Wire the limiter into both login routes + integration tests
|
||||||
|
|
||||||
|
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
`POST /api/login` and `POST /api/token-auth` enforce the 429 pre-check and record/reset the counter, and the integration suite pins the HTTP contract (429 + `Retry-After`, shared counter, success reset, unchanged 401s).
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/api/auth.py` — add the module constant `TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"` and import the limiter (`from app.core import rate_limit`).
|
||||||
|
2. `app/api/auth.py::login` — insert at the TOP of the handler (before `check_password`):
|
||||||
|
- `ip = request.client.host or "unknown"` (the `or` guards a scope with no client — TestClient edge; fail-open spirit);
|
||||||
|
- `if (wait := rate_limit.remaining_wait(ip)) > 0: raise HTTPException(status_code=429, detail=TOO_MANY_DETAIL, headers={"Retry-After": str(wait)})`;
|
||||||
|
- in the failure branch: `rate_limit.record_failure(ip)` immediately before the 401;
|
||||||
|
- in the success branch: `rate_limit.reset(ip)` immediately before the 204.
|
||||||
|
3. `app/api/auth.py::token_auth` — the identical three insertions (same `ip` derivation, same `TOO_MANY_DETAIL`, same pre-check/record/reset points — the counter is shared by design).
|
||||||
|
4. `app/api/auth.py` module docstring — extend the two route bullets with the 429 case (one line each: "while the per-IP failure window is exhausted → 429 + `Retry-After`, one generic detail — audit SEC-03").
|
||||||
|
5. `tests/integration/test_auth_api.py` — extend (keep every existing pin intact; the TestClient's `request.client.host` is a fixed "testclient" value, so these tests share one counter — **add a fixture or helper that calls `rate_limit.reset("testclient")` between tests** so existing tests can't trip the limiter, and document why in the test module docstring):
|
||||||
|
- 10 × wrong-password `POST /api/login` → 11th → **429**, `detail == TOO_MANY_DETAIL`, `Retry-After` header present and `int(...) > 0`;
|
||||||
|
- after the 429, a CORRECT password → still 429 (the window is not bypassed by success — success only RESETS a clean counter; while blocked, the pre-check fires first; pin this: it is the intended semantics — an exhausted window stays exhausted until it slides);
|
||||||
|
- reset: with a clean counter, 9 failures + 1 success (204) → the next 9 failures are all 401 (the counter is back to 0 after the success);
|
||||||
|
- shared counter: 5 login failures + 5 token failures → the next token-auth attempt → 429;
|
||||||
|
- the existing non-throttled 401 pins (generic `invalid password` / `invalid token`) still green with the clean-counter fixture.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/integration/test_auth_api.py -v` green (new + existing pins).
|
||||||
|
- `uv run pytest tests/unit/test_rate_limit.py -v` still green (no regressions to task 01).
|
||||||
|
- Coverage: **>90%** on the modified `app/api/auth.py` branches (429 hit/miss, record, reset paths all hit by the integration tests).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] The 11th rapid failed login from one client is a 429 with `Retry-After` (integration pin); the token route shares the counter (integration pin).
|
||||||
|
- [ ] A success on a clean counter resets it (integration pin); a success while blocked does not unblock (integration pin).
|
||||||
|
- [ ] All pre-existing `test_auth_api.py` pins green (the clean-counter fixture is documented).
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Task 03 — Full gate + atomic commit
|
||||||
|
|
||||||
|
**Phase:** `81_login_rate_limit` · **Story:** n/a (security hardening — audit SEC-03)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Run the complete phase gate, land the phase as one atomic commit, and move the phase directory to `complete/`.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. **Full regression gate** (AGENTS.md rule 9):
|
||||||
|
- `uv run pytest` — unit + integration green.
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**; confirm `app/core/rate_limit.py` and `app/api/auth.py` show no meaningful uncovered lines in the new branches.
|
||||||
|
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` — green **in isolation** (this phase's E2E contract: no UI change — `login.js`'s existing non-204 rendering covers the 429).
|
||||||
|
- `uv run ruff check . && uv run pyright` — clean.
|
||||||
|
2. **Manual live check** (keep the output in the session log): start the dev server (`uv run uvicorn app.main:app`) and, from the shell, 11 rapid wrong-password logins: `for i in $(seq 11); do curl -s -o /dev/null -w "%{http_code} " -X POST localhost:8000/api/login -H 'Content-Type: application/json' -d '{"password":"wrong"}'; done` → expect `401 ×10, 429` (and the 429 body carries `Retry-After`). Restart the server to clear the counter (the per-process state is by design).
|
||||||
|
3. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py`, and the phase files:
|
||||||
|
`fix(auth): rate-limit failed logins and token-auth attempts with 429 + Retry-After`
|
||||||
|
— body: security audit SEC-03 (2026-09-07) — neither login route throttled, so a reachable deployment took unlimited line-rate guesses; a stdlib-only sliding-window limiter (10 failures / IP / 15 min, shared by both routes, success resets, fail-open) now answers the 11th attempt with 429 + `Retry-After` while every non-throttled 401/204 is byte-identical. No dependency or frontend change.
|
||||||
|
4. Move the phase directory: `mv .agents/phases/todo/81_login_rate_limit .agents/phases/complete/` and include the move in the same commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- This task IS the phase-level gate — the commands above are the completion evidence.
|
||||||
|
- Coverage: >90% held.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] The live check shows `401 ×10, 429` (output kept in the session log); the 429 response carries `Retry-After`.
|
||||||
|
- [ ] `uv run pytest` green; coverage >90%; `tests/e2e/test_smoke.py` green in isolation; ruff + pyright clean.
|
||||||
|
- [ ] Exactly one new commit; `git show --stat HEAD` lists the five files above + the phase files (todo → complete move) — nothing else (in particular `pyproject.toml` / `uv.lock` / `frontend/` untouched).
|
||||||
|
- [ ] `.agents/phases/complete/81_login_rate_limit/` exists; `todo/` no longer contains it.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Phase 82 — Security headers on every response: strict CSP, no-framing, nosniff
|
||||||
|
|
||||||
|
**Source:** `.agents/remediation_plan.md` SEC-04 (security audit 2026-09-07, severity Medium — "No security headers (CSP, X-Frame-Options / frame-ancestors)")
|
||||||
|
**Story:** n/a (security hardening — audit-derived, no user story)
|
||||||
|
**Context:** no response in the app carries `Content-Security-Policy`, `X-Frame-Options`, or `X-Content-Type-Options` (checked `app/main.py`, `app/core/caching.py`, all templates — none do). The frontend is No-CDN and was verified for this phase: **no inline `<style>` blocks, no `style="…"` attributes, no `onclick`/`on*` handlers, no `javascript:` URLs, no external hosts** in any `frontend/*.html` or the JS-injected markup (inline `<svg>` elements are CSP-legal; the two `el.style.…` CSSOM writes in `app.js` are not CSP-blocked). `app/main.py` is the single app-assembly point (middleware added in order: `SessionMiddleware` → `configure_caching`'s `CachingMiddleware` → static mount last). `app/core/caching.py` is the existing transport-layer middleware precedent (but it BUFFERS page bodies — this phase's middleware must not).
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Every response — pages, static assets, API JSON, the SSE chat stream, even 404s — carries a strict same-origin CSP (`frame-ancestors 'none'` included → clickjacking closed), `X-Frame-Options: DENY`, and `X-Content-Type-Options: nosniff`, added by a header-only pure-ASGI middleware that never touches a body (the SSE stream passes through byte-identical).
|
||||||
|
|
||||||
|
## Audit basis (read this, not the chat)
|
||||||
|
- Every page (including the admin sign-in and the admin-only views) can be embedded in a third-party page's iframe: a malicious LAN page can overlay the admin UI and trick the signed-in owner into clicking Sync / Revoke / Delete-source actions (audit PoC: `<iframe src="http://<server>:8000/git-sources.html">` with a benign-looking overlay). `frame-ancestors 'none'` (in the CSP) + `X-Frame-Options: DENY` (legacy fallback) close it.
|
||||||
|
- The app serves no CSP today: the codebase is XSS-clean (escape-first renderer, `textContent` for data — audit-verified at every `innerHTML` site), but the CSP is the standard second line of defense against any future regression. It is cheap here precisely because of the No-CDN + no-inline-markup verification above: `default-src 'self'` is sufficient without any `'unsafe-inline'` — a genuinely strict policy.
|
||||||
|
- `nosniff` is the one-line belt-and-braces against MIME-confusion on the static mount.
|
||||||
|
|
||||||
|
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
|
||||||
|
- **A1 — the exact policy:** `Content-Security-Policy: default-src 'self'; base-uri 'none'; frame-ancestors 'none'` — no `script-src`/`style-src`/`connect-src` entries (they inherit `default-src 'self'`); no `'unsafe-inline'` anywhere (verified unnecessary); no `report-uri`/`report-to` (no collector in the homelab — a report would just vanish).
|
||||||
|
- **A2 — header-only, pure ASGI:** a small pure-ASGI middleware (not `BaseHTTPMiddleware`) — it wraps `send` and writes headers on `http.response.start` only; no body drain, no buffering, so the SSE chat stream and every other response pass through byte-identical (the explicit contrast with `CachingMiddleware`, which does buffer page bodies).
|
||||||
|
- **A3 — outermost position:** registered in `app/main.py` AFTER `configure_caching(app)` (last `add_middleware` = outermost), so even responses built by the static catch-all / 404 handler carry the headers.
|
||||||
|
- **A4 — the three headers only:** CSP + `X-Frame-Options: DENY` + `X-Content-Type-Options: nosniff`. No `Referrer-Policy`, no `Permissions-Policy`, no HSTS (the app serves plain homelab HTTP — HSTS would be wrong here; TLS is the documented non-goal).
|
||||||
|
|
||||||
|
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||||
|
- **`app/core/security_headers.py` (new)**:
|
||||||
|
```python
|
||||||
|
CSP = "default-src 'self'; base-uri 'none'; frame-ancestors 'none'"
|
||||||
|
|
||||||
|
class SecurityHeadersMiddleware:
|
||||||
|
"""Header-only, pure-ASGI — never reads or buffers a body (SSE-safe)."""
|
||||||
|
def __init__(self, app): self.app = app
|
||||||
|
async def __call__(self, scope, receive, send):
|
||||||
|
if scope["type"] != "http":
|
||||||
|
await self.app(scope, receive, send)
|
||||||
|
return
|
||||||
|
async def send_wrapper(message):
|
||||||
|
if message["type"] == "http.response.start":
|
||||||
|
headers = MutableHeaders(scope=message)
|
||||||
|
headers["Content-Security-Policy"] = CSP
|
||||||
|
headers["X-Frame-Options"] = "DENY"
|
||||||
|
headers["X-Content-Type-Options"] = "nosniff"
|
||||||
|
await send(message)
|
||||||
|
await self.app(scope, receive, send_wrapper)
|
||||||
|
```
|
||||||
|
Module docstring: the audit basis (SEC-04), the No-CDN verification that makes `'unsafe-inline'` unnecessary (re-verify if inline scripts/styles are ever introduced — the policy would break the page loudly, which is the feature), the pure-ASGI rationale (body untouched — SSE stream byte-identical, unlike `CachingMiddleware`'s page buffering).
|
||||||
|
- **`app/main.py`** — import + `app.add_middleware(SecurityHeadersMiddleware)` placed immediately AFTER the `configure_caching(app)` call (with a one-line comment: outermost on purpose — headers on every response incl. static 404s). Nothing else in `main.py` moves.
|
||||||
|
- **Not touched:** `app/core/caching.py`, any template, any JS, the API layer, the No-CDN surface.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
— (none; standalone transport hardening — wraps the whole app, no route or template change)
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
1. `01_headers_middleware.md` — `app/core/security_headers.py` + unit tests (incl. the SSE-stream passthrough pin).
|
||||||
|
2. `02_registration_and_integration.md` — `app/main.py` registration + integration tests on real app responses.
|
||||||
|
3. `03_e2e_and_commit.md` — the dedicated Playwright suite + full gate + atomic commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- Unit — `tests/unit/test_security_headers.py` (new): a minimal ASGI test-app wrapped by the middleware → a plain response carries all three headers with the exact values; a 404-shaped response carries them too; a **streaming** response (a generator yielding two chunks, `media_type="text/event-stream"`) — the body is consumed byte-identical (both chunks, in order) AND the headers are present (the SSE passthrough pin); a non-http scope (e.g. `websocket`) passes through with the app called and no `send` wrapping error.
|
||||||
|
- Integration — `tests/integration/test_security_headers.py` (new): against the real app (the `conftest` client): `GET /` (a page) → all three headers, CSP exactly the A1 string; `GET /api/health` → all three; `GET /assets/styles.css` → all three; `GET /nope` (404) → all three. (If the existing `tests/integration/test_caching_revalidation.py` fixture pattern for the app client exists, reuse it — same setup, no new fixtures.)
|
||||||
|
- E2E (mandatory, dedicated suite — this phase changes what the browser receives): `tests/e2e/test_security_headers.py` — (1) navigate to the chat page: the navigation `response.headers` carry all three (CSP exact); (2) the page actually works under the CSP — no CSP violations: collect `console` messages during load + one chat turn is NOT required (mock LLM is not needed for a header test) — assert zero console messages matching `/Content Security Policy/` and that the stylesheet applied (computed `background-color` of `body` is not `rgba(0, 0, 0, 0)` — the page painted, not a CSP-broken white page); (3) navigate to `/sources.html` (a second page) → headers present, no CSP violations. Run in isolation per AGENTS.md rule 9.
|
||||||
|
- Coverage: **>90%** on `app/` — the new module is small and unit-exercised; `app/main.py`'s registration is covered by every integration/E2E boot.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `curl -sI localhost:8000/ | grep -iE "content-security-policy|x-frame-options|x-content-type"` shows all three with the exact A1 CSP value (manual check, output kept in the session log); same for `/api/health`, `/assets/styles.css`, and a 404 path.
|
||||||
|
- [ ] The SSE chat stream is byte-identical through the middleware: the unit streaming pin + the existing chat E2E suites (e.g. `tests/e2e/test_chat_rag.py`) green — a body-touching regression would break them.
|
||||||
|
- [ ] `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` green **in isolation** — headers on real page loads + zero CSP violations + the page painted.
|
||||||
|
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||||
|
- [ ] No template/JS/`pyproject.toml`/`uv.lock` diff; `git diff --stat` limited to the two new files, `app/main.py`, the three test files, phase files.
|
||||||
|
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(security): add CSP, X-Frame-Options and nosniff headers to every response`); phase dir moved to `.agents/phases/complete/`.
|
||||||
|
|
||||||
|
## Locked decisions
|
||||||
|
- **No-CDN contract intact** — the CSP is `default-src 'self'` because the audit verified zero external hosts and zero inline scripts/styles; introducing either later breaks pages loudly (the policy does the enforcing).
|
||||||
|
- **No HSTS / Referrer-Policy / Permissions-Policy** (owner A4) — plain homelab HTTP is the documented posture; the three headers above are the scope.
|
||||||
|
- **SSE byte-identity** — the middleware wraps `send` for headers only; any body change is a phase failure (the chat E2E suites are the tripwire).
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# Task 01 — The pure-ASGI header middleware + unit suite
|
||||||
|
|
||||||
|
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
`app/core/security_headers.py` exists exactly per the phase design block, and `tests/unit/test_security_headers.py` pins the header values, the 404 path, the SSE streaming passthrough (byte-identical body + headers present), and the non-http scope passthrough.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/core/security_headers.py` (new) — implement per the phase overview's design block: the `CSP` constant (the exact A1 string), `SecurityHeadersMiddleware` as a pure-ASGI class wrapping `send` (headers written on `http.response.start` via `starlette.datastructures.MutableHeaders(scope=message)`), the `scope["type"] != "http"` fast path, and the module docstring content listed in the design block (audit basis, the No-CDN verification, the pure-ASGI/SSE rationale).
|
||||||
|
2. `tests/unit/test_security_headers.py` (new):
|
||||||
|
- a tiny ASGI `test_app` (a plain 200 `Response`-like dict sequence, or starlette's `Response` wrapped in an ASGI callable) + the middleware: assert all three headers with exact values;
|
||||||
|
- a 404-shaped response (status 404) → same three headers;
|
||||||
|
- a streaming response (an async generator yielding `data: a\n\n` then `data: b\n\n`, `media_type="text/event-stream"`) → consume the full body via the ASGI `receive`/`send` protocol (drive it with a small async client loop, or `httpx.AsyncClient(transport=ASGITransport(...))` if already used in the unit suite) → body is exactly the two frames in order AND the three headers are present (the SSE pin);
|
||||||
|
- a `websocket` scope → the app is called, no crash, no header injection attempt.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/unit/test_security_headers.py -v` green.
|
||||||
|
- Coverage: **>90%** on `app/core/security_headers.py` (every branch: http start, http other messages, non-http scope).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest tests/unit/test_security_headers.py -v` — all tests green, including the SSE byte-identity pin.
|
||||||
|
- [ ] `uv run ruff check app/core/security_headers.py tests/unit/test_security_headers.py && uv run pyright` clean.
|
||||||
|
- [ ] `git diff --stat` for this task: exactly the two new files.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Task 02 — Register the middleware + integration tests on real app responses
|
||||||
|
|
||||||
|
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
The middleware wraps the whole app (outermost), and the integration suite pins the three headers on a page, an API response, a static asset, and a 404 — against the real app.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/main.py` — import `SecurityHeadersMiddleware` and add `app.add_middleware(SecurityHeadersMiddleware)` immediately AFTER the `configure_caching(app)` call, with the one-line comment from the design block (outermost on purpose: headers on every response, including static catch-all 404s). Do not reorder or re-add any existing middleware.
|
||||||
|
2. `tests/integration/test_security_headers.py` (new) — use the same app/client fixture pattern as `tests/integration/test_caching_revalidation.py` (reuse its setup; do not duplicate the client factory if a shared helper exists):
|
||||||
|
- `GET /` → 200 + all three headers; `Content-Security-Policy` exactly `default-src 'self'; base-uri 'none'; frame-ancestors 'none'`;
|
||||||
|
- `GET /api/health` → 200 + all three;
|
||||||
|
- `GET /assets/styles.css` → 200 + all three;
|
||||||
|
- `GET /definitely-not-a-page` → 404 + all three (the static catch-all's 404 still carries them — the outermost-position proof);
|
||||||
|
- a regression pin: the page body is still the rewritten HTML with `?v=` asset refs (assert one `?v=` occurrence in the body — the two middlewares coexist and the caching rewrite still runs, i.e. header middleware did not swallow/alter the CachingMiddleware's work).
|
||||||
|
3. Run the existing caching integration suite (`tests/integration/test_caching_revalidation.py`) to prove no interaction regression (it must stay green untouched).
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/integration/test_security_headers.py tests/integration/test_caching_revalidation.py -v` green.
|
||||||
|
- Coverage: **>90%** on `app/` (the registration line is hit by every integration/E2E boot; no new uncovered branches).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] All four real-response pins pass (page / API / asset / 404) with the exact CSP string.
|
||||||
|
- [ ] The `?v=` coexistence pin passes (caching rewrite intact under the new outermost middleware).
|
||||||
|
- [ ] `tests/integration/test_caching_revalidation.py` green untouched.
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Task 03 — Dedicated Playwright suite + full gate + atomic commit
|
||||||
|
|
||||||
|
**Phase:** `82_security_headers` · **Story:** n/a (security hardening — audit SEC-04)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Prove the headers reach a real browser AND the strict CSP does not break the page (no violations, page painted), run the full gate, and land the phase as one atomic commit.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `tests/e2e/test_security_headers.py` (new) — follow the house E2E conventions (read `tests/e2e/conftest.py` + one existing suite, e.g. `test_header_consistency.py`, for the fixtures and the app-server pattern):
|
||||||
|
- **Headers on a real navigation:** `page.goto` the chat page (the shell at `/`) → the navigation response's headers carry all three, CSP exactly the A1 string; `page.goto` `/sources.html` → same (a second page).
|
||||||
|
- **No CSP violations:** attach a `page.on("console")` listener before navigation; after both loads assert zero console messages whose text matches `/Content Security Policy/i` (Chromium reports CSP denials to the console).
|
||||||
|
- **The page painted (CSP not breaking assets):** after the chat-page load, `document.querySelector('body')`'s computed `background-color` is not `rgba(0, 0, 0, 0)` (the dark-tech palette applied — styles loaded under the CSP) and the shell's root element is present (e.g. `#main` — the page booted, not a white-broken document).
|
||||||
|
- No LLM interaction needed (the page load is the contract) — the suite must not depend on the aipi endpoint.
|
||||||
|
2. **Manual live check** (keep the output in the session log): `curl -sI localhost:8000/ | grep -iE "content-security-policy|x-frame-options|x-content-type"` + the same for `/api/health`, `/assets/styles.css`, `/nope` (404).
|
||||||
|
3. **Full regression gate** (AGENTS.md rule 9):
|
||||||
|
- `uv run pytest` — unit + integration green.
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
|
||||||
|
- `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` — green **in isolation** (this phase's mandatory E2E).
|
||||||
|
- ONE chat-stream suite to prove SSE byte-identity end-to-end: `uv run pytest tests/e2e/test_chat_rag.py -v --no-cov` green (the mock-LLM chat suite — a body-touching regression in the middleware would break the stream).
|
||||||
|
- `uv run ruff check . && uv run pyright` — clean.
|
||||||
|
4. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/security_headers.py`, `app/main.py`, the three test files, and the phase files:
|
||||||
|
`fix(security): add CSP, X-Frame-Options and nosniff headers to every response`
|
||||||
|
— body: security audit SEC-04 (2026-09-07) — no response carried framing/XSS-mitigation headers: any page (incl. the admin UI) could be iframe-embedded for clickjacking. A header-only pure-ASGI middleware (outermost) now sets `default-src 'self'; base-uri 'none'; frame-ancestors 'none'` (no 'unsafe-inline' — the No-CDN + no-inline-markup audit made the strict policy possible), `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff` on every response including 404s; the SSE stream passes byte-identical (unit streaming pin + chat E2E regression). No template/JS/dependency change.
|
||||||
|
5. Move the phase directory: `mv .agents/phases/todo/82_security_headers .agents/phases/complete/` and include the move in the same commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- E2E: the dedicated suite green in isolation (headers + zero CSP violations + painted page).
|
||||||
|
- Regression: `test_chat_rag.py` green (SSE byte-identity through the real middleware).
|
||||||
|
- Coverage: >90% held.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` green in isolation (both pages, exact CSP, no violations, page painted).
|
||||||
|
- [ ] `uv run pytest tests/e2e/test_chat_rag.py -v --no-cov` green (the stream is untouched).
|
||||||
|
- [ ] The `curl` header check output (4 URLs) kept in the session log.
|
||||||
|
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean.
|
||||||
|
- [ ] Exactly one new commit; `git show --stat HEAD` lists the two app files, the three test files, and the phase files (todo → complete move) — nothing else.
|
||||||
|
- [ ] `.agents/phases/complete/82_security_headers/` exists; `todo/` no longer contains it.
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
# Phase 83 — Bound the anonymous saved-chat payload sizes at the schema boundary
|
||||||
|
|
||||||
|
**Source:** `.agents/remediation_plan.md` SEC-05 (security audit 2026-09-07, severity Medium — "Unbounded payload sizes on the anonymous public chat-save endpoints")
|
||||||
|
**Story:** n/a (security hardening — audit-derived, no user story)
|
||||||
|
**Context:** `app/schemas.py` — `ChatMessage` (`who`/`text: Field(min_length=1)` **no max**/`sources: list[SourceRef] | None`/`suggestions: list[str] | None`/`thinking: str | None` **no max**/`tools: list[ToolCall] | None`/`stopped`, `extra="forbid"`), `SourceRef` (`source`/`path`/`title` — all bare `str`, **no max** — shared with the server-built `ChatDoneEvent.sources` SSE shape), `ToolCall` (`name: str`/`argument: str | None` — bare, **no max**), `SavedChatCreate` (`title: max_length=500`/`messages: Field(min_length=1)` **no max_items**/`share`), `SavedChatUpdate` (same message list) — consumed by the **public** (no-session) routes `POST /api/chats` and `PUT /api/chats/{chat_id}` in `app/api/chats.py` (phase 55 A1: the write surface is public — the row id is the credential). `ChatMessage` is shared with `HistoryTurn` (`text: max_length=32_000`, `thinking: max_length=32_000` — the existing caps to mirror) and with the server-built `ChatDoneEvent`/SSE shapes. The DB side: `documents.source String(120)`, `documents.path String(1000)`, `documents.title String(500)` (the column lengths the caps mirror), `saved_chats.messages` is JSONB (unbounded at the DB level — the pydantic boundary is the only gate). `tests/unit/` has no existing `test_schemas.py`; `tests/integration/test_chats_api.py` exists (the phase-50/51/53/55 contract pins — the 201/404/409/share matrix).
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
An anonymous `POST/PUT /api/chats` can no longer carry a single arbitrarily large string, an unbounded message list, or unbounded nested lists: every `ChatMessage`/`SourceRef`/`ToolCall`/messages-list field gets a pydantic cap (422 at the boundary, house style), sized to the realistic `bor.chat.v1` record the UI produces — while a normal save (the E2E suites' payloads) still lands 201 byte-for-byte.
|
||||||
|
|
||||||
|
## Audit basis (read this, not the chat)
|
||||||
|
- `ChatMessage.text` has `min_length=1` and **no `max_length`** (unlike its sibling `HistoryTurn.text` at 32 000), and NO list field anywhere in the saved-chat shape has `max_items` — an anonymous caller can POST one 200 MB JSON string (`{"messages":[{"who":"user","text":"aaa…"}]}`) and create unlimited such rows: an unauthenticated memory + storage DoS on the app's only anonymous write surface (audit PoC in the plan).
|
||||||
|
- The fix is boundary-only (the phase-56 house style: fail loud at the schema with a 422 — no route code changes at all; FastAPI's request validation rejects before the handler runs, nothing is stored).
|
||||||
|
- The caps mirror the EXISTING `HistoryTurn` caps for the text fields (32 000 — a single chat message longer than that is already rejected on the chat path, so a saved chat can never legitimately carry more) and the DB column lengths for the source-ref fields (a `SourceRef` is built from `documents` rows server-side — `source ≤120`, `path ≤1000`, `title ≤500` — so the SSE `done` event can never trip the new caps: the server's own values always fit).
|
||||||
|
|
||||||
|
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
|
||||||
|
- **A1 — the exact caps** (pydantic boundary, 422 on overflow):
|
||||||
|
- `ChatMessage.text`: `max_length=32_000` (mirror `HistoryTurn.text`);
|
||||||
|
- `ChatMessage.thinking`: `max_length=32_000` (mirror `HistoryTurn.thinking`);
|
||||||
|
- `ChatMessage.sources`: `max_items=20` (top-N docs + agent reads — the UI shows a handful; 20 is 10× the realistic max);
|
||||||
|
- `ChatMessage.suggestions`: `max_items=50`, each item `max_length=200` (chips are short deterministic strings — `derive_suggestions` produces ≤ ~80 chars);
|
||||||
|
- `ChatMessage.tools`: `max_items=50` (one entry per tool call; the round cap is 10 and even generous multi-call turns stay far below 50);
|
||||||
|
- `SourceRef`: `source max_length=120`, `path max_length=1000`, `title max_length=500` (mirror the `documents` column lengths — server-built SSE values always fit);
|
||||||
|
- `ToolCall`: `name max_length=100`, `argument max_length=2000 | None` (the combined `source/path` identity is ≤ 120 + 1 + 1000; 2 000 is 2× headroom for a grep pattern);
|
||||||
|
- `SavedChatCreate.messages` / `SavedChatUpdate.messages`: `max_items=200` (well past any realistic conversation — the chat history budget itself is 40 turns — and far below a DoS-sized list).
|
||||||
|
- **A2 — boundary-only:** NO route/handler changes in `app/api/chats.py` (FastAPI validates the pydantic model before the handler — the 422 is the framework's standard validation response); the stored-row contract (JSONB round-trip, `extra="forbid"`, the null-safe restore path) is untouched.
|
||||||
|
- **A3 — `SourceRef` is shared:** the caps on `SourceRef` also constrain the client-side `sources` inside saved chats AND are satisfied by every server-built `ChatDoneEvent.sources` (column-length mirror) — the SSE path is provably unaffected (pinned by an integration test).
|
||||||
|
|
||||||
|
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||||
|
- **`app/schemas.py`** — the ONLY file changed in `app/`:
|
||||||
|
- `SourceRef`: `source: str = Field(max_length=120)`, `path: str = Field(max_length=1000)`, `title: str = Field(max_length=500)` (docstring: caps mirror the `documents` column lengths — server-built SSE refs always fit; client-saved refs are bounded at the boundary).
|
||||||
|
- `ToolCall`: `name: str = Field(max_length=100)`, `argument: str | None = Field(default=None, max_length=2000)`.
|
||||||
|
- `ChatMessage`: `text: str = Field(min_length=1, max_length=32_000)`, `thinking: str | None = Field(default=None, max_length=32_000)`, `sources: list[SourceRef] | None = Field(default=None, max_items=20)`, `suggestions: list[str] | None = Field(default=None, max_items=50)` with item length enforced by `Field(max_length=200)` on the list item type (pydantic v2: annotate `list[Annotated[str, Field(max_length=200)]]` or a small `_Chip = Annotated[str, Field(max_length=200)]` alias — keep the JSON shape identical), `tools: list[ToolCall] | None = Field(default=None, max_items=50)`.
|
||||||
|
- `SavedChatCreate.messages` / `SavedChatUpdate.messages`: `Field(min_length=1, max_items=200)`.
|
||||||
|
- Update the affected docstrings: each cap's one-line rationale (the A1 mirror sources) — the file's dense-docstring house style.
|
||||||
|
- **Not touched:** `app/api/chats.py` (validation happens before the handlers — zero route diff), the frontend (the UI's real payloads are far inside every cap — the E2E save suites prove it), the DB (no migration — JSONB stays unbounded at rest; the boundary is the gate, A2).
|
||||||
|
- **422 shape:** FastAPI's standard validation error body (the house boundary response — same shape the existing `min_length` violations already produce; no custom error copy).
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
— (none; boundary-only hardening on the completed phase-50/51/55 saved-chat surface — the 201/404/409/share contract is unchanged for in-cap payloads)
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
1. `01_schema_caps.md` — the `app/schemas.py` cap changes + the unit suite (one test per cap, boundary values included).
|
||||||
|
2. `02_integration_and_e2e.md` — the oversized-422 integration pins + the dedicated Playwright suite + the SSE `done`-event unaffected pin.
|
||||||
|
3. `03_verify_and_commit.md` — full gate (incl. the existing chat-save E2E suites) + atomic commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- Unit — `tests/unit/test_schemas.py` (new; the first schema-boundary suite): for EACH cap — a value exactly at the cap validates; one past it raises a pydantic `ValidationError` naming the field (`text` 32_000/32_001, `thinking` same, `sources` 20/21 items, `suggestions` 50/51 items + a 200/201-char item, `tools` 50/51, `SourceRef` source/path/title 120/1000/500 boundaries, `ToolCall` name 100/101 + argument 2000/2001, `SavedChatCreate.messages` 200/201 items, `SavedChatUpdate.messages` 200/201) — plus a regression pin: a realistic `bor.chat.v1` payload (a few messages, sources, tools, thinking) validates cleanly and round-trips `model_dump()` (the stored-shape contract).
|
||||||
|
- Integration — `tests/integration/test_chats_api.py` (extend, existing pins intact): anonymous `POST /api/chats` with a 32_001-char text → **422** AND no row created (list stays the same length); `messages` with 201 items → 422; a 21-item `sources` list → 422; the existing 201 create / share / unshare / stale pins green; an SSE regression pin: a `ChatDoneEvent` built from a full-length `Document` row (source 120 / path 1000 / title 500 — construct the row values at the column maxima) still serializes (A3: the server-built refs fit the new caps — build the event and `model_dump()` it in an integration test next to the existing chat pins).
|
||||||
|
- E2E (dedicated suite — this phase's contract is reachable from a browser's network layer): `tests/e2e/test_chat_save_payload_limits.py` — using Playwright's API request context against the running app (the house pattern — `page.request` or the context's request API): anonymous `POST /api/chats` with an oversized text (e.g. 40_000 chars) → **422** (no session needed — the surface is public, exactly the audit vector); a normal small save → 201 + `id` in the body (the happy path still works end-to-end). Run in isolation per AGENTS.md rule 9.
|
||||||
|
- Regression E2E: the existing save flows stay green — `tests/e2e/test_chat_history.py` (save/restore through the real UI — the in-cap proof).
|
||||||
|
- Coverage: **>90%** on `app/` (the changed file is `app/schemas.py` — declarative, exercised by every unit/integration test).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] Every A1 cap is pinned at both boundaries (at-cap passes, over-cap 422s) in `tests/unit/test_schemas.py`.
|
||||||
|
- [ ] Anonymous oversized `POST /api/chats` → 422 with **no row stored** (integration pin); the same vector from the Playwright suite → 422 (E2E pin).
|
||||||
|
- [ ] The SSE `done`-event pin passes at the column-maximum source-ref lengths (A3).
|
||||||
|
- [ ] `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` green in isolation; `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` green (real UI saves unaffected).
|
||||||
|
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||||
|
- [ ] `git diff --stat` limited to `app/schemas.py`, the three test files, phase files (NO `app/api/chats.py` diff, no migration, no `pyproject.toml`/`uv.lock`/`frontend/` diff).
|
||||||
|
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(api): bound anonymous saved-chat payload sizes at the schema boundary`); phase dir moved to `.agents/phases/complete/`.
|
||||||
|
|
||||||
|
## Locked decisions
|
||||||
|
- **A2 boundary-only** — no route/handler/migration change; the pydantic 422 IS the control (house style, phase 56 precedent).
|
||||||
|
- **Caps mirror existing invariants** (A1) — `HistoryTurn`'s 32 000 text cap and the `documents` column lengths are the sizing sources, so no legitimate payload (chat-path history OR server-built SSE refs) can ever trip a cap; only oversized anonymous input does.
|
||||||
|
- **JSONB stored shape untouched** — `extra="forbid"` and the null-safe round-trip contract (the phase-14/50 restore path) are byte-identical for in-cap payloads (the unit round-trip pin).
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Task 01 — The schema caps + unit boundary suite
|
||||||
|
|
||||||
|
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
`app/schemas.py` carries every A1 cap from the phase overview, and `tests/unit/test_schemas.py` pins each cap at both boundaries plus the realistic-payload round-trip regression.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/schemas.py` — apply the caps exactly as the phase overview's design block lists them:
|
||||||
|
- `SourceRef` — `source`/`path`/`title` → `Field(max_length=120/1000/500)`;
|
||||||
|
- `ToolCall` — `name` → `Field(max_length=100)`, `argument` → `Field(default=None, max_length=2000)`;
|
||||||
|
- `ChatMessage` — `text` → `Field(min_length=1, max_length=32_000)`, `thinking` → `Field(default=None, max_length=32_000)`, `sources` → `Field(default=None, max_items=20)`, `suggestions` → a list of `_Chip` (`Annotated[str, Field(max_length=200)]`) with `Field(default=None, max_items=50)`, `tools` → `Field(default=None, max_items=50)`;
|
||||||
|
- `SavedChatCreate.messages` + `SavedChatUpdate.messages` → `Field(min_length=1, max_items=200)`;
|
||||||
|
- docstrings: one rationale line per cap group (the A1 mirror sources — `HistoryTurn` caps / `documents` column lengths), matching the file's dense style.
|
||||||
|
- Verify no JSON-shape change: the models still accept/reject exactly the same KEYS (`extra="forbid"` untouched) — only value bounds are added.
|
||||||
|
2. `tests/unit/test_schemas.py` (new) — one test per cap at BOTH boundaries (at-cap validates; one-over raises `ValidationError` — assert the failing field name via `e.errors()[0]["loc"]`):
|
||||||
|
- `ChatMessage.text` 32_000 / 32_001; `thinking` 32_000 / 32_001 (and `None` still valid);
|
||||||
|
- `sources` 20 / 21 items; `suggestions` 50 / 51 items + a single 200 / 201-char item; `tools` 50 / 51;
|
||||||
|
- `SourceRef.source` 120 / 121, `.path` 1000 / 1001, `.title` 500 / 501;
|
||||||
|
- `ToolCall.name` 100 / 101, `.argument` 2000 / 2001 (and `None` still valid);
|
||||||
|
- `SavedChatCreate.messages` 200 / 201 items, `SavedChatUpdate.messages` 200 / 201;
|
||||||
|
- the realistic-payload regression: a full `bor.chat.v1`-shaped `SavedChatCreate` (4–8 messages mixing user/brain, one brain message with `thinking` + `tools` + `sources`, one with `suggestions` + `stopped`) → validates, and `model_dump()` of the messages equals the input dict (the stored-shape round-trip contract, `None`-keys preserved).
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/unit/test_schemas.py -v` green.
|
||||||
|
- Coverage: **>90%** on `app/schemas.py` (declarative — exercised by every test in the new file).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest tests/unit/test_schemas.py -v` — every boundary test green (at-cap passes, over-cap 422-shaped `ValidationError` with the right `loc`).
|
||||||
|
- [ ] The round-trip regression test green (stored shape unchanged for in-cap payloads).
|
||||||
|
- [ ] `uv run pytest tests/unit/ -q` green (no unit regression — in particular any test that constructs `ChatMessage`/`SourceRef` values still passes).
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
# Task 02 — Integration pins + the dedicated Playwright suite
|
||||||
|
|
||||||
|
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
The HTTP contract is pinned: anonymous oversized saves 422 and store nothing, the SSE `done`-event shape is provably unaffected at the column maxima, and the dedicated E2E suite drives the exact audit vector through a real browser's network layer.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `tests/integration/test_chats_api.py` — extend (every existing pin stays green; if the file's fixtures reset the DB between tests, the new tests follow the same pattern):
|
||||||
|
- anonymous `POST /api/chats` with one message whose `text` is 32_001 chars → **422** AND the subsequent admin `GET /api/chats` list length is unchanged (nothing stored);
|
||||||
|
- `POST /api/chats` with 201 messages (minimal valid each) → 422;
|
||||||
|
- `POST /api/chats` with one message carrying a 21-item `sources` list (valid `SourceRef` shapes) → 422;
|
||||||
|
- `PUT /api/chats/{id}` (an existing row) with an oversized message → 422 and the row content unchanged (a GET shows the original text);
|
||||||
|
- the happy path regression: a normal small save → 201 (the existing pins already cover this — confirm green).
|
||||||
|
2. **The A3 SSE pin** — in the same file (or the existing chat-API integration file where the `ChatDoneEvent` shape is already exercised — `tests/integration/test_chat_api.py`): build the event from maximum-length values — `SourceRef(source="s"*120, path="p"*1000, title="t"*500)` inside a `ChatDoneEvent(deflected=False, sources=[…], suggestions=[])` → `model_dump()` succeeds (the server-built refs fit the new caps; a failure here would mean the caps broke the SSE contract).
|
||||||
|
3. `tests/e2e/test_chat_save_payload_limits.py` (new) — house E2E conventions (read `tests/e2e/conftest.py` + `test_chat_history.py` for the server/client fixtures):
|
||||||
|
- the audit vector, end-to-end and anonymous (NO login): via the Playwright request API (`page.request.post("/api/chats", data={...})` on a fresh page) POST one message with a 40_000-char `text` → expect **422**;
|
||||||
|
- the happy path in the same suite: a small valid save → **201** with an `id` field (proves the boundary didn't break the real flow from the browser layer);
|
||||||
|
- no LLM dependency (the endpoints are DB-only).
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/integration/test_chats_api.py -v` green (new + existing).
|
||||||
|
- The A3 pin green wherever it lands.
|
||||||
|
- `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` green **in isolation** (this phase's mandatory E2E).
|
||||||
|
- Coverage: **>90%** on `app/` (no new `app/` code — the gate is regression + boundary proof).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] All four oversized-422 integration pins pass (text / message-count / sources / PUT), each with the "nothing stored / row unchanged" assertion.
|
||||||
|
- [ ] The A3 SSE pin passes at the column-maximum lengths.
|
||||||
|
- [ ] The dedicated E2E suite green in isolation (422 anonymous + 201 small save).
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Task 03 — Full gate + atomic commit
|
||||||
|
|
||||||
|
**Phase:** `83_chat_save_payload_limits` · **Story:** n/a (security hardening — audit SEC-05)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Run the complete phase gate (including the real-UI save regression), land the phase as one atomic commit, and move the phase directory to `complete/`.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. **Full regression gate** (AGENTS.md rule 9):
|
||||||
|
- `uv run pytest` — unit + integration green.
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
|
||||||
|
- `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` — green **in isolation** (this phase's E2E).
|
||||||
|
- `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` — green (the real UI save/restore flow with in-cap payloads — the no-regression proof for the surface the caps sit on).
|
||||||
|
- `uv run ruff check . && uv run pyright` — clean.
|
||||||
|
2. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/schemas.py`, `tests/unit/test_schemas.py`, `tests/integration/test_chats_api.py` (+ the chat-API file if the A3 pin landed there), `tests/e2e/test_chat_save_payload_limits.py`, and the phase files:
|
||||||
|
`fix(api): bound anonymous saved-chat payload sizes at the schema boundary`
|
||||||
|
— body: security audit SEC-05 (2026-09-07) — `POST/PUT /api/chats` are public (phase 55) and every `ChatMessage`/list field was unbounded, so an anonymous caller could store arbitrarily large JSONB rows (memory + storage DoS). Pydantic caps at the boundary (text/thinking 32 000 mirroring `HistoryTurn`, `SourceRef` mirroring the `documents` column lengths, list `max_items`, `messages` ≤ 200) → 422 on overflow, nothing stored; the 201/404/share contract and the stored JSONB shape are unchanged for in-cap payloads (round-trip pin + real-UI E2E regression). Boundary-only: no route, migration, or frontend change.
|
||||||
|
3. Move the phase directory: `mv .agents/phases/todo/83_chat_save_payload_limits .agents/phases/complete/` and include the move in the same commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- This task IS the phase-level gate — the commands above are the completion evidence.
|
||||||
|
- Coverage: >90% held.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest` green; coverage >90%; both E2E suites green (the dedicated one in isolation, `test_chat_history.py` as the regression).
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
|
- [ ] Exactly one new commit; `git show --stat HEAD` lists the files above + the phase files (todo → complete move) — in particular NO `app/api/chats.py`, NO `alembic/`, NO `frontend/`, NO `pyproject.toml`/`uv.lock`.
|
||||||
|
- [ ] `.agents/phases/complete/83_chat_save_payload_limits/` exists; `todo/` no longer contains it.
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Phase 84 — Mask credentials in docs-push and LLM error surfaces
|
||||||
|
|
||||||
|
**Source:** `.agents/remediation_plan.md` SEC-08 (security audit 2026-09-07, severity Medium — "Docs-push 502 surfaces git stderr verbatim (possible credential echo)"), with the SEC-13 fold-in (Low — "LLM error strings echo `llm_base_url`")
|
||||||
|
**Story:** n/a (security hardening — audit-derived, no user story)
|
||||||
|
**Context:** `app/api/sync.py` — the existing masker: `_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")` + `_sanitize_error(message)` (masks `user:pass@` in any error text; the sync failure detail runs through it — "no secrets in the UI"). `app/api/git_sources.py` — imports it (`from app.api.sync import _sanitize_error`) for upload-failure details. `app/api/doc_drafts.py::push_doc_draft` — the GAP: `except DocsPushError as exc: raise HTTPException(502, detail=str(exc))` — git's stderr verbatim (a `BOR_DOCS_REPO` URL with embedded credentials would reach the browser + server log on a failed push). `app/core/docs_push.py` — `push_document` raises `DocsPushError(str(err))` carrying `GitSyncError`'s "git … failed (exit N): <stderr>" text. `scripts/git_sync.py::run_git` — the single git invocation point (list args, no shell — the error text is git's own stderr). `app/rag/llm.py` — SEC-13: five error f-strings embed `self.settings.llm_base_url` raw (lines 288, 327, 331, 337, 486 — embed + chat + stream failure paths) — a base URL configured with embedded credentials would echo into SSE `error` frames and logs. `tests/integration/test_doc_drafts_api.py` (the existing 502-contract pins — git stderr in the detail) and `tests/unit/test_llm_client.py` (the error-string pins) are the regression anchors.
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
No error surface ever ships a `user:pass@` credential: the sync masker moves to a shared core module, the docs-push 502 detail runs through it (SEC-08), and the LLM error construction sanitizes the base URL (SEC-13) — while every non-credential error string (the sync/git/LLM tests' pinned copies) stays byte-identical.
|
||||||
|
|
||||||
|
## Audit basis (read this, not the chat)
|
||||||
|
- SEC-08: `BOR_DOCS_REPO` is documented as "any remote (URL or local path)" — an `https://user:token@host/…` URL is a normal config shape. On a failed push (revoked token, network), git's stderr echoes the remote URL; `push_doc_draft` returns it verbatim as the 502 `detail` → the credential lands in the browser (admin screen) and in the server log (`logger` writes the 502? the detail is the response body — and `run_git`'s stderr also surfaces into any traceback logging). The sync path already solved this exact problem (`_sanitize_error`) — the docs-push path simply never got the treatment.
|
||||||
|
- SEC-13 (fold-in): the five `llm.py` error f-strings interpolate `llm_base_url` raw — same class of leak on the LLM side (the real deployment uses a bare URL + header key, so today it is latent).
|
||||||
|
- The masker is deliberately a NARROW regex (only the `user:pass@` userinfo shape — git/HTTP convention): it must not rewrite ordinary text (`a: b @ c` without the userinfo run, plain hosts, emails in prose stay untouched as long as they don't match the userinfo pattern — the existing sync tests pin the behavior; the move must be byte-identical).
|
||||||
|
|
||||||
|
## Owner decisions (chat, 2026-09-07 — recorded per AGENTS.md rule 3)
|
||||||
|
- **A1 — shared core module:** `app/core/errors.py` (new) owns `sanitize_error(message: str) -> str` — the regex + logic move VERBATIM from `app/api/sync.py` (same `_CREDS_RE` pattern, same `sub("*****@", …)` replacement — byte-identical behavior). `app/api/sync.py` keeps the private name as an alias (`from app.core.errors import sanitize_error as _sanitize_error`) so `app/api/git_sources.py`'s existing import and every sync test stay untouched (zero caller diff outside the two target surfaces).
|
||||||
|
- **A2 — SEC-08 application point:** `app/api/doc_drafts.py::push_doc_draft` — the `except DocsPushError` clause becomes `raise HTTPException(502, detail=sanitize_error(str(exc))) from None` (the ONLY line that changes there).
|
||||||
|
- **A3 — SEC-13 application point:** `app/rag/llm.py` — the five error f-strings sanitize the URL at construction: a module-level `self._base = sanitize_error(settings.llm_base_url)` is NOT introduced (the client is constructed with settings; simpler and more local: each f-string uses `sanitize_error(self.settings.llm_base_url)`). With a credential-free URL (every real deployment) `sanitize_error` is a no-op → the existing error-string pins in `tests/unit/test_llm_client.py` stay byte-identical green.
|
||||||
|
- **A4 — no other surfaces in scope:** the sync/git-sources upload details already sanitize; the chat SSE error frames carry fixed operator copy (not git/URL text) — untouched.
|
||||||
|
|
||||||
|
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||||
|
- **`app/core/errors.py` (new):**
|
||||||
|
```python
|
||||||
|
_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")
|
||||||
|
|
||||||
|
def sanitize_error(message: str) -> str:
|
||||||
|
"""Mask user:pass@ userinfo in an error string (no secrets in the UI/logs)."""
|
||||||
|
return _CREDS_RE.sub("*****@", message)
|
||||||
|
```
|
||||||
|
Module docstring: the audit basis (SEC-08/SEC-13), the "narrow userinfo-regex only" contract (byte-identical for credential-free text — the sync tests are the proof), the replacement shape (`*****@` — the existing sync copy).
|
||||||
|
- **`app/api/sync.py`** — delete the local `_CREDS_RE` + `_sanitize_error` definitions; add `from app.core.errors import sanitize_error as _sanitize_error` (the `_run_sync` failure path and its tests see the identical function under the identical private name).
|
||||||
|
- **`app/api/doc_drafts.py`** — import `sanitize_error` from `app.core.errors`; the `push_doc_draft` except clause per A2; the route docstring's outcome-4 line updated (`502 with the SANITIZED git stderr in the detail — credential userinfo masked, the phase-59 `GitSyncError → detail` mapping kept`).
|
||||||
|
- **`app/rag/llm.py`** — import `sanitize_error`; the five f-strings (lines 288/327/331/337/486 today) wrap the URL: `f"embeddings request to {sanitize_error(self.settings.llm_base_url)} failed: {e}"` etc. (the message copy around it unchanged).
|
||||||
|
- **Not touched:** `scripts/git_sync.py` (the stderr source — unchanged), `app/core/docs_push.py` (raises the raw text — the SANITIZE point is the API boundary, where the response is built), the sync/upload failure paths' behavior (byte-identical via the alias).
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
— (none; standalone error-surface hardening — builds on the completed phase-32 sync sanitizer and phase-59 docs-push; no behavior change for credential-free errors)
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
1. `01_shared_sanitizer.md` — `app/core/errors.py` + the sync alias refactor + unit suite.
|
||||||
|
2. `02_apply_docs_push_and_llm.md` — the docs-push 502 + the five LLM error sites + the integration/unit pins.
|
||||||
|
3. `03_verify_and_commit.md` — full gate + atomic commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- Unit — `tests/unit/test_error_sanitization.py` (new): the exact existing sync behavior is pinned — `https://user:pass@host/x` → `https://*****@host/x`; multiple userinfo occurrences all masked; an ssh-style `git:token@host` userinfo masked; credential-free text (a git failure line with a bare `https://github.com/owner/repo.git` URL, a plain error sentence, an email-shaped `a@b.c` — no userinfo run) **byte-identical**; idempotent (sanitize(sanitize(x)) == sanitize(x)).
|
||||||
|
- Unit — `tests/unit/test_llm_client.py` (extend): with `llm_base_url="https://svc:topsecret@llm.local/v1"`, a forced transport failure on the embed path → the `EmbeddingError` message contains `https://*****@llm.local/v1` and NOT `topsecret` (one representative path — the embed failure; the other four sites share the same construction, pinned by the same test style if cheap); the credential-free base-URL error pins stay byte-identical (existing tests green).
|
||||||
|
- Integration — `tests/integration/test_doc_drafts_api.py` (extend): monkeypatch `app.core.docs_push.push_document` (or the doc_drafts module's reference) to raise `DocsPushError("git push failed (exit 128): fatal: Authentication failed for 'https://bot:ghp_LEAK@github.com/owner/docs.git/'")` → `POST /api/doc-drafts/{token}/push` → **502** whose `detail` contains `*****@github.com` and NOT `ghp_LEAK` (and the repo/exit/`fatal:` context still readable); the existing 502 pins (plain stderr detail) stay byte-identical green.
|
||||||
|
- Regression: `tests/integration/test_git_sources_upload.py` + the sync-status tests (the alias refactor's proof — the `_sanitize_error` import in `git_sources.py` still works; the sync `failed`-state error is masked exactly as before).
|
||||||
|
- E2E (isolation gate per AGENTS.md rule 9): `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green — no UI change (the doc-edit screen renders whatever detail string the API returns; the shape is unchanged).
|
||||||
|
- Coverage: **>90%** on `app/` — the new module is two lines + fully unit-pinned; `app/rag/llm.py`'s changed lines hit by the new unit tests + the existing error-path tests.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] A docs-push failure with userinfo in git's stderr → 502 detail shows `*****@` and never the token (integration pin); the row is untouched (status/branch/sha as found — the existing pin).
|
||||||
|
- [ ] A LLM transport failure with a userinfo-bearing `llm_base_url` → the error message is masked (unit pin); credential-free error strings byte-identical (existing pins green).
|
||||||
|
- [ ] `app/api/sync.py` no longer defines its own `_CREDS_RE`/`_sanitize_error` (the alias is the import); `git grep "_CREDS_RE" app/` shows only `app/core/errors.py`; the sync + git-sources upload suites green untouched.
|
||||||
|
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_smoke.py -v --no-cov` green in isolation; `uv run ruff check . && uv run pyright` clean.
|
||||||
|
- [ ] `git diff --stat` limited to `app/core/errors.py`, `app/api/sync.py`, `app/api/doc_drafts.py`, `app/rag/llm.py`, the three test files, phase files.
|
||||||
|
- [ ] One atomic `--no-gpg-sign` commit (e.g. `fix(security): mask credentials in docs-push and LLM error surfaces`); phase dir moved to `.agents/phases/complete/`.
|
||||||
|
|
||||||
|
## Locked decisions
|
||||||
|
- **Sanitize at the API boundary** (A2) — `DocsPushError` keeps carrying the full stderr (logs/inspection value); the RESPONSE is where the secret would leak, so that is where it is masked.
|
||||||
|
- **Byte-identical for credential-free text** (A1/A3) — the narrow-regex contract; the existing sync/LLM error pins are the tripwire, and they must pass unchanged.
|
||||||
|
- **No logging changes** — this phase changes what responses carry; the server-log side of SEC-08/SEC-13 is improved as a consequence (the 502 detail is the logged-adjacent surface) but no log-format change is in scope.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Task 01 — The shared sanitizer module + sync alias refactor + unit suite
|
||||||
|
|
||||||
|
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
`app/core/errors.py` owns `sanitize_error` (verbatim behavior of the sync masker), `app/api/sync.py` becomes a thin alias caller (zero behavior change), and `tests/unit/test_error_sanitization.py` pins the masking + the byte-identical-for-plain-text contract.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/core/errors.py` (new) — per the phase overview's design block: `_CREDS_RE` (the exact pattern from `app/api/sync.py`) + `sanitize_error(message: str) -> str` (`_CREDS_RE.sub("*****@", message)`) + the module docstring content (audit basis SEC-08/SEC-13, the narrow-userinfo contract, the `*****@` replacement shape — the sync copy).
|
||||||
|
2. `app/api/sync.py` — remove the local `_CREDS_RE` definition and the `_sanitize_error` function body; add `from app.core.errors import sanitize_error as _sanitize_error`; the `_run_sync` failure path (`_status.error = _sanitize_error(str(e))`) and everything else in the file stays byte-identical (the private name keeps working — `app/api/git_sources.py`'s `from app.api.sync import _sanitize_error` continues to import the same function through the alias).
|
||||||
|
- Update the `app/api/sync.py` module docstring's mention of the masker (one line: it now lives in `app/core/errors.py`, imported under the private name).
|
||||||
|
3. `tests/unit/test_error_sanitization.py` (new):
|
||||||
|
- `https://user:pass@host/repo.git` → `https://*****@host/repo.git`;
|
||||||
|
- two userinfo occurrences in one string → both masked;
|
||||||
|
- `git push failed (exit 128): fatal: Authentication failed for 'https://bot:tok@github.com/o/r.git/'` → the token masked, `fatal:`/`exit 128`/host intact;
|
||||||
|
- byte-identical cases (the contract): a plain git error with a bare `https://github.com/owner/repo.git` (no userinfo), a sentence with a colon + space, an email `owner@example.com` (no userinfo run before the `@`… assert exactly what the regex does — if the pattern masks it, pin THAT and note it; the point is determinism, not guessing), the empty string;
|
||||||
|
- idempotence: `sanitize_error(sanitize_error(x)) == sanitize_error(x)` for the masked cases.
|
||||||
|
4. Run the sync-related suites to prove the alias refactor: `uv run pytest tests/integration/test_git_sources_upload.py -q` + any existing sync-status integration test file (`tests/integration/` — locate the sync suite, e.g. via `rg -l "sync" tests/integration/`) → all green untouched.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/unit/test_error_sanitization.py -v` green.
|
||||||
|
- The sync/upload regression suites green (the alias is behavior-identical).
|
||||||
|
- Coverage: **>90%** on `app/core/errors.py` (trivial — both branches of the regex hit).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `git grep -n "_CREDS_RE" app/` shows exactly one definition — `app/core/errors.py` — and `app/api/sync.py` imports under the private alias.
|
||||||
|
- [ ] `uv run pytest tests/unit/test_error_sanitization.py -v` green.
|
||||||
|
- [ ] The sync/upload integration suites green with NO test edits.
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Task 02 — Apply the sanitizer to the docs-push 502 and the LLM error sites
|
||||||
|
|
||||||
|
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08 + SEC-13)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
The two leaking surfaces are closed: `push_doc_draft`'s 502 detail is sanitized (SEC-08), and the five `llm.py` error f-strings sanitize the base URL (SEC-13) — with integration/unit pins for both and the credential-free error strings byte-identical.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. `app/api/doc_drafts.py` — import `from app.core.errors import sanitize_error`; in `push_doc_draft`'s `except DocsPushError as exc:` clause change the raise to `raise HTTPException(status_code=502, detail=sanitize_error(str(exc))) from None`; update the route docstring's outcome-4 line (502 detail = the SANITIZED git stderr — userinfo masked, the `GitSyncError → detail` mapping otherwise kept) and the module docstring's one-line error-contract sentence if it names the raw stderr.
|
||||||
|
2. `tests/integration/test_doc_drafts_api.py` — extend (the existing 502 pins stay green):
|
||||||
|
- a draft with `status="draft"` + `docs_configured` (the file's existing configured-settings pattern — monkeypatch `get_settings` or use the fixture's env);
|
||||||
|
- monkeypatch the module-level `push_document` reference in `app.api.doc_drafts` to `raise DocsPushError("git push origin bor-docs failed (exit 128): fatal: Authentication failed for 'https://bot:ghp_LEAKTOKEN@github.com/owner/docs.git/'")`;
|
||||||
|
- `POST /api/doc-drafts/{token}/push` → **502**; the `detail` contains `*****@github.com` and `exit 128` and `fatal: Authentication failed` but NOT `ghp_LEAKTOKEN`;
|
||||||
|
- the row is untouched (status/branch/commit_sha as found — the existing pin style).
|
||||||
|
3. `app/rag/llm.py` — import `sanitize_error`; wrap the base URL in the five error f-strings (the lines building `EmbeddingError`/`LLMError` with `…to/from {self.settings.llm_base_url}…`): each becomes `{sanitize_error(self.settings.llm_base_url)}`; the surrounding copy is byte-identical.
|
||||||
|
4. `tests/unit/test_llm_client.py` — extend:
|
||||||
|
- a client built with `llm_base_url="https://svc:topsecret@llm.local/v1"` (the file's existing fake-settings pattern) + a forced transport failure on the EMBED path → the raised `EmbeddingError`'s message contains `https://*****@llm.local/v1` and NOT `topsecret`;
|
||||||
|
- if the file's structure makes it cheap, the same for one of the chat paths (the `chat_stream`/`chat` failure f-string) — otherwise the embed path alone plus a comment (the five sites share the construction; the byte-identical regression is the existing pins);
|
||||||
|
- the existing credential-free error-string pins (bare `https://aipi.example/v1`-style base URLs) stay byte-identical green.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- `uv run pytest tests/integration/test_doc_drafts_api.py -v` green (new + existing).
|
||||||
|
- `uv run pytest tests/unit/test_llm_client.py -v` green (new + existing — the byte-identical proof).
|
||||||
|
- Coverage: **>90%** on `app/` — the changed lines in `doc_drafts.py`/`llm.py` are hit by the new pins; the other four LLM sites share the identical construction (the existing error-path tests still exercise them).
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] The 502 pin passes: `*****@github.com` present, `ghp_LEAKTOKEN` absent, context readable; the row untouched.
|
||||||
|
- [ ] The LLM unit pin passes: masked URL in the error message, `topsecret` absent; the pre-existing error-string tests green unchanged.
|
||||||
|
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# Task 03 — Full gate + atomic commit
|
||||||
|
|
||||||
|
**Phase:** `84_docs_push_error_sanitization` · **Story:** n/a (security hardening — audit SEC-08 + SEC-13)
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Run the complete phase gate, land the phase as one atomic commit, and move the phase directory to `complete/`.
|
||||||
|
|
||||||
|
## Work
|
||||||
|
1. **Full regression gate** (AGENTS.md rule 9):
|
||||||
|
- `uv run pytest` — unit + integration green (the sync/upload + doc-drafts + LLM suites are the regression anchors for the alias refactor and the byte-identical contract).
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**.
|
||||||
|
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` — green **in isolation** (this phase's E2E contract: no UI change — the doc-edit screen renders the detail string it always rendered; the shape is unchanged, only the userinfo run is masked).
|
||||||
|
- `uv run ruff check . && uv run pyright` — clean.
|
||||||
|
2. **Commit** (AGENTS.md rule 8 — one atomic, Conventional-Commits commit, always `--no-gpg-sign`), staging `app/core/errors.py`, `app/api/sync.py`, `app/api/doc_drafts.py`, `app/rag/llm.py`, `tests/unit/test_error_sanitization.py`, `tests/unit/test_llm_client.py`, `tests/integration/test_doc_drafts_api.py`, and the phase files:
|
||||||
|
`fix(security): mask credentials in docs-push and LLM error surfaces`
|
||||||
|
— body: security audit SEC-08 + SEC-13 (2026-09-07) — a docs-push failure returned git's stderr verbatim as the 502 detail (a `BOR_DOCS_REPO` URL with embedded credentials would reach the browser + logs), and the LLM error f-strings echoed `llm_base_url` raw; the sync-phase userinfo masker (`user:pass@` → `*****@`) now lives in `app/core/errors.py`, the docs-push 502 and the five LLM error sites run through it, and the sync import keeps working via a private alias. Credential-free error strings are byte-identical (existing pins green).
|
||||||
|
3. Move the phase directory: `mv .agents/phases/todo/84_docs_push_error_sanitization .agents/phases/complete/` and include the move in the same commit.
|
||||||
|
|
||||||
|
## Testing & Quality
|
||||||
|
- This task IS the phase-level gate — the commands above are the completion evidence.
|
||||||
|
- Coverage: >90% held.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
- [ ] `uv run pytest` green (all regression anchors included); coverage >90%; `tests/e2e/test_smoke.py` green in isolation; ruff + pyright clean.
|
||||||
|
- [ ] Exactly one new commit; `git show --stat HEAD` lists the files above + the phase files (todo → complete move) — in particular NO `scripts/`, NO `app/core/docs_push.py`, NO `frontend/`, NO `pyproject.toml`/`uv.lock`.
|
||||||
|
- [ ] `.agents/phases/complete/84_docs_push_error_sanitization/` exists; `todo/` no longer contains it.
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
All gates verified. The phase is complete — everything was already implemented by prior task runs; this pass verified each completion criterion end-to-end and found no defects.
|
||||||
|
|
||||||
|
**Phase 81 final verification report** (tasks 01–03 all present in `complete/`; no code changes needed this pass)
|
||||||
|
|
||||||
|
- Verified: `app/core/rate_limit.py` (stdlib sliding window, fail-open) + both login routes' 429 pre-check/record/reset wiring + all unit/integration pins present (11th-429, blocked-success-still-429, clean-counter reset, shared counter, autouse `clean_rate_limit` fixture documented).
|
||||||
|
- Live check (task 03): dev server + 11 rapid wrong logins → `401 ×10, 429`; 429 body carries `retry-after: 900` + generic detail; server restarted (per-process counter cleared by design).
|
||||||
|
- `uv run pytest tests/unit/test_rate_limit.py -v --no-cov` → 10 passed
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed, **TOTAL 99%** (>90%; rate_limit.py 100%, auth.py 100%)
|
||||||
|
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` (isolation) → 3 passed
|
||||||
|
- `uv run ruff check . && uv run pyright` → All checks passed / 0 errors, 0 warnings
|
||||||
|
- Completion criteria: all met, except commit + phase-dir move — per harness rules I left all changes uncommitted in the working tree (harness commits atomically and moves the phase).
|
||||||
|
- Diff scope: exactly `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py` + phase files; `pyproject.toml` / `uv.lock` / `frontend/` untouched.
|
||||||
|
- Deviations: none in code; commit/move deferred to harness as instructed.
|
||||||
|
- Next pending phase: `82_security_headers`.
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
........................................................................ [ 4%]
|
||||||
|
........................................................................ [ 8%]
|
||||||
|
........................................................................ [ 13%]
|
||||||
|
........................................................................ [ 17%]
|
||||||
|
........................................................................ [ 21%]
|
||||||
|
........................................................................ [ 26%]
|
||||||
|
........................................................................ [ 30%]
|
||||||
|
........................................................................ [ 34%]
|
||||||
|
........................................................................ [ 39%]
|
||||||
|
........................................................................ [ 43%]
|
||||||
|
........................................................................ [ 47%]
|
||||||
|
........................................................................ [ 52%]
|
||||||
|
........................................................................ [ 56%]
|
||||||
|
........................................................................ [ 61%]
|
||||||
|
........................................................................ [ 65%]
|
||||||
|
........................................................................ [ 69%]
|
||||||
|
........................................................................ [ 74%]
|
||||||
|
........................................................................ [ 78%]
|
||||||
|
........................................................................ [ 82%]
|
||||||
|
........................................................................ [ 87%]
|
||||||
|
........................................................................ [ 91%]
|
||||||
|
........................................................................ [ 95%]
|
||||||
|
.................................................................... [100%]
|
||||||
|
=============================== warnings summary ===============================
|
||||||
|
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||||
|
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||||
|
from starlette.testclient import TestClient as TestClient # noqa
|
||||||
|
|
||||||
|
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||||
|
================================ tests coverage ================================
|
||||||
|
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||||
|
|
||||||
|
Name Stmts Miss Cover
|
||||||
|
-----------------------------------------------
|
||||||
|
app/__init__.py 1 0 100%
|
||||||
|
app/api/__init__.py 0 0 100%
|
||||||
|
app/api/auth.py 52 0 100%
|
||||||
|
app/api/chat.py 178 0 100%
|
||||||
|
app/api/chats.py 110 0 100%
|
||||||
|
app/api/config.py 7 0 100%
|
||||||
|
app/api/doc_drafts.py 93 0 100%
|
||||||
|
app/api/docs.py 50 0 100%
|
||||||
|
app/api/git_sources.py 212 0 100%
|
||||||
|
app/api/health.py 10 0 100%
|
||||||
|
app/api/steering.py 42 0 100%
|
||||||
|
app/api/suggestions.py 29 0 100%
|
||||||
|
app/api/sync.py 101 0 100%
|
||||||
|
app/api/tokens.py 28 0 100%
|
||||||
|
app/config.py 141 0 100%
|
||||||
|
app/core/__init__.py 0 0 100%
|
||||||
|
app/core/auth.py 45 0 100%
|
||||||
|
app/core/caching.py 108 0 100%
|
||||||
|
app/core/debugging.py 29 2 93%
|
||||||
|
app/core/docs_push.py 39 0 100%
|
||||||
|
app/core/logging.py 13 0 100%
|
||||||
|
app/core/rate_limit.py 44 0 100%
|
||||||
|
app/core/tokens.py 33 0 100%
|
||||||
|
app/db.py 21 0 100%
|
||||||
|
app/main.py 62 0 100%
|
||||||
|
app/models.py 94 0 100%
|
||||||
|
app/rag/__init__.py 0 0 100%
|
||||||
|
app/rag/agent.py 222 0 100%
|
||||||
|
app/rag/archive_upload.py 128 0 100%
|
||||||
|
app/rag/chunker.py 206 4 98%
|
||||||
|
app/rag/git_sources.py 14 0 100%
|
||||||
|
app/rag/importer.py 180 3 98%
|
||||||
|
app/rag/llm.py 216 0 100%
|
||||||
|
app/rag/overview.py 71 0 100%
|
||||||
|
app/rag/prompts.py 88 0 100%
|
||||||
|
app/rag/retriever.py 150 3 98%
|
||||||
|
app/rag/scaffolding.py 55 0 100%
|
||||||
|
app/rag/source_removal.py 41 0 100%
|
||||||
|
app/rag/sources_meta.py 16 0 100%
|
||||||
|
app/rag/suggestions.py 27 0 100%
|
||||||
|
app/rag/summarizer.py 24 0 100%
|
||||||
|
app/schemas.py 231 0 100%
|
||||||
|
-----------------------------------------------
|
||||||
|
TOTAL 3211 12 99%
|
||||||
|
coverage gate: app/ 99% (>90%) OK
|
||||||
|
All checks passed!
|
||||||
|
0 errors, 0 warnings, 0 informations
|
||||||
|
validation OK
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
Task 01 complete — all gates green.
|
||||||
|
|
||||||
|
**Implemented**
|
||||||
|
- `app/core/rate_limit.py` (new): stdlib-only sliding-window limiter — `MAX_FAILURES = 10` / `WINDOW_SECONDS = 900` module constants, per-IP `deque[float]` of `time.monotonic()` timestamps under a `threading.Lock`; `record_failure` (append + prune, never raises, drops the count silently on corrupt state), `remaining_wait` (0 when < 10 in-window; else `max(1, ceil(900 − (now − oldest)))`, never raises), `reset` (pop, no-op on unknown). Docstring carries the SEC-03 audit basis, shared-counter intent, fail-open rationale, and the `request.client.host` / no-proxy assumption.
|
||||||
|
- `tests/unit/test_rate_limit.py` (new): 10 tests pinning, in task order — 9 failures allowed → 10th blocks (`0 < wait ≤ 900`, int) → second IP independent → expiry via pre-seeded `WINDOW_SECONDS + 1` state AND a fake monotonic clock (whole-window wait from the oldest: 891 → 300 → clamped 1 → 0 the moment the oldest slides out) → `reset` unblocks → unknown IP 0 → fail-open on non-deque state and on a raising clock → `reset` no-op on unknown IP.
|
||||||
|
|
||||||
|
**Results**
|
||||||
|
- `uv run pytest tests/unit/test_rate_limit.py -v` → 10 passed
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` → 1648 passed; `app/core/rate_limit.py` **100%**, TOTAL **99%** (>90%)
|
||||||
|
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
|
||||||
|
- `git status`: only the two new files are code changes (pre-existing untracked phase/report files untouched); no tracked file modified
|
||||||
|
|
||||||
|
**Notable decisions**
|
||||||
|
- Corrupt state entry → `record_failure` is a literal no-op (entry left in place, treated as not-blocked) per the phase's "record_failure no-ops" wording; over-denial is the only forbidden outcome.
|
||||||
|
- Removed one unreachable defensive branch after the first coverage run to reach 100% on the module.
|
||||||
|
- `.agents/PLAN.md` is absent from the repo (AGENTS.md references it); the phase overview's design block was followed as the binding spec — no deviation.
|
||||||
|
|
||||||
|
**Next pending task:** `02_auth_routes_wiring.md` (429 pre-check + bookkeeping on both login routes + integration tests).
|
||||||
+82
@@ -0,0 +1,82 @@
|
|||||||
|
........................................................................ [ 4%]
|
||||||
|
........................................................................ [ 8%]
|
||||||
|
........................................................................ [ 13%]
|
||||||
|
........................................................................ [ 17%]
|
||||||
|
........................................................................ [ 21%]
|
||||||
|
........................................................................ [ 26%]
|
||||||
|
........................................................................ [ 30%]
|
||||||
|
........................................................................ [ 34%]
|
||||||
|
........................................................................ [ 39%]
|
||||||
|
........................................................................ [ 43%]
|
||||||
|
........................................................................ [ 48%]
|
||||||
|
........................................................................ [ 52%]
|
||||||
|
........................................................................ [ 56%]
|
||||||
|
........................................................................ [ 61%]
|
||||||
|
........................................................................ [ 65%]
|
||||||
|
........................................................................ [ 69%]
|
||||||
|
........................................................................ [ 74%]
|
||||||
|
........................................................................ [ 78%]
|
||||||
|
........................................................................ [ 83%]
|
||||||
|
........................................................................ [ 87%]
|
||||||
|
........................................................................ [ 91%]
|
||||||
|
........................................................................ [ 96%]
|
||||||
|
................................................................ [100%]
|
||||||
|
=============================== warnings summary ===============================
|
||||||
|
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||||
|
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||||
|
from starlette.testclient import TestClient as TestClient # noqa
|
||||||
|
|
||||||
|
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||||
|
================================ tests coverage ================================
|
||||||
|
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||||
|
|
||||||
|
Name Stmts Miss Cover
|
||||||
|
-----------------------------------------------
|
||||||
|
app/__init__.py 1 0 100%
|
||||||
|
app/api/__init__.py 0 0 100%
|
||||||
|
app/api/auth.py 40 0 100%
|
||||||
|
app/api/chat.py 178 0 100%
|
||||||
|
app/api/chats.py 110 0 100%
|
||||||
|
app/api/config.py 7 0 100%
|
||||||
|
app/api/doc_drafts.py 93 0 100%
|
||||||
|
app/api/docs.py 50 0 100%
|
||||||
|
app/api/git_sources.py 212 0 100%
|
||||||
|
app/api/health.py 10 0 100%
|
||||||
|
app/api/steering.py 42 0 100%
|
||||||
|
app/api/suggestions.py 29 0 100%
|
||||||
|
app/api/sync.py 101 0 100%
|
||||||
|
app/api/tokens.py 28 0 100%
|
||||||
|
app/config.py 141 0 100%
|
||||||
|
app/core/__init__.py 0 0 100%
|
||||||
|
app/core/auth.py 45 0 100%
|
||||||
|
app/core/caching.py 108 0 100%
|
||||||
|
app/core/debugging.py 29 2 93%
|
||||||
|
app/core/docs_push.py 39 0 100%
|
||||||
|
app/core/logging.py 13 0 100%
|
||||||
|
app/core/rate_limit.py 44 0 100%
|
||||||
|
app/core/tokens.py 33 0 100%
|
||||||
|
app/db.py 21 0 100%
|
||||||
|
app/main.py 62 0 100%
|
||||||
|
app/models.py 94 0 100%
|
||||||
|
app/rag/__init__.py 0 0 100%
|
||||||
|
app/rag/agent.py 222 0 100%
|
||||||
|
app/rag/archive_upload.py 128 0 100%
|
||||||
|
app/rag/chunker.py 206 4 98%
|
||||||
|
app/rag/git_sources.py 14 0 100%
|
||||||
|
app/rag/importer.py 180 3 98%
|
||||||
|
app/rag/llm.py 216 0 100%
|
||||||
|
app/rag/overview.py 71 0 100%
|
||||||
|
app/rag/prompts.py 88 0 100%
|
||||||
|
app/rag/retriever.py 150 3 98%
|
||||||
|
app/rag/scaffolding.py 55 0 100%
|
||||||
|
app/rag/source_removal.py 41 0 100%
|
||||||
|
app/rag/sources_meta.py 16 0 100%
|
||||||
|
app/rag/suggestions.py 27 0 100%
|
||||||
|
app/rag/summarizer.py 24 0 100%
|
||||||
|
app/schemas.py 231 0 100%
|
||||||
|
-----------------------------------------------
|
||||||
|
TOTAL 3199 12 99%
|
||||||
|
coverage gate: app/ 99% (>90%) OK
|
||||||
|
All checks passed!
|
||||||
|
0 errors, 0 warnings, 0 informations
|
||||||
|
validation OK
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
**Task 02 complete — report**
|
||||||
|
|
||||||
|
**Implemented**
|
||||||
|
- `app/api/auth.py`: `TOO_MANY_DETAIL` constant + `rate_limit` import; on both `login` and `token_auth` — 429 pre-check (`Retry-After`, generic detail) at handler top, `record_failure` before the 401, `reset` before the 204 (shared per-IP counter); module + handler docstrings extended with the 429 case.
|
||||||
|
- `tests/integration/test_auth_api.py`: autouse `clean_rate_limit` fixture (`rate_limit.reset("testclient")` around every test — documented: TestClient's host is a fixed shared IP, counter is process state) + 4 new pins: 11th failed login → 429 + detail + `Retry-After` (≤ window); correct password while blocked → still 429; 9 failures + success → counter reset (next 9 all 401); 5 login + 5 token failures → 11th (valid token) → 429.
|
||||||
|
|
||||||
|
**Results**
|
||||||
|
- `uv run pytest tests/integration/test_auth_api.py tests/unit/test_rate_limit.py` → 27 passed
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed; **99%** total (`app/api/auth.py` 100%, `app/core/rate_limit.py` 100%)
|
||||||
|
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (isolation)
|
||||||
|
- `uv run ruff check . && uv run pyright` → clean (0 errors)
|
||||||
|
- `pyproject.toml`/`uv.lock` untouched; diff limited to `app/api/auth.py` + the two test files (plus task-01's two new files, untracked for the harness commit)
|
||||||
|
|
||||||
|
**Deviation**: used `request.client.host if request.client is not None else "unknown"` instead of the task's literal `or "unknown"` — starlette types `client` as optional; pyright (non-negotiable gate) requires the explicit check. Same fail-open semantics.
|
||||||
|
|
||||||
|
**Next pending task**: `.agents/phases/todo/81_login_rate_limit/03_verify_and_commit.md`
|
||||||
+82
@@ -0,0 +1,82 @@
|
|||||||
|
........................................................................ [ 4%]
|
||||||
|
........................................................................ [ 8%]
|
||||||
|
........................................................................ [ 13%]
|
||||||
|
........................................................................ [ 17%]
|
||||||
|
........................................................................ [ 21%]
|
||||||
|
........................................................................ [ 26%]
|
||||||
|
........................................................................ [ 30%]
|
||||||
|
........................................................................ [ 34%]
|
||||||
|
........................................................................ [ 39%]
|
||||||
|
........................................................................ [ 43%]
|
||||||
|
........................................................................ [ 47%]
|
||||||
|
........................................................................ [ 52%]
|
||||||
|
........................................................................ [ 56%]
|
||||||
|
........................................................................ [ 61%]
|
||||||
|
........................................................................ [ 65%]
|
||||||
|
........................................................................ [ 69%]
|
||||||
|
........................................................................ [ 74%]
|
||||||
|
........................................................................ [ 78%]
|
||||||
|
........................................................................ [ 82%]
|
||||||
|
........................................................................ [ 87%]
|
||||||
|
........................................................................ [ 91%]
|
||||||
|
........................................................................ [ 95%]
|
||||||
|
.................................................................... [100%]
|
||||||
|
=============================== warnings summary ===============================
|
||||||
|
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||||
|
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||||
|
from starlette.testclient import TestClient as TestClient # noqa
|
||||||
|
|
||||||
|
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||||
|
================================ tests coverage ================================
|
||||||
|
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||||
|
|
||||||
|
Name Stmts Miss Cover
|
||||||
|
-----------------------------------------------
|
||||||
|
app/__init__.py 1 0 100%
|
||||||
|
app/api/__init__.py 0 0 100%
|
||||||
|
app/api/auth.py 52 0 100%
|
||||||
|
app/api/chat.py 178 0 100%
|
||||||
|
app/api/chats.py 110 0 100%
|
||||||
|
app/api/config.py 7 0 100%
|
||||||
|
app/api/doc_drafts.py 93 0 100%
|
||||||
|
app/api/docs.py 50 0 100%
|
||||||
|
app/api/git_sources.py 212 0 100%
|
||||||
|
app/api/health.py 10 0 100%
|
||||||
|
app/api/steering.py 42 0 100%
|
||||||
|
app/api/suggestions.py 29 0 100%
|
||||||
|
app/api/sync.py 101 0 100%
|
||||||
|
app/api/tokens.py 28 0 100%
|
||||||
|
app/config.py 141 0 100%
|
||||||
|
app/core/__init__.py 0 0 100%
|
||||||
|
app/core/auth.py 45 0 100%
|
||||||
|
app/core/caching.py 108 0 100%
|
||||||
|
app/core/debugging.py 29 2 93%
|
||||||
|
app/core/docs_push.py 39 0 100%
|
||||||
|
app/core/logging.py 13 0 100%
|
||||||
|
app/core/rate_limit.py 44 0 100%
|
||||||
|
app/core/tokens.py 33 0 100%
|
||||||
|
app/db.py 21 0 100%
|
||||||
|
app/main.py 62 0 100%
|
||||||
|
app/models.py 94 0 100%
|
||||||
|
app/rag/__init__.py 0 0 100%
|
||||||
|
app/rag/agent.py 222 0 100%
|
||||||
|
app/rag/archive_upload.py 128 0 100%
|
||||||
|
app/rag/chunker.py 206 4 98%
|
||||||
|
app/rag/git_sources.py 14 0 100%
|
||||||
|
app/rag/importer.py 180 3 98%
|
||||||
|
app/rag/llm.py 216 0 100%
|
||||||
|
app/rag/overview.py 71 0 100%
|
||||||
|
app/rag/prompts.py 88 0 100%
|
||||||
|
app/rag/retriever.py 150 3 98%
|
||||||
|
app/rag/scaffolding.py 55 0 100%
|
||||||
|
app/rag/source_removal.py 41 0 100%
|
||||||
|
app/rag/sources_meta.py 16 0 100%
|
||||||
|
app/rag/suggestions.py 27 0 100%
|
||||||
|
app/rag/summarizer.py 24 0 100%
|
||||||
|
app/schemas.py 231 0 100%
|
||||||
|
-----------------------------------------------
|
||||||
|
TOTAL 3211 12 99%
|
||||||
|
coverage gate: app/ 99% (>90%) OK
|
||||||
|
All checks passed!
|
||||||
|
0 errors, 0 warnings, 0 informations
|
||||||
|
validation OK
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
**Task 03 (phase-level gate) complete — report**
|
||||||
|
|
||||||
|
**Gate results (all run in this session):**
|
||||||
|
- `uv run pytest` → 1652 passed
|
||||||
|
- `uv run pytest --cov=app --cov-report=term-missing` → 99% total; `app/api/auth.py` 100%, `app/core/rate_limit.py` 100%
|
||||||
|
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (in isolation, server on :8123)
|
||||||
|
- `uv run ruff check . && uv run pyright` → all checks passed; 0 errors
|
||||||
|
- Live check (fresh `uv run uvicorn app.main:app` on :8000): 11 rapid wrong logins → `401 ×10, 429`; the 429 carries `retry-after: 900` + generic detail; correct password while blocked → 429; kill + restart → counter cleared (wrong login → 401). Server stopped, port free.
|
||||||
|
- `git diff` scope: only `app/api/auth.py`, `tests/integration/test_auth_api.py` (modified) + `app/core/rate_limit.py`, `tests/unit/test_rate_limit.py` (new); `pyproject.toml`/`uv.lock`/`frontend/` untouched (0 diff lines).
|
||||||
|
|
||||||
|
**Notable decisions:**
|
||||||
|
- Per harness rules (which override the task file), I did **not** run `git add`/`git commit` nor move the phase dir to `complete/` — the harness makes the atomic commit and the move. All changes left in the working tree.
|
||||||
|
- No defects found in tasks 01/02; one operational slip of my own (first restart check killed the wrong pid, so a stale blocked server answered) — redone cleanly with a full process kill; final results above are from a verified-fresh server.
|
||||||
|
|
||||||
|
**Next pending task:** `.agents/phases/todo/82_security_headers/01_headers_middleware.md` (phase 81 is fully gated — harness commit + move remains).
|
||||||
+82
@@ -0,0 +1,82 @@
|
|||||||
|
........................................................................ [ 4%]
|
||||||
|
........................................................................ [ 8%]
|
||||||
|
........................................................................ [ 13%]
|
||||||
|
........................................................................ [ 17%]
|
||||||
|
........................................................................ [ 21%]
|
||||||
|
........................................................................ [ 26%]
|
||||||
|
........................................................................ [ 30%]
|
||||||
|
........................................................................ [ 34%]
|
||||||
|
........................................................................ [ 39%]
|
||||||
|
........................................................................ [ 43%]
|
||||||
|
........................................................................ [ 47%]
|
||||||
|
........................................................................ [ 52%]
|
||||||
|
........................................................................ [ 56%]
|
||||||
|
........................................................................ [ 61%]
|
||||||
|
........................................................................ [ 65%]
|
||||||
|
........................................................................ [ 69%]
|
||||||
|
........................................................................ [ 74%]
|
||||||
|
........................................................................ [ 78%]
|
||||||
|
........................................................................ [ 82%]
|
||||||
|
........................................................................ [ 87%]
|
||||||
|
........................................................................ [ 91%]
|
||||||
|
........................................................................ [ 95%]
|
||||||
|
.................................................................... [100%]
|
||||||
|
=============================== warnings summary ===============================
|
||||||
|
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||||
|
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||||
|
from starlette.testclient import TestClient as TestClient # noqa
|
||||||
|
|
||||||
|
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||||
|
================================ tests coverage ================================
|
||||||
|
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||||
|
|
||||||
|
Name Stmts Miss Cover
|
||||||
|
-----------------------------------------------
|
||||||
|
app/__init__.py 1 0 100%
|
||||||
|
app/api/__init__.py 0 0 100%
|
||||||
|
app/api/auth.py 52 0 100%
|
||||||
|
app/api/chat.py 178 0 100%
|
||||||
|
app/api/chats.py 110 0 100%
|
||||||
|
app/api/config.py 7 0 100%
|
||||||
|
app/api/doc_drafts.py 93 0 100%
|
||||||
|
app/api/docs.py 50 0 100%
|
||||||
|
app/api/git_sources.py 212 0 100%
|
||||||
|
app/api/health.py 10 0 100%
|
||||||
|
app/api/steering.py 42 0 100%
|
||||||
|
app/api/suggestions.py 29 0 100%
|
||||||
|
app/api/sync.py 101 0 100%
|
||||||
|
app/api/tokens.py 28 0 100%
|
||||||
|
app/config.py 141 0 100%
|
||||||
|
app/core/__init__.py 0 0 100%
|
||||||
|
app/core/auth.py 45 0 100%
|
||||||
|
app/core/caching.py 108 0 100%
|
||||||
|
app/core/debugging.py 29 2 93%
|
||||||
|
app/core/docs_push.py 39 0 100%
|
||||||
|
app/core/logging.py 13 0 100%
|
||||||
|
app/core/rate_limit.py 44 0 100%
|
||||||
|
app/core/tokens.py 33 0 100%
|
||||||
|
app/db.py 21 0 100%
|
||||||
|
app/main.py 62 0 100%
|
||||||
|
app/models.py 94 0 100%
|
||||||
|
app/rag/__init__.py 0 0 100%
|
||||||
|
app/rag/agent.py 222 0 100%
|
||||||
|
app/rag/archive_upload.py 128 0 100%
|
||||||
|
app/rag/chunker.py 206 4 98%
|
||||||
|
app/rag/git_sources.py 14 0 100%
|
||||||
|
app/rag/importer.py 180 3 98%
|
||||||
|
app/rag/llm.py 216 0 100%
|
||||||
|
app/rag/overview.py 71 0 100%
|
||||||
|
app/rag/prompts.py 88 0 100%
|
||||||
|
app/rag/retriever.py 150 3 98%
|
||||||
|
app/rag/scaffolding.py 55 0 100%
|
||||||
|
app/rag/source_removal.py 41 0 100%
|
||||||
|
app/rag/sources_meta.py 16 0 100%
|
||||||
|
app/rag/suggestions.py 27 0 100%
|
||||||
|
app/rag/summarizer.py 24 0 100%
|
||||||
|
app/schemas.py 231 0 100%
|
||||||
|
-----------------------------------------------
|
||||||
|
TOTAL 3211 12 99%
|
||||||
|
coverage gate: app/ 99% (>90%) OK
|
||||||
|
All checks passed!
|
||||||
|
0 errors, 0 warnings, 0 informations
|
||||||
|
validation OK
|
||||||
+41
-2
@@ -2,11 +2,16 @@
|
|||||||
|
|
||||||
* ``POST /api/login`` — 204 + signed session cookie on success; 401
|
* ``POST /api/login`` — 204 + signed session cookie on success; 401
|
||||||
``invalid password`` on any mismatch (constant-time, one generic
|
``invalid password`` on any mismatch (constant-time, one generic
|
||||||
message, no session set).
|
message, no session set); while the per-IP failure window is exhausted
|
||||||
|
→ 429 + ``Retry-After``, one generic detail — audit SEC-03.
|
||||||
* ``POST /api/token-auth`` — 204 + signed session cookie for a valid,
|
* ``POST /api/token-auth`` — 204 + signed session cookie for a valid,
|
||||||
unrevoked API token (PUBLIC — it is the token holders' login route);
|
unrevoked API token (PUBLIC — it is the token holders' login route);
|
||||||
every failure shape (malformed / unknown / revoked / empty) is ONE
|
every failure shape (malformed / unknown / revoked / empty) is ONE
|
||||||
generic 401 ``invalid token`` (no enumeration, the phase-16 pattern).
|
generic 401 ``invalid token`` (no enumeration, the phase-16 pattern);
|
||||||
|
while the per-IP failure window is exhausted → 429 + ``Retry-After``,
|
||||||
|
one generic detail — audit SEC-03 (the 429 counter is SHARED with
|
||||||
|
``/api/login``: any auth failure from an IP counts, a success on
|
||||||
|
either route resets it).
|
||||||
* ``POST /api/logout`` — 204; clears the session and expires the
|
* ``POST /api/logout`` — 204; clears the session and expires the
|
||||||
cookie (idempotent for anonymous callers — one logout wipes BOTH
|
cookie (idempotent for anonymous callers — one logout wipes BOTH
|
||||||
roles, the session is one dict).
|
roles, the session is one dict).
|
||||||
@@ -26,6 +31,7 @@ from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
|
from app.core import rate_limit
|
||||||
from app.core import tokens as token_service
|
from app.core import tokens as token_service
|
||||||
from app.core.auth import (
|
from app.core.auth import (
|
||||||
ADMIN_SESSION_KEY,
|
ADMIN_SESSION_KEY,
|
||||||
@@ -38,6 +44,10 @@ from app.core.auth import (
|
|||||||
from app.db import get_db
|
from app.db import get_db
|
||||||
from app.schemas import LoginRequest, TokenAuthRequest, WhoamiResponse
|
from app.schemas import LoginRequest, TokenAuthRequest, WhoamiResponse
|
||||||
|
|
||||||
|
#: One generic 429 detail shared by BOTH throttled login routes (audit
|
||||||
|
#: SEC-03): no enumeration between a login failure and a token failure.
|
||||||
|
TOO_MANY_DETAIL = "too many failed sign-in attempts — try again later"
|
||||||
|
|
||||||
router = APIRouter(tags=["auth"])
|
router = APIRouter(tags=["auth"])
|
||||||
|
|
||||||
|
|
||||||
@@ -49,11 +59,27 @@ def login(payload: LoginRequest, request: Request) -> Response:
|
|||||||
12 h default lifetime). Failure: one generic 401 — the admin count is
|
12 h default lifetime). Failure: one generic 401 — the admin count is
|
||||||
one, so there is nothing else to leak, and a wrong password must not
|
one, so there is nothing else to leak, and a wrong password must not
|
||||||
set any session state.
|
set any session state.
|
||||||
|
|
||||||
|
Rate limiting (audit SEC-03): while the per-IP failure window is
|
||||||
|
exhausted (10 misses in 15 min, shared with ``/api/token-auth``),
|
||||||
|
the pre-check fires BEFORE the password is compared — 429 +
|
||||||
|
``Retry-After``, one generic detail; even a correct password stays
|
||||||
|
429 until the window slides. A success on a CLEAN counter resets it;
|
||||||
|
a failure is recorded on the 401 path.
|
||||||
"""
|
"""
|
||||||
|
ip = request.client.host if request.client is not None else "unknown"
|
||||||
|
if (wait := rate_limit.remaining_wait(ip)) > 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=429,
|
||||||
|
detail=TOO_MANY_DETAIL,
|
||||||
|
headers={"Retry-After": str(wait)},
|
||||||
|
)
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
if check_password(payload.password, settings.admin_password):
|
if check_password(payload.password, settings.admin_password):
|
||||||
sign_in(request.session)
|
sign_in(request.session)
|
||||||
|
rate_limit.reset(ip)
|
||||||
return Response(status_code=204)
|
return Response(status_code=204)
|
||||||
|
rate_limit.record_failure(ip)
|
||||||
raise HTTPException(status_code=401, detail="invalid password")
|
raise HTTPException(status_code=401, detail="invalid password")
|
||||||
|
|
||||||
|
|
||||||
@@ -77,15 +103,28 @@ def token_auth(
|
|||||||
generic 401 ``invalid token``: the lookup is by hash (the service
|
generic 401 ``invalid token``: the lookup is by hash (the service
|
||||||
returns ``None`` for anything that is not an active row), so there
|
returns ``None`` for anything that is not an active row), so there
|
||||||
is no enumeration surface (the phase-16 pattern).
|
is no enumeration surface (the phase-16 pattern).
|
||||||
|
|
||||||
|
Rate limiting (audit SEC-03): identical bookkeeping to ``/api/login``
|
||||||
|
on the SAME per-IP counter — pre-check → 429 + ``Retry-After`` while
|
||||||
|
the window is exhausted; failures recorded, a success resets.
|
||||||
"""
|
"""
|
||||||
|
ip = request.client.host if request.client is not None else "unknown"
|
||||||
|
if (wait := rate_limit.remaining_wait(ip)) > 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=429,
|
||||||
|
detail=TOO_MANY_DETAIL,
|
||||||
|
headers={"Retry-After": str(wait)},
|
||||||
|
)
|
||||||
token = payload.token.strip()
|
token = payload.token.strip()
|
||||||
row = token_service.find_active_by_token(db, token) if token else None
|
row = token_service.find_active_by_token(db, token) if token else None
|
||||||
if row is None:
|
if row is None:
|
||||||
|
rate_limit.record_failure(ip)
|
||||||
raise HTTPException(status_code=401, detail="invalid token")
|
raise HTTPException(status_code=401, detail="invalid token")
|
||||||
token_service.mark_used(row)
|
token_service.mark_used(row)
|
||||||
db.commit()
|
db.commit()
|
||||||
request.session[USER_SESSION_KEY] = True
|
request.session[USER_SESSION_KEY] = True
|
||||||
request.session[USER_TOKEN_ID_KEY] = str(row.id)
|
request.session[USER_TOKEN_ID_KEY] = str(row.id)
|
||||||
|
rate_limit.reset(ip)
|
||||||
return Response(status_code=204)
|
return Response(status_code=204)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
"""Sliding-window rate limiting for failed sign-ins (phase 81, audit SEC-03).
|
||||||
|
|
||||||
|
Audit basis
|
||||||
|
-----------
|
||||||
|
SEC-03 (security audit 2026-09-07, severity Medium): neither ``POST
|
||||||
|
/api/login`` nor ``POST /api/token-auth`` throttled failed attempts —
|
||||||
|
the password is compared in constant time (``secrets.compare_digest``),
|
||||||
|
so an attacker who can reach ``:8000`` could guess at line rate (audit
|
||||||
|
PoC: 10 000 sequential logins, all answered at line rate). This module
|
||||||
|
is the fix: a per-client-IP sliding window over FAILED sign-in
|
||||||
|
attempts only.
|
||||||
|
|
||||||
|
Threshold (owner decision A2 — module constants, NOT ``BOR_`` settings:
|
||||||
|
a security control with a fixed, tested default and no env surface to
|
||||||
|
mistype): ``MAX_FAILURES = 10`` failed attempts per client IP within
|
||||||
|
``WINDOW_SECONDS = 900`` (15 min). The window slides — only failures
|
||||||
|
within the last ``WINDOW_SECONDS`` count. The IP is blocked while its
|
||||||
|
in-window failure count is >= ``MAX_FAILURES``: the 10th failure is
|
||||||
|
recorded, and the 11th attempt is the first 429 (the routes' pre-check
|
||||||
|
runs BEFORE the attempt).
|
||||||
|
|
||||||
|
Shared counter
|
||||||
|
--------------
|
||||||
|
The counter is SHARED by both login routes (``/api/login`` and
|
||||||
|
``/api/token-auth``): any auth failure from an IP counts, so a
|
||||||
|
token-spraying attack cannot dodge the window by alternating routes,
|
||||||
|
and a SUCCESS on either route clears that IP's counter (a legitimate
|
||||||
|
owner who fat-fingers twice is not poisoned). The routes do the
|
||||||
|
bookkeeping (pre-check / record / reset); this module only knows about
|
||||||
|
failure timestamps.
|
||||||
|
|
||||||
|
Client IP
|
||||||
|
---------
|
||||||
|
Callers pass ``request.client.host`` — the DIRECT peer address (owner
|
||||||
|
decision A4). Assumption locked in the phase: the app is served
|
||||||
|
directly in the homelab, with NO reverse proxy in front. If a proxy is
|
||||||
|
ever put in front, this module would need ``X-Forwarded-For`` handling
|
||||||
|
(out of scope now).
|
||||||
|
|
||||||
|
Fail-open
|
||||||
|
---------
|
||||||
|
The limiter is a friction bump, not the security boundary (owner
|
||||||
|
decision A1 — in-memory, per-process, stdlib-only; the counter is lost
|
||||||
|
on restart, an accepted reset). A limiter bug must NEVER lock the
|
||||||
|
owner out: any internal error (a corrupted state entry, a clock
|
||||||
|
anomaly) makes ``remaining_wait`` return 0 (allow) and
|
||||||
|
``record_failure`` drop the count silently. Over-denial is the one
|
||||||
|
outcome this module is forbidden to produce.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from collections import deque
|
||||||
|
|
||||||
|
#: An IP is blocked while it has at least this many failed sign-ins in
|
||||||
|
#: the window (owner decision A2: 10 attempts per 15 minutes).
|
||||||
|
MAX_FAILURES = 10
|
||||||
|
#: Sliding-window length in seconds (15 minutes).
|
||||||
|
WINDOW_SECONDS = 900
|
||||||
|
|
||||||
|
#: Failed-attempt timestamps per client IP, oldest first. Module-global
|
||||||
|
#: on purpose (A1: per-process state, no new service or table); the
|
||||||
|
#: lock keeps the dict honest if the event loop is ever shared.
|
||||||
|
_failures: dict[str, deque[float]] = {}
|
||||||
|
_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def _prune(dq: deque[float], now: float) -> None:
|
||||||
|
"""Drop failures that have slid out of the window (oldest first).
|
||||||
|
|
||||||
|
A timestamp exactly ``WINDOW_SECONDS`` old is expired: the window
|
||||||
|
holds the last ``WINDOW_SECONDS`` strictly.
|
||||||
|
"""
|
||||||
|
cutoff = now - WINDOW_SECONDS
|
||||||
|
while dq and dq[0] <= cutoff:
|
||||||
|
dq.popleft()
|
||||||
|
|
||||||
|
|
||||||
|
def record_failure(client_ip: str) -> None:
|
||||||
|
"""Count one failed sign-in attempt from ``client_ip``.
|
||||||
|
|
||||||
|
Appends ``time.monotonic()`` to the IP's deque and prunes expired
|
||||||
|
entries. NEVER raises (fail-open): any internal error — including a
|
||||||
|
corrupted, non-deque state entry — drops the count silently instead
|
||||||
|
of risk blocking the owner. A corrupt entry is left in place (and
|
||||||
|
treated as "not blocked" by ``remaining_wait``) until ``reset`` or a
|
||||||
|
process restart clears it.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with _lock:
|
||||||
|
now = time.monotonic()
|
||||||
|
dq = _failures.get(client_ip)
|
||||||
|
if dq is None:
|
||||||
|
dq = deque()
|
||||||
|
_failures[client_ip] = dq
|
||||||
|
elif not isinstance(dq, deque):
|
||||||
|
return # corrupt state → drop this count (fail-open)
|
||||||
|
dq.append(now)
|
||||||
|
_prune(dq, now)
|
||||||
|
except Exception:
|
||||||
|
return # a limiter bug must never break the login route
|
||||||
|
|
||||||
|
|
||||||
|
def remaining_wait(client_ip: str) -> int:
|
||||||
|
"""Seconds until ``client_ip`` may attempt again (0 = allowed now).
|
||||||
|
|
||||||
|
Prunes expired failures; if fewer than ``MAX_FAILURES`` remain in
|
||||||
|
the window the IP is allowed (0). Otherwise returns the WHOLE-window
|
||||||
|
wait — ``ceil(WINDOW_SECONDS - (now - oldest))``, clamped to >= 1 —
|
||||||
|
i.e. until the oldest counted failure expires (the value the routes
|
||||||
|
put in ``Retry-After``; the window, not any single failure's age,
|
||||||
|
governs). NEVER raises (fail-open → 0).
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with _lock:
|
||||||
|
now = time.monotonic()
|
||||||
|
dq = _failures.get(client_ip)
|
||||||
|
if not isinstance(dq, deque) or not dq:
|
||||||
|
return 0 # unknown IP or corrupt state → allow (fail-open)
|
||||||
|
_prune(dq, now)
|
||||||
|
if len(dq) < MAX_FAILURES:
|
||||||
|
return 0 # includes the all-expired (empty) case
|
||||||
|
wait = WINDOW_SECONDS - (now - dq[0])
|
||||||
|
return max(1, math.ceil(wait))
|
||||||
|
except Exception:
|
||||||
|
return 0 # a limiter bug must never deny the owner
|
||||||
|
|
||||||
|
|
||||||
|
def reset(client_ip: str) -> None:
|
||||||
|
"""Clear ``client_ip``'s failure count (called on a SUCCESS).
|
||||||
|
|
||||||
|
A successful sign-in on either route unblocks the IP immediately —
|
||||||
|
a legitimate owner who fat-fingers twice is not poisoned by their
|
||||||
|
own earlier misses. Unknown IPs are a silent no-op.
|
||||||
|
"""
|
||||||
|
with _lock:
|
||||||
|
_failures.pop(client_ip, None)
|
||||||
@@ -14,6 +14,15 @@ the ONLY anonymous content is the shared chats (plus the login/infra
|
|||||||
endpoints the gate itself needs) — anonymous chat / suggestions /
|
endpoints the gate itself needs) — anonymous chat / suggestions /
|
||||||
document content now 401 ``authentication required``.
|
document content now 401 ``authentication required``.
|
||||||
|
|
||||||
|
Rate limiting (phase 81, audit SEC-03): the sign-in failure counter is
|
||||||
|
PROCESS state (in-memory per IP), and TestClient's
|
||||||
|
``request.client.host`` is the fixed value ``"testclient"`` — so every
|
||||||
|
client in this module (including ``_admin_client``'s separate
|
||||||
|
TestClient) shares ONE limiter entry. The ``clean_rate_limit`` autouse
|
||||||
|
fixture resets that entry around every test so the deliberate 429 pins
|
||||||
|
can't poison the other tests (and the table TRUNCATEs can't help: the
|
||||||
|
counter is not row state).
|
||||||
|
|
||||||
Requires: podman compose up -d db
|
Requires: podman compose up -d db
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -28,6 +37,9 @@ from sqlalchemy import text
|
|||||||
from test_chat_api import FakeRagLLM, _stream_chat
|
from test_chat_api import FakeRagLLM, _stream_chat
|
||||||
|
|
||||||
from app.api import chat as chat_api
|
from app.api import chat as chat_api
|
||||||
|
from app.api.auth import TOO_MANY_DETAIL
|
||||||
|
from app.core import rate_limit
|
||||||
|
from app.core.rate_limit import WINDOW_SECONDS
|
||||||
from app.main import app as fastapi_app
|
from app.main import app as fastapi_app
|
||||||
from app.models import Document
|
from app.models import Document
|
||||||
from app.rag.importer import import_sources
|
from app.rag.importer import import_sources
|
||||||
@@ -37,6 +49,24 @@ FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
|
|||||||
QUESTION = "How is my Kubernetes cluster set up?"
|
QUESTION = "How is my Kubernetes cluster set up?"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def clean_rate_limit() -> Iterator[None]:
|
||||||
|
"""Reset the in-memory sign-in-failure counter around every test.
|
||||||
|
|
||||||
|
TestClient's ``request.client.host`` is the fixed value
|
||||||
|
``"testclient"``, so every client in this module — including the
|
||||||
|
``_admin_client`` helper's separate TestClient — shares ONE limiter
|
||||||
|
entry, and the counter is process state the table TRUNCATEs cannot
|
||||||
|
clear. Without this reset, the deliberate 429 tests below (which
|
||||||
|
block the shared IP on purpose) would make every later test in the
|
||||||
|
process hit the throttled 429 path instead of the 401/204 contract
|
||||||
|
it pins.
|
||||||
|
"""
|
||||||
|
rate_limit.reset("testclient")
|
||||||
|
yield
|
||||||
|
rate_limit.reset("testclient")
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def clean_tables(db) -> Iterator[None]:
|
def clean_tables(db) -> Iterator[None]:
|
||||||
"""Docs + steering + query log + tokens are global state: reset
|
"""Docs + steering + query log + tokens are global state: reset
|
||||||
@@ -425,3 +455,69 @@ def test_admin_chat_still_streams(client: TestClient, db, seeded_kb: FakeRagLLM)
|
|||||||
assert frames[-1]["type"] == "done"
|
assert frames[-1]["type"] == "done"
|
||||||
assert frames[-1]["deflected"] is False
|
assert frames[-1]["deflected"] is False
|
||||||
assert frames[-1]["sources"][0]["path"] == "homelab/kubernetes.md"
|
assert frames[-1]["sources"][0]["path"] == "homelab/kubernetes.md"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- phase 81: rate-limited failed sign-ins (audit SEC-03) ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_eleventh_failed_login_429_with_retry_after(client: TestClient) -> None:
|
||||||
|
"""10 wrong passwords → 401 (the phase-16 contract, unchanged
|
||||||
|
detail); the 11th attempt is the FIRST 429 — one generic detail + a
|
||||||
|
``Retry-After`` header carrying whole-window seconds."""
|
||||||
|
for _ in range(10):
|
||||||
|
r = client.post("/api/login", json={"password": "nope"})
|
||||||
|
assert r.status_code == 401
|
||||||
|
assert r.json() == {"detail": "invalid password"}
|
||||||
|
r = client.post("/api/login", json={"password": "nope"})
|
||||||
|
assert r.status_code == 429
|
||||||
|
assert r.json()["detail"] == TOO_MANY_DETAIL
|
||||||
|
retry_after = int(r.headers["Retry-After"])
|
||||||
|
assert retry_after > 0
|
||||||
|
assert retry_after <= WINDOW_SECONDS # the whole-window wait, never more
|
||||||
|
|
||||||
|
|
||||||
|
def test_correct_password_while_blocked_still_429(client: TestClient) -> None:
|
||||||
|
"""An exhausted window stays exhausted until it slides: the pre-check
|
||||||
|
fires BEFORE the password is compared, so even a CORRECT password
|
||||||
|
gets the 429 while blocked (success only resets a CLEAN counter —
|
||||||
|
this is the intended semantics, pinned)."""
|
||||||
|
for _ in range(10):
|
||||||
|
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
|
||||||
|
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||||
|
assert r.status_code == 429
|
||||||
|
assert r.json()["detail"] == TOO_MANY_DETAIL
|
||||||
|
assert "bor_session" not in client.cookies # not signed in while throttled
|
||||||
|
|
||||||
|
|
||||||
|
def test_success_on_clean_counter_resets_it(client: TestClient) -> None:
|
||||||
|
"""9 failures + 1 success (204) → the counter is back to 0: the next
|
||||||
|
9 failures are all plain 401s (a fresh 10 would be needed to block
|
||||||
|
again). A fat-fingered owner is not poisoned by earlier misses."""
|
||||||
|
for _ in range(9):
|
||||||
|
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
|
||||||
|
assert client.post("/api/login", json={"password": ADMIN_PASSWORD}).status_code == 204
|
||||||
|
for _ in range(9):
|
||||||
|
r = client.post("/api/login", json={"password": "nope"})
|
||||||
|
assert r.status_code == 401
|
||||||
|
assert r.json() == {"detail": "invalid password"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_login_and_token_failures_share_the_counter(client: TestClient) -> None:
|
||||||
|
"""The counter is SHARED by both routes (a token-spraying attack must
|
||||||
|
not dodge the window by alternating routes): 5 login failures + 5
|
||||||
|
token failures exhaust it, so the 11th attempt of EITHER shape — even
|
||||||
|
with a VALID token — is a 429."""
|
||||||
|
# Admin + token FIRST: the successful admin login resets the shared
|
||||||
|
# counter, so it must not happen after failures have accumulated.
|
||||||
|
admin = _admin_client()
|
||||||
|
_token_id, token = _create_token(admin)
|
||||||
|
for _ in range(5):
|
||||||
|
assert client.post("/api/login", json={"password": "nope"}).status_code == 401
|
||||||
|
for _ in range(5):
|
||||||
|
r = client.post("/api/token-auth", json={"token": "bor_" + "0" * 32})
|
||||||
|
assert r.status_code == 401
|
||||||
|
assert r.json() == {"detail": "invalid token"}
|
||||||
|
r = client.post("/api/token-auth", json={"token": token})
|
||||||
|
assert r.status_code == 429
|
||||||
|
assert r.json()["detail"] == TOO_MANY_DETAIL
|
||||||
|
assert int(r.headers["Retry-After"]) > 0
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
"""Unit: the sliding-window failed sign-in limiter (phase 81, task 01).
|
||||||
|
|
||||||
|
Covers ``app.core.rate_limit`` — the per-client-IP sliding window that
|
||||||
|
both login routes share (the audit SEC-03 fix). Pins, in the task's
|
||||||
|
order:
|
||||||
|
|
||||||
|
* 9 failures → still allowed (``remaining_wait == 0``);
|
||||||
|
* the 10th failure → blocked (``remaining_wait > 0``, ``<= WINDOW_SECONDS``);
|
||||||
|
* independent IPs never share counters;
|
||||||
|
* window slide/expiry — a failure older than ``WINDOW_SECONDS`` stops
|
||||||
|
counting (both pre-seeded module state and a fake monotonic clock);
|
||||||
|
* ``remaining_wait`` is the WHOLE-window wait until the OLDEST in-window
|
||||||
|
failure expires (the ``Retry-After`` contract), not a per-failure age;
|
||||||
|
* ``reset`` (a success on either route) unblocks immediately;
|
||||||
|
* an unknown IP never blocks;
|
||||||
|
* FAIL-OPEN — a corrupted, non-deque state entry and a broken clock
|
||||||
|
both degrade to "allow": ``remaining_wait`` returns 0 and
|
||||||
|
``record_failure`` drops the count without raising. A limiter bug
|
||||||
|
must never lock the owner out.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
from collections import deque
|
||||||
|
from collections.abc import Iterator
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.core import rate_limit as rl
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clean_state() -> Iterator[None]:
|
||||||
|
"""The limiter is module-global: no counter may leak between tests."""
|
||||||
|
rl._failures.clear()
|
||||||
|
yield
|
||||||
|
rl._failures.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def test_nine_failures_still_allowed() -> None:
|
||||||
|
"""Below the threshold: 9 in-window failures → no wait (the 10th
|
||||||
|
failure is the one that crosses the bar)."""
|
||||||
|
for _ in range(rl.MAX_FAILURES - 1):
|
||||||
|
rl.record_failure("ip")
|
||||||
|
assert rl.remaining_wait("ip") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_tenth_failure_blocks_until_the_window_clears() -> None:
|
||||||
|
"""The 10th recorded failure crosses the threshold: the routes'
|
||||||
|
pre-check (the 11th attempt) sees a positive wait — bounded by the
|
||||||
|
whole window and integral for the ``Retry-After`` header."""
|
||||||
|
for _ in range(rl.MAX_FAILURES):
|
||||||
|
rl.record_failure("ip")
|
||||||
|
wait = rl.remaining_wait("ip")
|
||||||
|
assert isinstance(wait, int)
|
||||||
|
assert wait > 0
|
||||||
|
assert wait <= rl.WINDOW_SECONDS
|
||||||
|
|
||||||
|
|
||||||
|
def test_ips_are_independent() -> None:
|
||||||
|
"""One IP's failures never count toward another IP's window."""
|
||||||
|
for _ in range(rl.MAX_FAILURES):
|
||||||
|
rl.record_failure("ip")
|
||||||
|
assert rl.remaining_wait("ip") > 0
|
||||||
|
assert rl.remaining_wait("ip2") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_failures_older_than_the_window_stop_counting_preseeded() -> None:
|
||||||
|
"""Reach into the module state: timestamps ``WINDOW_SECONDS + 1`` in
|
||||||
|
the past have slid out — 10 such 'failures' leave the IP unblocked."""
|
||||||
|
now = time.monotonic()
|
||||||
|
rl._failures["ip"] = deque([now - rl.WINDOW_SECONDS - 1.0] * rl.MAX_FAILURES)
|
||||||
|
assert rl.remaining_wait("ip") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_window_slides_and_wait_is_measured_from_the_oldest(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
"""Fake monotonic clock, 10 failures one second apart (t = 1000..1009):
|
||||||
|
|
||||||
|
* the wait is the WHOLE-window seconds until the OLDEST in-window
|
||||||
|
failure expires — ``ceil(900 - 9) == 891`` at t = 1009, not a
|
||||||
|
per-failure age;
|
||||||
|
* the wait counts down with the oldest (300 at t = 1600, clamped to
|
||||||
|
1 at t = 1899);
|
||||||
|
* the block lifts the moment the oldest failure slides out (t =
|
||||||
|
1900): 9 failures are then still in the window, yet the count is
|
||||||
|
back below the threshold.
|
||||||
|
"""
|
||||||
|
clock = {"t": 1000.0}
|
||||||
|
monkeypatch.setattr(time, "monotonic", lambda: clock["t"])
|
||||||
|
for i in range(rl.MAX_FAILURES):
|
||||||
|
clock["t"] = 1000.0 + i # failures at 1000, 1001, ..., 1009
|
||||||
|
rl.record_failure("ip")
|
||||||
|
|
||||||
|
assert rl.remaining_wait("ip") == 891 # ceil(900 - 9)
|
||||||
|
clock["t"] = 1600.0
|
||||||
|
assert rl.remaining_wait("ip") == 300 # oldest is 600 s old
|
||||||
|
clock["t"] = 1899.0
|
||||||
|
assert rl.remaining_wait("ip") == 1 # ceil(900 - 899), clamped >= 1
|
||||||
|
clock["t"] = 1900.0
|
||||||
|
assert rl.remaining_wait("ip") == 0 # oldest just expired → 9 left
|
||||||
|
|
||||||
|
|
||||||
|
def test_reset_unblocks_immediately() -> None:
|
||||||
|
"""A success on either route clears the IP's counter: the owner who
|
||||||
|
fat-fingered is not poisoned by their own earlier misses."""
|
||||||
|
for _ in range(rl.MAX_FAILURES):
|
||||||
|
rl.record_failure("ip")
|
||||||
|
assert rl.remaining_wait("ip") > 0
|
||||||
|
rl.reset("ip")
|
||||||
|
assert rl.remaining_wait("ip") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_ip_never_blocks() -> None:
|
||||||
|
"""No failures recorded → no wait (the routes' pre-check fast path)."""
|
||||||
|
assert rl.remaining_wait("never-seen") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_fail_open_on_corrupt_state_entry() -> None:
|
||||||
|
"""A non-deque entry in the module state (corruption):
|
||||||
|
``remaining_wait`` must return 0 (never block) and ``record_failure``
|
||||||
|
must drop the count WITHOUT raising (fail-open — a limiter bug never
|
||||||
|
locks the owner out). The IP stays unblocked afterwards."""
|
||||||
|
rl._failures["bad"] = object() # type: ignore[assignment]
|
||||||
|
assert rl.remaining_wait("bad") == 0
|
||||||
|
rl.record_failure("bad") # must not raise
|
||||||
|
assert rl.remaining_wait("bad") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_fail_open_on_broken_clock(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""A clock anomaly (monotonic raising) is equally fail-open: 0 wait
|
||||||
|
and a silent drop — the login route itself must never 500 because
|
||||||
|
of the limiter."""
|
||||||
|
|
||||||
|
def broken_clock() -> float:
|
||||||
|
raise ValueError("clock skew")
|
||||||
|
|
||||||
|
monkeypatch.setattr(time, "monotonic", broken_clock)
|
||||||
|
assert rl.remaining_wait("any") == 0
|
||||||
|
rl.record_failure("any") # must not raise
|
||||||
|
assert rl.remaining_wait("any") == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_reset_unknown_ip_is_a_noop() -> None:
|
||||||
|
"""No entry to drop: ``reset`` must not raise (the routes call it on
|
||||||
|
every success unconditionally)."""
|
||||||
|
rl.reset("ghost")
|
||||||
|
assert rl.remaining_wait("ghost") == 0
|
||||||
Reference in New Issue
Block a user