feat: phases 77–80 — navbar view refresh, static background, API tokens, history suggestion chips
Single consolidated commit for four completed, validated phases (77, 78, 79, 80). The pipeline run left all work uncommitted because the harness commits only with PHASE_COMMIT=1 while child executors are forbidden from committing; the phases themselves all passed validation and moved to .agents/phases/complete/. Phase 77 — navbar view refresh - router.js dispatches bor:view-refresh on re-show / active re-click / popstate (gated on wasMounted; first show and boot exempt) - History / RAG / Sources / Tuning re-fetch on refresh (admin branch); Chat deliberately excluded (stream survival) - History "Refresh" button (admin-only, in-flight disable + status line) - New story suite tests/e2e/test_navbar_refresh.py (7 tests) Phase 78 — static background - Removed the animated glow layers; static 44px grid over the flat --bg canvas; default and reduced-motion renders byte-identical - Updated background/theme E2E suites; removed bg-glow test pins Phase 79 — API tokens - api_tokens model + migration 0012; hash-only token service - Admin tokens API + Tokens admin view; POST /api/token-auth; live-revoking require_user on chat / suggestions / document content - Frontend token gate with localStorage cache; anonymous E2E suites migrated to token login - New story suite tests/e2e/test_api_tokens.py (9 tests) Phase 80 — history suggestion chips - last_questions() endpoint with SEED fallback; startNewChat() refetch - Seed-semantics docs (config.py, .env.example, README) - Integration state matrix + E2E suite rewritten to the 4 chip states Also included: phase-76 report artifacts and the repo restore-test-db skill (previously untracked), scripts/* ruff fixes from phase 77. Final gate state (phase 80 final pass, covers everything above): - uv run pytest --cov=app → 1637 passed, 0 failed, app/ coverage 99% - uv run ruff check . && uv run pyright → clean, 0 errors - Per-phase story E2E suites green in isolation
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
# Phase 79 — API tokens: admin-issued access to the app (only shared chats stay open)
|
||||
|
||||
**Source:** `TODO.md` L5 — "Add api tokens that the admin can generate and hand out so people can log in to use the app. The only thing that should be accessible without an API token is shared chats. The web ui should ask for a token before letting a user through and should cache that token in browser storage so they don't have to keep entering it."
|
||||
**Story:** n/a (TODO-derived — extends the phase-16 single-admin auth, `.agents/user_stories/admin-auth.md`)
|
||||
**Context:** `app/core/auth.py` (the SessionMiddleware cookie session, `require_admin`, `sign_in`/`sign_out`, `ADMIN_SESSION_KEY`), `app/api/auth.py` (`/api/login`, `/api/logout`, `/api/whoami` — `WhoamiResponse{authenticated, role: "admin"|"anonymous"}`), `app/models.py` (SQLAlchemy 2.0 mapped-column models — `SavedChat` is the last one) + `alembic/versions/` (latest is `0011_doc_drafts.py` — the format to mirror), `app/api/chat.py` (`POST /api/chat` — public today), `app/api/suggestions.py` (public today), `app/api/docs.py` (`GET /api/documents/content` — the phase-16 soft rule: deliberately public), `app/schemas.py` (`LoginRequest`, `WhoamiResponse`, …), `frontend/assets/header.js` (the single `/api/whoami` call site — `fetchIsAdmin()` returns `authenticated === true`; the admin-link reveal; the sign-out binding), `frontend/index.html` (the shell — `#app-nav`, the `#view-*` sections, the sign-in/out links, `#main`), `frontend/document.html` + `frontend/assets/document.js` (the document viewer — `fetchIsAdmin` gates the admin-only edit affordance; the viewer itself is public today), `tests/e2e/auth_helpers.py` (the real-form `login` helper), `tests/integration/test_auth_api.py` (pins the phase-16 contract — "viewer stays public (soft rule) and `POST /api/chat` still streams" — that soft rule is SUPERSEDED by this phase).
|
||||
|
||||
## Objective
|
||||
The admin can generate named API tokens and hand them out; a token holder signs in at the in-app gate and uses the app — chat, suggestion chips, cited documents. The ONLY anonymous content is the shared chats (plus the login/infra endpoints the gate itself needs). Every existing admin-only surface stays admin-only.
|
||||
|
||||
## Owner decisions (chat, 2026-09-06 — recorded per AGENTS.md rule 3)
|
||||
- **A3 confirmed — token-user scope:** a token user (role `user`) may: `POST /api/chat`, `GET /api/suggestions`, `GET /api/documents/content`, `GET /api/whoami`, `POST /api/logout`. Admin-only UNCHANGED: the docs list/import/sync, tuning, git sources, doc drafts, and the saved-chats list/save/share/delete (saved chats have no per-user attribution — token users get NO History view; only the admin sees saved chats).
|
||||
- **A4 confirmed — token shape & lifecycle:** `bor_` + 32 hex chars (`secrets.token_hex(16)`); only the SHA-256 hex digest of the FULL token string is stored (unique index) — the plaintext is returned EXACTLY ONCE at creation. `revoked_at` set = dead, and revocation is enforced IMMEDIATELY on the user's next request (the session stores the token id; `require_user` live-checks the row is unrevoked — no server-side session store is added, just a PK lookup).
|
||||
- **A5 confirmed — the gate:** an in-app token-entry overlay on the shell + the same inline gate on `document.html`; `login.html` (admin password) and `shared.html` (anonymous) are UNCHANGED. The entered token is cached in `localStorage["bor.token"]` and silently re-sent to `POST /api/token-auth` on every page load (a failed silent re-auth — revoked token — drops the key and shows the gate). Sign out clears the key. `/api/config` stays public (the gate UI itself needs the branding).
|
||||
- **Auth error semantics:** an unauthenticated (or revoked) caller to a `require_user` endpoint gets 401 `{"detail": "authentication required"}` — 401, not 403 (there is no higher privilege that would unblock them); `require_admin` keeps its 403 `admin only`. `POST /api/token-auth` failures (malformed / unknown / revoked) all get ONE generic 401 `{"detail": "invalid token"}` (no enumeration — the phase-16 pattern).
|
||||
- **`whoami` shape:** `WhoamiResponse{authenticated: bool, role: "admin"|"user"|"anonymous"}` — `authenticated` is true for admin AND user; ALL UI gating switches from `authenticated` to `role === "admin"` (the frontend change is owned by task 05). An admin-signed-in session keeps working exactly as today (a browser that holds BOTH an admin and a user session reports admin; `sign_out` clears everything — one session dict, one logout).
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **Model — `api_tokens` (migration `0012_api_tokens.py`):** `id` UUID PK (uuid4 default); `label` String(120) NOT NULL (the hand-out name, e.g. "alice" — display-only, no index, not unique); `token_hash` String(64) NOT NULL UNIQUE (the sha256 hex digest of the full `bor_…` string — the `documents.content_hash` String(64) precedent); `created_at` TIMESTAMPTZ NOT NULL server-default now; `last_used_at` TIMESTAMPTZ NULL; `revoked_at` TIMESTAMPTZ NULL.
|
||||
- **Service — `app/core/tokens.py` (new):** `generate_token() -> str` (`"bor_" + secrets.token_hex(16)`); `hash_token(token) -> str` (sha256 hexdigest of the FULL token — hashing the full string, not the suffix, so a stripped prefix can never collide); `create_token(db, label) -> tuple[ApiToken, str]` (returns the row + the plaintext exactly once — the row only ever carries the hash); `find_active_by_token(db, token) -> ApiToken | None` (hash → `token_hash ==` lookup → `revoked_at IS NULL`); `mark_used(tok)` (bump `last_used_at` to now — the caller commits); `revoke(db, token_id) -> bool` (set `revoked_at` when not already — False when the row is missing). Module docstring: the lookup is by HASH (a unique-index hit) — sha256's pre-image resistance means there is no token-enumeration or timing surface beyond the DB lookup (the contrast with `check_password`'s constant-time compare is documented, not replicated — there is nothing to compare in constant time here, only to look up).
|
||||
- **Admin API — `app/api/tokens.py` (new router, `tags=["tokens"]`, router-level `dependencies=[Depends(require_admin)]` — the `doc_drafts.py` pattern):** `POST /tokens` body `TokenCreateRequest{label}` → 201 `TokenCreated{id, label, token, created_at}` — the ONLY response that ever carries the plaintext; `GET /tokens` → `TokenList{tokens: [TokenListItem{id, label, created_at, last_used_at, revoked: bool}]}` newest-first (no hashes, no plaintext); `POST /tokens/{id}/revoke` → 204, idempotent (already-revoked → still 204; unknown id → 404 `token not found`). Registered in `app/main.py` with the other API routers (before the static mount).
|
||||
- **Auth API — `app/api/auth.py`:** new `POST /token-auth` (PUBLIC — it is the login): body `TokenAuthRequest{token}` → `find_active_by_token` → miss → 401 `invalid token`; hit → `mark_used` + commit + `session[USER_SESSION_KEY] = True` + `session[USER_TOKEN_ID_KEY] = str(token.id)` → 204. `whoami` reports the three roles. `logout` is unchanged (its `session.clear()` already wipes both roles).
|
||||
- **`require_user` (in `app/core/auth.py`):** `def require_user(request: Request, db: Session = Depends(get_db))` — admin key set → pass; `user` key set → fetch the `ApiToken` row by `user_token_id` (PK hit) — row missing OR `revoked_at` set → pop BOTH user keys from the session + raise 401 `authentication required`; else pass; neither key → 401 same detail. Applied to exactly three endpoints: `POST /api/chat` (`app/api/chat.py`), `GET /api/suggestions` (`app/api/suggestions.py`), `GET /api/documents/content` (`app/api/docs.py` — update its docstring: the phase-16 "deliberately PUBLIC soft rule" is SUPERSEDED — the shared chats page is now the anonymous surface). Everything else: unchanged.
|
||||
- **Public list (the only anonymous access — the owner's sentence):** `/api/health`, `/api/config`, `/api/whoami`, `/api/login`, `/api/token-auth`, `/api/shared/<token>` (JSON snapshot) + the `/shared/<token>` page + `shared.html`, the static assets, and the page documents themselves (`login.html`, `document.html`, the shell — the documents load; their GATED DATA does not: the shell shows the gate, `document.html` shows its inline gate).
|
||||
- **Frontend gate (task 05):** new `frontend/assets/token-gate.js` (module) exposing `mountGate(lockRoot, onAuthed)`: at call — (1) if `localStorage["bor.token"]` exists → `POST /api/token-auth` with it (silent; on failure remove the key — it may have been revoked — and fall through); (2) `fetchWhoami()` → `user` or `admin` → `onAuthed()` (the gate never shows); `anonymous` → show the gate AND `lockRoot.inert = true` (the shell passes `#main`; `document.html` passes its content wrapper) + focus the token input. Submit → token-auth → 204 → `localStorage.setItem("bor.token", …)` → whoami → user → hide the gate (`hidden` + `inert` on the gate — the ship-hidden pattern), `lockRoot.inert = false`, `onAuthed()`. 401 → `#auth-gate-error` (`role="alert"`) visible, input cleared + re-focused. All `localStorage` access in try/catch (private mode → the gate still works, caching is a no-op — the fail-silence storage contract). `header.js`: the single whoami now caches the FULL `{authenticated, role}` in one module promise (`fetchWhoami()`); `fetchIsAdmin()` becomes `fetchWhoami().then(w => w.role === "admin")` — SAME single request, all existing callers keep working; `initSharedHeader()` switches its admin variable to `role === "admin"` (byte-identical behavior for admin/anonymous; a `user` gets: sign-in hidden, sign-out visible, all admin nav links hidden, steering panel removed — the anonymous branch); the sign-out binding gains `localStorage.removeItem("bor.token")` (try/catch, before the reload).
|
||||
- **Gate markup (shell — `index.html`):** body-level `<section class="auth-gate" id="auth-gate" hidden inert aria-labelledby="auth-gate-title">` AFTER `#main` (a `position: fixed; inset: 0` overlay — the body-level doc-modal precedent): the `#sources-gate` visual language (glyph, h2 `#auth-gate-title` "Enter your access token", sub-text pointing at the admin, a `<form id="auth-gate-form">` with a visually-hidden label + `<input id="auth-gate-input" type="text" autocomplete="off" autocapitalize="none" spellcheck="false" required>` (mono), a [Sign in] submit, `#auth-gate-error` (`role="alert"`, hidden), and a "Sign in as admin" link to `/login.html?next=/` (the header's `?next=` convention). `document.html` carries the same markup as `#doc-auth-gate` (task 05).
|
||||
- **Tokens admin view (task 06):** a sixth navbar view folded per the phase-76 pattern — `#nav-tokens` (ships hidden; `header.js` reveals it for role admin, same contract as the other four links), `#view-tokens` section in `index.html`, new `frontend/assets/tokens.js` (`export async function mount(root)`, admin-gated via `fetchIsAdmin()` like `history.js`), `router.js` entries in `VIEW` / `VIEW_PATH` / `VIEW_MODULES` / `TITLES` / `DESCRIPTIONS` (the brand-composition `replaceAll` contract carries over), `"/tokens.html"` in BOTH `app/main.py`'s `_shell_routes` tuple and `app/core/caching.py`'s `HTML_PAGES` (the no-cache + `?v=` contract — the phase-76 task-03 integration-test updates apply: the shell-route / title-table / `_page_file` override assertions gain the path). UI: a create row (label input + [Generate]) → the plaintext appears ONCE in a mono read-only field + [Copy] (the clipboard + inline-fallback house pattern — `tokens.js` keeps its own ~10-line copy, the per-page duplication house style); the once-block is NOT re-shown on a re-render/re-show (the plaintext is gone); a full-width table Label | Created | Last used | Status (Active em-dash vs rose Revoked pill — the stale-pill visual language) | Actions (Revoke — the inline two-step confirm, the `history-confirm-*` pattern, focus to Yes); a `role="status"` live region.
|
||||
- **E2E migration (task 04):** ten chat suites POST to `/api/chat` anonymously today and must sign in first (`auth_helpers.login(page, app_url, next="/")`): `test_agent_document_tools.py`, `test_agent_unlimited_tools.py`, `test_chat_rag.py`, `test_grep_regex_teaching.py`, `test_harness_aligned_tools.py`, `test_honest_deflection.py`, `test_llm_retry.py`, `test_search_tool.py`, `test_tool_path_teaching.py`, `test_tool_scaffolding_guardrails.py`. `auth_helpers.py` gains `login_with_token(page, app_url, token)` — it drives the REAL gate (fill `#auth-gate-input` → submit → wait for the gate to hide); `tests/e2e/test_admin_auth.py`'s anonymous pins that the app is open (chat streams, viewer public) are updated to the 401/gate contract (its password-flow assertions stay). The shared-chat suites stay ANONYMOUS — that is the point of the item.
|
||||
- **Integration test updates (task 03):** the tests that hit the three gated endpoints anonymously (`tests/integration/test_api.py`, `test_chat_api.py`, `test_auth_api.py`, …) sign in as admin first or assert the new 401 where the test's purpose IS the auth contract.
|
||||
|
||||
## Dependencies
|
||||
— (none; extends the completed phase-16 auth; phase 80 builds on this phase's `/api/suggestions` gating)
|
||||
|
||||
## Tasks
|
||||
1. `01_token_model_migration.md` — the `api_tokens` model + migration `0012_api_tokens.py`.
|
||||
2. `02_token_admin_api.md` — the token service + the admin create/list/revoke endpoints.
|
||||
3. `03_token_auth_enforcement.md` — `POST /api/token-auth`, the three-role whoami, `require_user` (live revoke check), enforcement on chat/suggestions/document-content, the integration-contract updates.
|
||||
4. `04_migrate_anonymous_e2e.md` — the `login_with_token` helper + the ten anonymous chat suites sign in; the E2E inventory is green against the gated app.
|
||||
5. `05_frontend_token_gate.md` — the header role plumbing + the shell gate + the localStorage caching + the `document.html` gate.
|
||||
6. `06_tokens_admin_view.md` — the admin Tokens view (phase-76 fold pattern) with generate / list / revoke.
|
||||
7. `07_e2e_story_suite.md` — `tests/e2e/test_api_tokens.py` — the owner's sentence, pinned in a browser.
|
||||
8. `08_regression_sweep_commit.md` — the full pipeline + the README auth section + the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_tokens.py` (the service — shape, hash, create/find round-trip, revocation, last-used, the malformed/unknown/revoked miss paths) + `tests/unit/test_auth.py` extended (the `require_user` matrix: admin pass, active user pass, revoked user 401 + session keys popped, missing row 401, anonymous 401).
|
||||
- Integration: the admin API (201 plaintext-once, list shape without hashes, revoke idempotency, 403 anonymous, 403 token-user); token-auth (valid / invalid / revoked / malformed); the enforcement matrix on the three endpoints; whoami's three roles; logout clearing the token session; the existing anonymous-chat pins updated to the 401 contract.
|
||||
- E2E: the new story suite (task 07) + the migrated suites (task 04) + `test_admin_auth.py` updated + the shared-chat suites green ANONYMOUS.
|
||||
- Coverage: **>90%** on `app/` (the delta: `app/core/tokens.py`, `app/api/tokens.py`, and the modified auth/chat/suggestions/docs files — every new branch tested).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_api_tokens.py -v --no-cov` green in isolation.
|
||||
- [ ] A token user (fresh browser context) can chat end-to-end (mock LLM) and open a cited document; an anonymous caller gets the gate in the UI and 401s on the API; shared chats open anonymously; every admin surface 403s the token user.
|
||||
- [ ] The cached token survives a reload with no re-entry; sign out clears it; a revoked token is refused on the next request AND on a fresh login attempt.
|
||||
- [ ] Full suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean; one atomic `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user