feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport

feat(chat): stream model thinking over SSE and show it in a collapsible block
feat(auth): single-admin password login (signed cookie) — gate tuning + Sources catalog, keep chat and document viewer public
2026-08-24 10:53:04 -04:00 · 2026-08-24 09:52:27 -04:00 · 2026-08-23 19:58:39 -04:00 · 2026-08-22 16:44:42 -04:00 · 2026-08-22 15:52:53 -04:00 · 2026-08-22 15:26:43 -04:00
91 changed files with 9097 additions and 856 deletions
@@ -5,7 +5,9 @@
 > Anchors table are settled — do not re-litigate them in a phase.
 > **Revisions (2026-08-21, owner permission):** A7/A8/A9 revised (multi-format
 > ingestion, hybrid FTS+vector retrieval, re-tuned honesty gate); dark tech
-> theme (Phase 08); clickable document viewer (Phase 10). See roadmap §12.
+> theme (Phase 08); clickable document viewer (Phase 10); thinking display
 > (Phase 17, owner permission 2026-08-23); follow-the-bottom scroll
 > (Phase 18, owner choice 2026-08-23). See roadmap §12.
 ---
@@ -134,14 +136,28 @@ All endpoints stateless (A10). Errors: standard JSON `{detail: str}`.
 ### SSE contract (`POST /api/chat`)
 ```
 data: {"type":"thinking","text":"…"}\n\n
 data: {"type":"thinking","text":"…"}\n\n
 data: {"type":"delta","text":"Hey! "}\n\n
 data: {"type":"delta","text":"Good "}\n\n
 ...
 data: {"type":"done","deflected":false,"sources":[{"source":"Homelab","path":"kubernetes.md","title":"Kubernetes Homelab Cluster"}],"suggestions":[]}\n\n
 ```
-Client rules: render deltas as they arrive; on `done` append source chips /
+Client rules: render deltas as they arrive; render `thinking` text in a
-suggestion chips and clear the busy state; on HTTP/stream error show the
+collapsible block above the answer; auto-collapse on the first `delta`;
-error banner + retry (never a stuck button).
+tolerate interleaved `thinking` events (append — never reopen once the
 answer started); the `done` shape is unchanged (thinking never travels on
 `done`); on `done` append source chips / suggestion chips and clear the
 busy state; on HTTP/stream error show the error banner + retry (never a
 stuck button).
 > **SSE revision (phase 17, owner permission 2026-08-23):** the contract
 > gains one event type — `{"type":"thinking","text":"…"}` — carrying the
 > model's reasoning ahead of the `delta` events (the `turbo` model emits
 > `delta.reasoning_content` chunks before the first content chunk, verified
 > live 2026-08-23; `BOR_STREAM_THINKING=0` suppresses the frames
 > server-side). `delta` and `done` shapes are unchanged — a recorded
 > extension of A15, not a silent deviation.
 ---
@@ -286,12 +302,20 @@ Rules:
 |-------|----|
 | **Idle** | Send button enabled, label "Send". |
 | **Thinking (pre-token)** | 3-dot typing bubble + button disabled with spinner, label "Thinking…". |
 | **Thinking (model reasoning)** | Collapsible `.thinking` block streams open (replaces the typing dots as the live indicator), auto-collapses on the first answer token, toggleable afterwards, persisted with the message (phase 14); 120s guard clears on the first `thinking` *or* `delta` event. |
 | **Streaming** | Deltas append live into the brain bubble; button stays busy. |
 | **Done (answer)** | Source chips under the bubble (mono, path-based); button re-enabled. |
 | **Done (deflected)** | Amber-bordered bubble + "Maybe try:" suggestion chips. |
 | **Error** | Red banner (`role="alert"`) with retry hint; button re-enabled. |
 | **KB offline** | Amber banner at top of chat ("start Postgres…"); chat disabled with explanation. |
 | **Guard** | 120s client-side timeout → error state (a button can never sit "stuck" forever). |
 | **Scroll (follow-the-bottom, phase 18)** | The page auto-scrolls only while the user is pinned to the bottom (≤200px band, `NEAR_BOTTOM_PX` — the composer zone; submitting reveals the user's message through the same gate, which holds in real use); scrolling up holds the viewport for the rest of the turn (thinking and answer alike); restore lands one-shot on the latest message. |
 > The **Thinking (model reasoning)** row is a phase-17 addition (owner
 > permission 2026-08-23) — see the §4 SSE revision.
 >
 > The **Scroll** row is a phase-18 addition (owner choice 2026-08-23 —
 > option 1: follow-the-bottom, no "↓ new content" pill).
 ### 7.5 Component inventory (ids used by tests)
 `#messages` (stream), `#empty-state`, `#suggestions`, `.suggestion-chip`,
@@ -301,7 +325,9 @@ Rules:
 `#stat-last`, `#docs-table`, `#docs-tbody`, `#sources-empty`; viewer
 (Phase 10): `/document.html`, `#doc-title`, `#doc-meta`, `#doc-content`,
 `.doc-raw`, `.format-badge`, `#doc-not-found`, `.doc-link` (Sources table
-path links).
+path links); thinking (phase 17, owner permission 2026-08-23):
 `.thinking`, `.thinking-text` (collapsible thinking block; plain
 `<summary>`, no id).
 ---
@@ -322,7 +348,10 @@ path links).
 - **App logs:** single-line `timestamp LEVEL logger :: message` on stdout;
  uvicorn access logs on. INFO by default (`BOR_LOG_LEVEL`).
 - **Per-chat-turn log line (required):**
-  `question=… embed_ms=… top_score=… fts_hits=… threshold=… deflected=… sources=… total_ms=…`
+  `question=… embed_ms=… top_score=… fts_hits=… tuning=N threshold=… deflected=… sources=… thinking_chars=… total_ms=…`
  (`thinking_chars=` counts the turn's reasoning chars — phase 17, owner
  permission 2026-08-23 — and is counted even when `BOR_STREAM_THINKING=0`
  suppresses the frames.)
 - **Importer logs:** per-file `added|updated|unchanged|pruned` + summary
  (counts, embedding batches, total time).
 - **`query_log` table:** durable record of every question (score, deflection,
@@ -385,6 +414,15 @@ ranks live hybrid results for a question (retrieval tuning).
 | 08 | `08_story_dark_tech_theme.md` | `dark-tech-theme.md` | `tests/e2e/test_dark_tech_theme.py` |
 | 09 | `09_story_retrieval_quality.md` | `retrieval-quality.md` | `tests/e2e/test_retrieval_quality.py` |
 | 10 | `10_story_document_viewer.md` | `document-viewer.md` | `tests/e2e/test_document_viewer.py` |
 | 17 | `17_thinking_display.md` | `thinking-display.md` | `tests/e2e/test_thinking_display.py` |
 | 18 | `18_follow_bottom_scroll.md` | `follow-bottom-scroll.md` | `tests/e2e/test_follow_bottom_scroll.py` |
 > Row 17 (thinking display) added 2026-08-23 with owner permission — the
 > A15 SSE extension recorded in §4.
 >
 > Row 18 (follow-the-bottom scroll) added 2026-08-23 with owner choice —
 > option 1: follow-the-bottom, no "↓ new content" pill (UI-behavior-only
 > change; no anchor revised).
 Completion = unit+integration green, coverage >90%, story E2E green in
 isolation, UI verification passed, **one `--no-gpg-sign` commit**.
@@ -1,76 +0,0 @@
 # Phase 02 — Story: Import Documents
 **Story:** `.agent/user_stories/import-documents.md`
 **Context:** `.agent/PLAN.md` §5 (data model), §9 (logging), §11 (import workflow)
 ## Goal
 The importer (`scripts/import_docs.py`) + `GET /api/docs` + the Sources page
 rendering the indexed documents — the knowledge base becomes refreshable.
 ## Implementation steps
 1. `app/rag/__init__.py`, `app/rag/chunker.py` — markdown-aware chunker
   (PLAN §5 policy: heading splits, 2000-char target, 200 overlap, keep
   nearest heading). Pure functions, fully unit-testable.
 2. `app/rag/llm.py` — `LLMClient` (openai async) with `embed(texts) ->
   list[list[float]]` (batched, `BOR_EMBED_BATCH_SIZE`) and a
   `embed_one`; dimension check vs `settings.embedding_dim` with a loud,
   actionable error. (Chat streaming is added in Phase 03 on this client.)
 3. `app/rag/importer.py` — the core: directory walk (exclusion list, PLAN
   A9; `*.md` only), sha256 delta vs `documents.content_hash`,
   upsert-or-skip, two-phase chunk replace (insert doc → replace chunks →
   embed → commit), `--prune` support, per-file + summary logging.
 4. `scripts/import_docs.py` — CLI wrapper (argparse): repeatable
   `--source` (default `~/Homelab` `~/Deployments`, `expanduser`),
   `--prune`, `--limit`.
 5. `app/api/docs.py` — `GET /api/docs` → `{"documents": [DocSummary]}`
   (include `chunks` count via `func.count`); mount in `app/main.py`
   **before** the static mount.
 6. `frontend/assets/sources.js` + `sources.html` polish — wire the real
   endpoint (already scaffolded to expect this shape); keep the empty state.
 7. Update `README.md` §Knowledge Base Import with the final commands +
   exclusion list + "update your docs → re-run the script" workflow.
 ## UI Verification
 Compare `/sources.html` against the story's "UI Visualization & Structure":
 stat cards `auto-fit minmax(170px,1fr)`; full-width table (≥85% container);
 mono path column with `title` ellipsis; empty state with the exact command;
 `<caption class="visually-hidden">`, `scope="col"`, scroll wrapper
 `role="region" tabindex="0"`. No CDN refs. Take a 1280px and 375px
 screenshot pass before finishing.
 ## Testing & Quality
 - Unit: chunker (heading splits, overlap, short-doc single chunk, code
  fences kept intact), exclusion walk (temp tree with `.venv` junk),
  delta logic (unchanged/changed/pruned via tmp Postgres or in-memory fakes
  — real DB preferred since compose runs locally).
 - Integration: `GET /api/docs` empty shape + populated shape; importer
  end-to-end against `tests/fixtures/docs/` into a test schema.
 - Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**
  on `app/` (importer + chunker + client are the bulk; test them hard).
 ## Playwright Execution Phase
 Run ONLY this story's suite (DB must be up: `podman compose up -d db`):
 ```bash
 uv run pytest tests/e2e/test_import_documents.py -v --no-cov
 ```
 The test file implements the story's Playwright Mapping Rule (seed via the
 import function against `tests/fixtures/docs/` with the mock LLM; assert
 Sources page rows, layout width, and the empty state).
 ## Success criteria
 - [ ] `uv run python -m scripts.import_docs` (fixtures) imports all 3 docs,
      re-run reports `unchanged`
 - [ ] `GET /api/docs` + Sources page show the docs (real run: `~/Homelab`
      + `~/Deployments` counts logged)
 - [ ] unit + integration green, coverage >90%
 - [ ] UI verification passed (screenshots attached to the phase record)
 - [ ] story E2E green in isolation
 - [ ] README import section updated
 - [ ] committed
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(kb): markdown importer with sha256 deltas, chunking, batched embeddings, and Sources page"
 ```
@@ -1,74 +0,0 @@
 # Phase 03 — Story: Chat RAG Answer (happy path)
 **Story:** `.agent/user_stories/chat-rag-answer.md`
 **Context:** `.agent/PLAN.md` §3 (data flow), §4 (SSE contract), §6 (persona), §9 (logging)
 ## Goal
 The core product loop: question → embed → cosine top-4 → full top-2
 documents → `turbo` (streamed) → chippy grounded answer with source chips.
 ## Implementation steps
 1. `app/rag/retriever.py` — `retrieve(db, question_embedding) ->
   list[RetrievedChunk]` (score = 1 − distance, `ORDER BY embedding <=> $1
   LIMIT BOR_TOP_K_CHUNKS`) + `select_documents(chunks, n) -> list[Document]`
   (distinct by `document_id`, ranked by best chunk score, cap content at
   `BOR_MAX_CONTEXT_CHARS` with `[…truncated…]`).
 2. `app/rag/prompts.py` — locked persona + HONESTY GATE prompt builder
   (PLAN §6 verbatim, `<relevance>HIGH|LOW</relevance>`, `<documents>`
   block; LOW mode includes the `DEFLECT_MODE` marker + weak-hit titles).
 3. `app/rag/llm.py` — add `chat_stream(messages) -> AsyncIterator[str]`
   (openai async, `stream=True`, `model=turbo`, temperature 0.4,
   max_tokens ~700).
 4. `app/api/chat.py` — `POST /api/chat` (ChatRequest) → `StreamingResponse`
   (SSE): emit `delta` events from the stream, then the `done` event
   (deflected, sources, suggestions); insert `query_log` row (deflected=
   false this phase); per-turn log line (PLAN §9); structured error events
   (`{"type":"error","detail":…}`) on LLM/DB failure.
 5. `frontend/assets/app.js` — replace the placeholder handler: `fetch` +
   `ReadableStream` SSE parser; render deltas live into a brain bubble
   (reuse the typing-indicator → streaming handoff); on `done`, append
   `.source-chip`s under the bubble; on error, show the banner (full
   state machine is Phase 06 — keep it simple-correct here).
 6. Tune `settings.suggestions` if the real Homelab import revealed better
   defaults (optional here; Phase 05 owns the chips).
 ## UI Verification
 Against the story's "UI Visualization & Structure": bubbles right/left
 (brand vs surface, ≥4.5:1 text), avatar 🧠, source chips mono/brand-soft
 with `source/path` and ellipsis, safe markdown (paste an answer containing
 `<script>alert(1)</script>` from the mock to prove it's escaped). Chat
 column 46rem centered. 1280px + 375px screenshot pass.
 ## Testing & Quality
 - Unit: retriever ordering/dedup/cap (fake rows), prompt builder (HIGH
  contains documents + `HIGH`, LOW contains `DEFLECT_MODE` + titles only,
  persona rules present verbatim), SSE event serialization.
 - Integration: `/api/chat` against the mock LLM with a seeded temp schema —
  assert SSE delta sequence, `done` payload (sources non-empty,
  deflected false), `query_log` row, error event when LLM unreachable.
 - Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**.
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_chat_rag.py -v --no-cov
 ```
 Implements the story mapping: streamed grounded answer + `kubernetes.md`
 source chip + button recovery; DB `query_log` assertion; raw SSE shape
 check via `httpx`.
 ## Success criteria
 - [ ] end-to-end: question → streamed chippy answer citing `kubernetes.md`
 - [ ] `query_log` row per turn; per-turn log line in stdout
 - [ ] LLM-down path shows error banner, no stuck button
 - [ ] unit + integration green, coverage >90%
 - [ ] UI verification passed
 - [ ] story E2E green in isolation
 - [ ] committed
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(rag): stream grounded chat answers via pgvector cosine retrieval with source citations"
 ```
@@ -1,66 +0,0 @@
 # Phase 04 — Story: Honest Deflection
 **Story:** `.agent/user_stories/honest-deflection.md`
 **Context:** `.agent/PLAN.md` §4, §6 (honesty gate), §9
 ## Goal
 When retrieval finds nothing relevant, Brain says so — plainly, chippily —
 and offers real alternatives. No hallucinated confidence.
 ## Implementation steps
 1. `app/api/chat.py` — apply the gate: `best_score < settings.relevance_
   threshold` ⇒ build LOW prompt (`DEFLECT_MODE`, weak-hit titles only),
   else HIGH prompt. Set `deflected` on the `done` event + `query_log`.
 2. Deflection `suggestions[]`: ask `turbo` (same stream) to include 2–3
   alternative questions; simplest robust approach — have the LLM emit them
   inline in the answer AND have the server derive 2–3 chips from the
   weak-hit document titles (deterministic fallback if the model doesn't
   produce a parsable list). Ship the deterministic title-derived chips as
   the v1 behavior; model-generated list is a bonus if trivially parseable.
 3. `frontend/assets/app.js` — on `done.deflected`: add `.is-deflected`
   class to the bubble, render "Maybe try:" chips below it (same
   `.suggestion-chip` component; clicking fills the input — full submit
   behavior lands with Phase 05's chip component; wire what exists).
 4. `README.md` — document `BOR_RELEVANCE_THRESHOLD` tuning + the
   deflection behavior in Troubleshooting.
 ## UI Verification
 Against the story: amber bubble (`#fff7e8` bg / `#f59e0b` border) distinct
 from normal answers; "Maybe try:" chips ≥44px, brand-soft/brand-ink;
 contrast pairs verified (ink on accent-bg ≥ 9:1, accent-ink ≥ 8:1);
 chip group has an accessible name; mobile wraps cleanly.
 ## Testing & Quality
 - Unit: gate boundary with a fake retriever — score exactly 0.30 → HIGH;
  0.2999 → LOW; LOW prompt contains `DEFLECT_MODE` + titles, no full docs;
  HIGH unaffected. Suggestions derivation (2–3, non-empty, derived from
  titles).
 - Integration: mock LLM — off-topic question ("sourdough") ⇒ `done`
  `deflected: true`, `query_log.deflected=true`, weak `top_score` stored;
  on-topic question ⇒ `deflected: false`.
 - Coverage: **>90%** on `app/`.
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov
 ```
 Implements the story mapping: off-topic question ⇒ `.is-deflected` bubble
 matching /haven't done anything like that/i + ≥2 "Maybe try:" chips; chip
 click behavior; (unit boundary test lives in pytest, not here).
 ## Success criteria
 - [ ] off-topic question never gets a confident fake answer
 - [ ] deflected bubble visually distinct + alternative chips render
 - [ ] `query_log.deflected` accurate; threshold env-tunable
 - [ ] unit + integration green, coverage >90%
 - [ ] UI verification passed
 - [ ] story E2E green in isolation
 - [ ] committed
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(rag): honest deflection gate with amber UI state and alternative-question chips"
 ```
@@ -1,59 +0,0 @@
 # Phase 05 — Story: Suggestion Chips
 **Story:** `.agent/user_stories/suggestion-chips.md`
 **Context:** `.agent/PLAN.md` §7 (UI/UX), story file for chip spec
 ## Goal
 Zero-friction onboarding: 3–4 real example questions on first load,
 clickable → filled → submitted, keyboard-first, mobile-scrollable.
 ## Implementation steps
 1. `app/config.py` — confirm `suggestions` is env-overridable
   (`BOR_SUGGESTIONS` as JSON list via pydantic-settings) and tune the
   defaults against the *actually imported* Homelab/Deployments topics
   (read a sample of `documents` titles; pick questions real answers
   exist for).
 2. `app.js` — extract a `renderChips(container, items, {onSelect})` helper;
   real `<button type="button" class="suggestion-chip" role="listitem">`
   inside `#suggestions[role="list"]`; onboarding `onSelect` = fill
   `#message-input` + focus + `composer.requestSubmit()`. Reuse the same
   helper for deflection chips (Phase 04) with the same submit behavior.
 3. Empty-state lifecycle: first user message hides `#empty-state` (already
   done in `addMessage`) — verify chips don't linger in the conversation.
 4. Mobile CSS check: chip row `nowrap + overflow-x auto` at ≤640px (tokens
   already exist — verify, don't duplicate).
 ## UI Verification
 Against the story: pills 999px radius, ≥44px, brand-soft/brand-ink (≥6:1),
 hover/active states; desktop centered wrap vs mobile single scroll row;
 Tab order reaches chips before the composer input is required; screen
 reader: group labeled "Suggested questions". Screenshot pass 1280px + 375px.
 ## Testing & Quality
 - Unit/integration: `GET /api/suggestions` honors `BOR_SUGGESTIONS` env
  override (JSON list); default list has ≥3 non-empty strings.
 - Coverage: **>90%** on `app/` (JS is covered by E2E).
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_suggestion_chips.py -v --no-cov
 ```
 Implements the story mapping: chips render (≥3, role=list); chip click
 submits (user bubble with exact chip text + mock reply); keyboard Tab+Enter
 activates; 375px chip row is a horizontal scroll row.
 ## Success criteria
 - [ ] onboarding chips render from the API; click = one-tap question
 - [ ] keyboard + SR usable; mobile scroll row
 - [ ] deflection chips share the component + submit behavior
 - [ ] unit + integration green, coverage >90%
 - [ ] story E2E green in isolation
 - [ ] committed
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(ui): onboarding suggestion chips with one-tap submit, keyboard access, and mobile scroll row"
 ```
@@ -1,64 +0,0 @@
 # Phase 06 — Story: Loading Feedback & Progress
 **Story:** `.agent/user_stories/loading-feedback.md`
 **Context:** `.agent/PLAN.md` §7.4 ("never stale" contract), §9
 ## Goal
 An unambiguous state machine — `idle → thinking → streaming → done |
 error → idle` — so the user always knows what's happening, and a stale
 Send button is impossible.
 ## Implementation steps
 1. `app.js` — formalize the state machine (single `setUiState(state)`
   function driving: typing indicator, send button disabled/spinner/label,
   `#send-status` live text). Replace ad-hoc busy handling from Phase 03.
 2. Pre-token: typing indicator (`role="status"`,
   `aria-label="Brain of Reese is thinking"`); after 10s pre-token, update
   the label with elapsed seconds (setInterval, cleared on state change).
 3. Streaming: first `delta` removes the typing indicator and starts
   appending to the answer bubble; button stays busy.
 4. Error paths: `{"type":"error"}` SSE event, non-2xx response, or
   **120s client-side timeout** (clear on first delta) → red banner
   `role="alert"` ("Try again — if this persists, check the LLM is
   reachable") + state → idle.
 5. `prefers-reduced-motion`: CSS already slows animations — verify; add a
   static fallback for the dots if needed.
 6. Server: confirm the per-turn log line includes `embed_ms` and
   `total_ms` (add if Phase 03 omitted it).
 ## UI Verification
 Walk the full state machine by hand (dev server + mock LLM slow path):
 submit → indicator + "Thinking…" disabled button → live tokens → done
 (enabled, focused input). Kill the mock mid-stream → banner + recovery.
 Contrast of disabled button + spinner OK; reduced-motion pass.
 ## Testing & Quality
 - Unit/integration: SSE error event serialization; timeout constant
  exported/testable; (JS logic is E2E-covered).
 - Coverage: **>90%** on `app/`.
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_loading_feedback.py -v --no-cov
 ```
 Implements the story mapping (mock LLM's 3s "pretend to think slowly"
 warm-up + a fixture that stops the mock): typing indicator visible during
 pre-token and gone by answer; button disabled→"Thinking…"→enabled "Send";
 streaming appends (two-timestamp length check); LLM-down ⇒ `role=alert`
 banner + button recovered.
 ## Success criteria
 - [ ] every in-flight state has a visible indicator; button never zombies
 - [ ] error + 120s timeout paths both recover cleanly
 - [ ] reduced-motion respected
 - [ ] unit + integration green, coverage >90%
 - [ ] story E2E green in isolation
 - [ ] committed
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(ui): explicit chat state machine — typing indicator, streaming progress, timeout and error recovery"
 ```
@@ -1,57 +0,0 @@
 # Phase 07 — Story: Responsive, Polished, Accessible UI
 **Story:** `.agent/user_stories/responsive-polish.md`
 **Context:** `.agent/PLAN.md` §7 (the whole UI/UX strategy)
 ## Goal
 The final visual + accessibility audit pass across chat and Sources. No
 new features — enforce PLAN §7 end-to-end and fix every deviation.
 ## Implementation steps
 1. Viewport sweep (360 / 375 / 768 / 1280 / 1600) on both pages: fix
   overflow, pinched columns, dead whitespace. Chat column stays ≤46rem
   centered; Sources table full-width with horizontal scroll <640px.
 2. A11y sweep (both pages): landmarks, skip link, labels on every input,
   `aria-label` on every icon-only control, `:focus-visible` outline on
   every focusable, `aria-live` regions intact, no contrast <4.5:1
   (compute, don't eyeball — use the E2E helper).
 3. Reduced-motion + long-content pass (60-char paths, long answers).
 4. No-CDN re-verification on **both** pages (extend the integration test
   to `/sources.html` if it only covers `/`).
 5. Final README polish pass: screenshots section (optional), quickstart
   sanity, "Update your documents" workflow prominent.
 ## UI Verification
 This phase IS the verification: the E2E below is the acceptance test.
 Additionally, manual screenshot pass at 1280px + 375px for both pages,
 reviewed against PLAN §7.1–7.4 before committing.
 ## Testing & Quality
 - Integration: no-CDN check extended to Sources page; health unchanged.
 - Coverage: **>90%** on `app/` (final state of the whole app).
 - Whole suite green: `uv run pytest` (unit+integration) — the entire
  repo must be green at this phase.
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_responsive_polish.py -v --no-cov
 ```
 Implements the story mapping: no horizontal overflow at 5 viewports (both
 pages); chat column capped + centered at 1600px; Sources table ≥80%
 container at 1280px; landmarks/labels/skip-link sweep; WCAG contrast
 pairs ≥4.5:1 (computed); reduced-motion honored.
 ## Success criteria
 - [ ] all six mapping tests pass at every viewport
 - [ ] zero known a11y deviations against PLAN §7.2
 - [ ] whole pytest suite green + coverage >90%
 - [ ] README polished
 - [ ] committed (this commit marks v1.0 feature-complete)
 ## Commit
 ```bash
 git add -A && git commit --no-gpg-sign -m "feat(ui): responsive + WCAG AA polish pass across chat and sources — v1 feature complete"
 ```
@@ -1,136 +0,0 @@
 # Phase 08 — Story: Dark Tech Theme
 **Story:** `.agent/user_stories/dark-tech-theme.md`
 **Context:** `.agent/PLAN.md` §7 (UI/UX strategy), §10 (testing)
 ## Goal
 Re-skin the whole UI to a dark, techy, emoji-free look with a subtly
 animated pure-CSS background — zero behavior or layout changes, WCAG 2.1
 AA re-proven on the new palette.
 ## Dependencies
 Phases 01–07 (all complete). Independent of 09/10; phase 10's viewer page
 inherits this theme, so 08 must land first.
 ## Locked decisions
 No anchors changed. Replaces the §7.2 light contrast pairs with the dark
 palette below (PLAN §7.2 already updated 2026-08-21 with owner
 permission). No new technology — pure CSS/HTML/inline SVG (A11).
 ## Implementation steps
 1. **Palette swap** — `frontend/assets/styles.css` `:root` tokens (all
   pairs computed, ≥4.5:1):
   | token | dark value | computed pair |
   |---|---|---|
   | `--bg` | `#0a0e17` | ink on bg 16.2:1 |
   | `--surface` | `#121a2e` | ink on surface 14.5:1 |
   | `--ink` | `#e8ebf4` | — |
   | `--ink-soft` | `#9aa4bd` | ink-soft on surface 6.9:1 |
   | `--line` | `#26304a` | decorative |
   | `--brand` | `#6d78f2` | **dark ink `--bg` on brand 5.2:1** |
   | `--brand-soft` | `#232b52` | brand-ink on brand-soft 6.9:1 |
   | `--brand-ink` | `#a5b4fc` | brand-ink on surface 8.7:1 |
   | `--accent-bg` | `#2b2110` | accent-ink on accent-bg 9.5:1 |
   | `--accent-ink` | `#fbbf24` | — |
   | `--accent-line` | `#f59e0b` | unchanged (8.9:1 on bg) |
   | `--err-bg` / `--err-ink` | `#2d1318` / `#fca5a5` | 9.1:1 |
   | `--err-line` | `#ef4444` | 4.6:1 on err-bg (UI boundary) |
   | `--ok-bg` / `--ok-ink` | `#10241b` / `#6ee7a8` | 10.6:1 |
   Button text is `--bg` (dark) on `--brand` — **never white on brand**
   (3.7:1, fails). Busy button: keep the `#a5b4fc` background (the
   `tests/unit/test_frontend_feedback.py` assertion greps this token)
   with a **dark** arc (`--bg`, 9.7:1). Update derived light-mode values:
   shadows (black-based, lower alpha), selection, typing dots, chip
   hover.
 2. **Emoji purge** — replace every emoji in chrome with inline SVG
   (`aria-hidden` kept, ~16–20px, `currentColor` where sensible):
   - `frontend/assets/app.js` (~L116, ~L132): avatars 🧠/🧑 → SVG
     circuit-node glyph (brain) / minimal silhouette (user) as JS string
     constants.
   - `frontend/index.html`: favicon 🧠 data-URI → SVG tech mark (hex +
     node, brand color on dark, <1 KB), still a `data:` URI;
     `.brand-mark` 🧠 → same mark; ⚠️ banner icon → SVG triangle; 👋
     empty state → SVG glyph.
   - `frontend/sources.html`: favicon, `.brand-mark`, 📂 empty state →
     SVG marks.
 3. **Tech details** — mono wordmark with letter-spacing; stat values
   mono; radii `10px`/`6px`; 1px `--line` borders on cards/bubbles/table;
   2px gradient hairline (brand→cyan, low alpha) under the sticky header.
 4. **Animated background (pure CSS, zero JS)** — working recipe:
   `html { background: var(--bg) }`, `body { background: transparent;
   position: relative }` (body must not create a stacking context):
   - `body::before` — fine grid: two `linear-gradient`s (1px lines,
     `--line` at ~35% alpha), `background-size: 44px 44px`, masked with a
     radial fade (visible center-top, fading to the edges), animated
     `background-position` `0 0 → 44px 44px`, 60s linear infinite
     (seamless loop — the delta equals one cell).
   - `body::after` — two large soft radial glows: indigo
     `rgba(109,120,242,0.14)` top-left, cyan `rgba(34,211,238,0.10)`
     bottom-right; 14s ease-in-out infinite alternate breathing
     (opacity/scale). No `filter: blur` (perf).
   - Both: `position: fixed; inset: 0; pointer-events: none; z-index:
     -1`. Keep glow alpha low — subtle, never competing with text.
 5. **Reduced motion** — `@media (prefers-reduced-motion: reduce)`:
   `body::before, body::after { animation: none }` (static grid + glows
   remain). Existing typing/spinner reduced-motion handling stays.
 6. **Test updates (behavior unchanged):**
   - `tests/e2e/test_honest_deflection.py` (~L104): deflection bubble
     `backgroundColor` assertion `rgb(255, 247, 232)` → `rgb(43, 33,
     16)`; the border assertion `rgb(245, 158, 11)` is unchanged.
   - New integration test
     `tests/integration/test_api.py::test_ui_chrome_has_no_emoji`: GET
     `/`, `/sources.html`, `/assets/app.js`, `/assets/styles.css` — assert
     no characters in the emoji code-point set (U+1F300–U+1FAFF,
     U+2600–U+27BF, U+2B00–U+2BFF, U+FE0F, U+200D, plus the specific
     glyphs previously used: 🧠 🧑 👋 📂 ⚠️).
 7. **PLAN.md §7.2** — dark contrast table already applied (2026-08-21,
   owner permission); no further plan edits in this phase.
 ## UI Verification
 Manual screenshot pass (1280px + 375px, both pages): grid is faint
 (barely-there), glows soft, no banding; brand button legible (dark
 text); deflection bubble distinct from normal answers; avatars crisp at
 16px; reduced-motion preview (DevTools emulation) shows the static
 background.
 ## Testing & Quality
 - Integration: no-CDN check still green on both pages (it covers
  `/sources.html` since Phase 07); new emoji-guard test above.
 - **Existing E2E regression check:** after the reskin, run the existing
  story suites in isolation and confirm they stay green — at minimum
  `test_chat_rag.py`, `test_honest_deflection.py`, `test_responsive_
  polish.py` (the contrast helper computes from live styles and must pass
  on the new palette).
 - Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**
  on `app/`.
 - `uv run ruff check . && uv run pyright` green.
 ## Playwright Execution Phase
 Run ONLY this story's suite:
 ```bash
 uv run pytest tests/e2e/test_dark_tech_theme.py -v --no-cov
 ```
 Implements the story mapping: dark bg + computed contrast pairs ≥4.5:1;
 no emoji in innerText/outerHTML on both pages; `body::before`/`::after`
 animate; reduced-motion context → `animation-name: none`; on-topic smoke
 (stream + chip + button recovery) unchanged; all assets local.
 ## Success criteria
 - [ ] both pages dark; every text pair ≥4.5:1 (computed in E2E)
 - [ ] zero emoji in chrome (E2E + new integration guard)
 - [ ] animated background subtle, pure CSS, reduced-motion honored
 - [ ] layout metrics + chat behavior unchanged (smoke E2E)
 - [ ] existing story E2E suites still green in isolation
 - [ ] unit + integration green, coverage >90%, ruff + pyright green
 - [ ] committed (force-add `.agent/PLAN.md` + this phase record — rule 8)
 ## Commit
 ```bash
 git add -A
 git add -f .agent/PLAN.md .agent/phases/todo/08_story_dark_tech_theme.md
 git commit --no-gpg-sign -m "feat(ui): dark tech theme — emoji-free chrome, subtle animated CSS background, WCAG AA dark palette"
 ```
@@ -0,0 +1,34 @@
 # Story: Long Answers (No Truncation)
 **Phase:** `11_long_answers.md` · **E2E:** `tests/e2e/test_long_answers.py`
 ## Narrative
 As **a user**, I want Brain to be able to answer at full length (up to
 32 768 output tokens) so complex questions ("walk me through the whole
 setup", "list every service and its config") get a **complete** answer
 instead of one that stops mid-sentence.
 - **Given** any question that deserves a long answer
 - **When** Brain streams its reply
 - **Then** the reply runs to its natural end — the model is allowed up to
  32 768 output tokens, not a hard 700-token cap.
 ## Acceptance criteria
 1. `LLMClient.chat_stream` sends `max_tokens` from settings
   (`BOR_MAX_OUTPUT_TOKENS`, default **32 768**) — the hard-coded 700 is
   gone.
 2. A genuinely long streamed answer (several thousand words) arrives
   **complete** in the browser — final line intact (E2E).
 3. Setting is overridable via env; unit-tested.
 4. Unit + integration green, `app/` coverage >90%, story E2E green in
   isolation, one `--no-gpg-sign` commit.
 ## Playwright Mapping Rule
 **Test Scenario → `tests/e2e/test_long_answers.py`** (mock LLM, seeded KB):
 1. `test_long_answer_streams_to_completion` — question with the
   "write a long answer" trigger → mock emits a ~4 000-word deterministic
   answer and **honors `max_tokens`** (word-based) → the browser shows the
   final line of the answer; under the old 700 cap the tail is missing.
 2. `test_normal_answer_unaffected` — a normal question still streams a
   complete, short answer.
@@ -16,17 +16,35 @@ BOR_LLM_API_KEY=                  # falls back to $AIPI_KEY, then "not-needed"
 BOR_LLM_CHAT_MODEL=turbo
 BOR_LLM_EMBED_MODEL=embed
 BOR_EMBEDDING_DIM=768             # verified 2026-08 via scripts/llm_probe.py
 BOR_STREAM_THINKING=1             # stream the model's thinking as `thinking` SSE events (0 to suppress)
 # --- RAG tuning ---
 BOR_TOP_K_CHUNKS=4
 BOR_TOP_N_DOCS=2
-BOR_RELEVANCE_THRESHOLD=0.30      # max cosine similarity required to answer (else honest deflection)
+BOR_RELEVANCE_THRESHOLD=0.62      # answer when best cosine >= this OR an FTS hit; else honest deflection
 BOR_MAX_CONTEXT_CHARS=24000       # cap on total document text sent to the LLM
 BOR_MAX_OUTPUT_TOKENS=32768       # max answer length in tokens (answers must not be cut off)
 BOR_STEERING_MAX_CHARS=8000       # char budget for the <tuning> (steering notes) prompt section
 BOR_CHUNK_TARGET_CHARS=2000
 BOR_CHUNK_OVERLAP_CHARS=200
 BOR_EMBED_BATCH_SIZE=16
 # --- Hybrid retrieval (vector + Postgres FTS, RRF-fused) ---
 BOR_HYBRID_VECTOR_CANDIDATES=100  # cosine list width for the fusion
 BOR_HYBRID_LEXICAL_CANDIDATES=30  # FTS list width for the fusion
 BOR_RRF_K=60                      # Reciprocal Rank Fusion damping constant
 # --- Import scope (A9 formats; may only narrow, never widen) ---
 # BOR_IMPORT_EXTENSIONS=md,markdown,txt,yaml,yml,json,py
 # BOR_SUGGESTIONS=["How is my Kubernetes cluster set up?"]   # JSON list of onboarding chips
 # --- Admin & sign-in (single-admin password login; BOTH required) ---
 # The app refuses to start while either is empty (names the missing
 # variable(s) — README "Admin & sign-in"). Generate the secret with:
 #   python -c 'import secrets;print(secrets.token_hex(32))'
 BOR_ADMIN_PASSWORD=
 BOR_SESSION_SECRET=
 # BOR_SESSION_MAX_AGE=43200           # signed-cookie lifetime, seconds (default 12 h, sliding)
 # --- Debugging (0/1 — 1 enables attach-on-demand debugpy on port 5678) ---
 DEBUGPY=0
 # DEBUGPY_PORT=5678
@@ -1,10 +1,12 @@
 # 🧠 Brain of Reese
 A chippy, honest **RAG chatbot** over the `~/Homelab` and `~/Deployments`
-projects. Point it at your markdown docs, ask it anything — it retrieves
+projects. Point it at your notes — markdown, YAML, JSON, Python, plain
-the relevant notes with **Postgres 17 + pgvector** cosine search, feeds the
+text — ask it anything, and it retrieves the relevant chunks with
-**whole relevant document** to a **self-hosted LLM** (`turbo` via
+**hybrid search** (pgvector cosine ∪ Postgres full-text search, fused with
-`https://aipi.reeseapps.com/v1`), and streams a grounded answer back.
+Reciprocal Rank Fusion), feeds the **whole relevant document** to a
 **self-hosted LLM** (`turbo` via `https://aipi.reeseapps.com/v1`), and
 streams a grounded answer back.
 If it doesn't have notes for your question, it admits it:
 *"I haven't done anything like that"* — plus suggestions for what it **does** know.
@@ -70,6 +72,124 @@ uv run uvicorn app.main:app --reload
 > 📝 **After this, day-to-day is just: edit markdown → re-run the import.**
 > See [Updating the documents](#updating-the-documents) below.
 ## Using the UI
 - **Chat** (`/`) — ask questions; answers stream in with **source chips**
  that cite the exact documents used. Clicking a chip opens that document
  **in a new tab**.
 - **Document viewer** (`/document.html?source=…&path=…`) — the full text of
  any indexed document, served from the database (no filesystem access):
  markdown is rendered, every other format (`yaml`, `json`, `py`, `txt`, …)
  is shown as escaped monospace text. Unknown documents get a designed
  not-found state with a link back to the index.
 - **Sources** (`/sources.html`) — the indexed document list; the *Path*
  column links each document to the viewer in a new tab. **Admin-only** —
  anonymous visitors see a sign-in gate instead (the catalog is what the
  login locks; the document viewer itself stays open to everyone).
 ## Thinking
 The self-hosted `turbo` model reasons before it answers. That reasoning is
 streamed with the turn as `thinking` SSE events and shown in a
 **collapsible "Thinking" block** above the answer bubble: it opens and
 fills in live while the model thinks, tucks itself away the moment the
 first answer token lands, and stays click-toggleable afterwards. Thinking
 persists with the message, so a reloaded conversation restores the block
 (collapsed) alongside the answer. How much the model thinks — or whether
 it thinks at all — is the model's call: turns without reasoning render
 exactly as before.
 To hide it, set `BOR_STREAM_THINKING=0` — the `thinking` events stop
 (the per-turn log line still counts `thinking_chars`).
 ## Admin & sign-in
 Brain of Reese has exactly **one account: the admin (you)**. Signing in
 unlocks the **full Sources catalog** and the **answer-tuning** controls;
 everyone else stays anonymous and keeps **chat** and the **document
 viewer** (any document an answer cites can be opened by its direct URL —
 the catalog is gated, not the viewer).
 ### Setup (one-time)
 ```bash
 python -c 'import secrets;print(secrets.token_hex(32))'   # → paste into .env
 ```
 ```env
 BOR_ADMIN_PASSWORD=your-password        # plaintext — homelab scope, by design
 BOR_SESSION_SECRET=<the hex from above> # signs the session cookie
 ```
 **Fail-loud:** while either variable is empty the app refuses to start,
 naming the missing one(s):
 ```
 RuntimeError: Brain of Reese cannot start: admin auth is not configured.
 Set the missing variable(s): BOR_ADMIN_PASSWORD, BOR_SESSION_SECRET …
 ```
 ### How it works
 - `POST /api/login {"password": …}` → `204` + signed `bor_session` cookie
  (Starlette `SessionMiddleware` — an itsdangerous-signed cookie, no
  server-side store, no new service, no DB table); any mismatch → `401`
  `{"detail": "invalid password"}` (constant-time compare, one generic
  message — no user enumeration, there is only one user).
 - `POST /api/logout` → `204` (session cleared and cookie expired;
  idempotent for anonymous callers).
 - `GET /api/whoami` → `{"authenticated": bool, "role": "admin"|"anonymous"}`
  — the single source of truth for every UI gating decision.
 - Cookie flags: `same_site="lax"`, `https_only` off — **no HTTPS
  enforcement on purpose** (homelab HTTP; the cookie is single-admin
  convenience, not a cloud boundary). Max age `BOR_SESSION_MAX_AGE`
  (default `43200` = 12 h, refreshed while active).
 - Sign in from the chat header (**Sign in**) or `/login.html` directly;
  the header then offers **Sign out** (logout + reload).
 ### Who can do what
 | Capability | Anonymous | Admin (signed in) |
 |---|---|---|
 | Chat (`/`) + suggestion chips | yes | yes |
 | Document viewer (`/document.html?source=…&path=…`) | yes — any indexed doc by direct URL | yes |
 | Sources catalog (`/sources.html`, `GET /api/docs`) | sign-in gate | full catalog |
 | Tuning (Tune button, Tuning panel, `/api/steering`) | UI hidden | full |
 The public API endpoints stay stateless — the signed cookie is the only
 session state in the system.
 ## Tuning your answers
 *Admin-only* — sign in first (see **Admin & sign-in** above); anonymous
 visitors never see the Tune button or the Tuning panel.
 If an answer isn't quite right — too chatty, wrong assumption, missing
 context — **tune** Brain right there:
 1. Press **“Tune”** in the meta row under any completed answer (deflected
   ones included).
 2. Type a short instruction (1–2000 chars), e.g. *“be more concise”* or
   *“assume I'm on NixOS”*, and **Save**.
 The note is stored in Postgres (`steering_notes`) and read into the
 **system prompt of every subsequent chat turn** as a `<tuning>` section
 (numbered, oldest first, capped at `BOR_STEERING_MAX_CHARS` chars —
 default 8000, overflow marked `[…truncated…]`). With no stored notes the
 prompt is byte-identical to the un-tuned one, so tuning is opt-in per
 note.
 List or remove notes at any time from the **“Tuning”** button in the chat
 header (count badge, newest-first, per-note delete). The API is stateless
 JSON if you prefer curl:
 ```bash
 curl -s localhost:8000/api/steering                                  # list (newest first)
 curl -s -X POST localhost:8000/api/steering \
     -H 'Content-Type: application/json' -d '{"note": "be more concise"}'
 curl -s -X DELETE localhost:8000/api/steering/<note-id>              # remove
 ```
 ## Updating the documents
 **This is the workflow you'll use most.** The knowledge base is refreshed by
@@ -77,9 +197,9 @@ uv run uvicorn app.main:app --reload
 file), so a refresh after a normal editing session takes seconds:
 ```bash
-# After editing/adding/removing markdown in your projects:
+# After editing/adding/removing notes in your projects:
 uv run python -m scripts.import_docs                 # re-index what changed
-uv run python -m scripts.import_docs --prune         # also drop deleted files
+uv run python -m scripts.import_docs --prune         # also drop deleted/out-of-scope files
 # Point it at extra directories (repeatable):
 uv run python -m scripts.import_docs --source ~/SomeOtherDocs
@@ -92,16 +212,61 @@ embedded.
 - The import prints one line per file (`import: added|updated|unchanged|
  pruned …`) and ends with a greppable summary (`import: summary files=…
-  added=… updated=… unchanged=… pruned=… chunks=… embed_batches=…`), so it
+  added=… updated=… unchanged=… pruned=… chunks=… embed_batches=…
-  is safe to run from a cron job or after every commit.
+  formats=md:203,yaml:267,…`), so it is safe to run from a cron job or
- Only **`*.md`** files are indexed. Directories like `.venv`,
+  after every commit.
-  `node_modules`, `.git`, `__pycache__`, `.pytest_cache`, `dist`, `build`
+- Indexed formats (A9): **`md, markdown, txt, yaml, yml, json, py`**
-  are skipped (see `.agent/PLAN.md` anchor A9).
+  (case-insensitive; narrow with `BOR_IMPORT_EXTENSIONS`). Any path with a
  **dot-prefixed component** — hidden files or vendored caches like
  `.esphome/.espressif/**` — is skipped, along with `.venv`,
  `node_modules`, `.git`, `__pycache__`, `.pytest_cache`, `dist`, `build`.
  `--prune` also drops documents whose files no longer match the filter —
  that's how previously imported junk leaves the index.
 - Non-markdown files get format-aware chunking (YAML top-level keys /
  `---` docs, JSON top-level keys, Python top-level defs/classes via
  stdlib `ast`) and their title comes from the file stem.
 - Unchanged files are **not re-embedded** — only new/changed ones, so
  refreshes are cheap.
 - To sanity-check the LLM backend (models + embedding dimension) after any
  aipi change: `uv run python -m scripts.llm_probe`.
 ## Checking retrieval quality
 Ask the *real* pipeline (live aipi embeddings + the current KB) whether a
 question lands on the right document, with the gate verdict and per-document
 cosine / FTS / fused scores:
 ```bash
 uv run python -m scripts.eval_retrieval "How did I install gitlab?"
 uv run python -m scripts.eval_retrieval --from-file questions.txt --top 8
 ```
 Requires `AIPI_KEY` in the environment (same convention as
 `scripts/llm_probe.py`) and an imported knowledge base.
 ## How retrieval works (hybrid)
 Every question is embedded and also lexically tokenized (OR-joined, English
 stemming) and searched **twice** against Postgres:
 1. **Vector** — pgvector cosine top-N (default `BOR_HYBRID_VECTOR_CANDIDATES=100`)
 2. **Lexical** — a stored `tsvector` (GIN-indexed) matched with `to_tsquery`,
   top-N by `ts_rank` (default `BOR_HYBRID_LEXICAL_CANDIDATES=30`)
 The two ranked lists are fused with **Reciprocal Rank Fusion**
 (`score = Σ 1/(k + rank)`, `BOR_RRF_K=60`) — a chunk in both lists scores
 nearly double, which is what lets a name-your-tool question ("gitlab") find
 its own document even when the question embeds close to generic templates.
 The **honesty gate** (A8) then answers (HIGH) when the best cosine is ≥
 `BOR_RELEVANCE_THRESHOLD` (default `0.62`) **or** at least one chunk matched
 lexically (`fts_hits > 0`) — it deflects (LOW) only when *both* signals are
 absent. The top `BOR_TOP_N_DOCS` full documents are still what the LLM sees.
 `query_log` records every turn (`top_score` = best cosine, `fts_hits`,
 `chunk_hits`, `deflected`, `sources`, `latency_ms`) — the raw material for
 tuning: `psql … -c 'SELECT question, top_score, fts_hits, deflected FROM query_log ORDER BY created_at DESC LIMIT 20'`.
 ## Debugging
 `debugpy` is **off by default** and *never imported* unless you opt in —
@@ -203,11 +368,18 @@ served locally (no CDN), `BOR_ENVIRONMENT=production`.
 | `BOR_LLM_CHAT_MODEL` | `turbo` | chat model |
 | `BOR_LLM_EMBED_MODEL` | `embed` | embedding model |
 | `BOR_EMBEDDING_DIM` | `768` | vector dimension (fixed at table creation) |
 | `BOR_TOP_K_CHUNKS` | `4` | chunks retrieved per question |
 | `BOR_TOP_N_DOCS` | `2` | full documents fed to the LLM |
-| `BOR_RELEVANCE_THRESHOLD` | `0.30` | best cosine similarity required to answer; below ⇒ honest deflection |
+| `BOR_RELEVANCE_THRESHOLD` | `0.62` | answer when best cosine ≥ this **or** an FTS hit; below + no FTS ⇒ honest deflection |
 | `BOR_HYBRID_VECTOR_CANDIDATES` | `100` | cosine list width for the RRF fusion |
 | `BOR_HYBRID_LEXICAL_CANDIDATES` | `30` | FTS list width for the RRF fusion |
 | `BOR_RRF_K` | `60` | RRF damping constant (`1/(k + rank)`) |
 | `BOR_IMPORT_EXTENSIONS` | `md,markdown,txt,yaml,yml,json,py` | csv of importable formats (may only narrow the A9 set) |
 | `BOR_MAX_CONTEXT_CHARS` | `24000` | cap on total document text sent to the LLM |
 | `BOR_STEERING_MAX_CHARS` | `8000` | char budget for the `<tuning>` (steering notes) prompt section |
 | `BOR_SUGGESTIONS` | built-in list | JSON list of onboarding chips |
 | `BOR_ADMIN_PASSWORD` | *(required)* | the single admin's password (plaintext, `.env`); app refuses to start when empty |
 | `BOR_SESSION_SECRET` | *(required)* | signing key for the `bor_session` cookie; `python -c 'import secrets;print(secrets.token_hex(32))'` |
 | `BOR_SESSION_MAX_AGE` | `43200` | session-cookie lifetime in seconds (12 h, sliding) |
 | `DEBUGPY` | `0` | `1` ⇒ attach-on-demand debugpy on `DEBUGPY_PORT` (default 5678) |
 | `BOR_LOG_LEVEL` | `INFO` | app log level |
@@ -227,23 +399,26 @@ served locally (no CDN), `BOR_ENVIRONMENT=production`.
  drop + recreate the chunks table (new migration or manual `TRUNCATE
  chunks, documents`).
 - **Honest deflection (the amber “I haven't done anything like that”
-  bubble)** — every question passes the honesty gate: when the best
+  bubble)** — every question passes the honesty gate: deflection happens
-  cosine similarity is below `BOR_RELEVANCE_THRESHOLD` (default `0.30`),
+  only when the best cosine similarity is below
-  Brain switches to deflection mode instead of guessing. The LLM prompt
+  `BOR_RELEVANCE_THRESHOLD` (default `0.62`) **and** no chunk matched the
-  then carries weak-hit *titles only* (no document content), the reply
+  question lexically (`fts_hits = 0`). A weak cosine with a lexical hit
-  opens with “I haven't done anything like that”, the bubble renders
+  (name-your-tool questions) still gets a grounded answer. When it does
-  amber with “Maybe try” chips derived from the closest indexed titles,
+  deflect, the LLM prompt carries weak-hit *titles only* (no document
-  the SSE `done` event carries `deflected: true` + `suggestions[]`, and
+  content), the reply opens with “I haven't done anything like that”, the
-  the `query_log` row records `deflected=true` + the weak `top_score`.
+  bubble renders amber with “Maybe try” chips derived from the closest
-  This is a feature, not a bug — the KB simply has no notes that close;
+  indexed titles, the SSE `done` event carries `deflected: true` +
-  the chips always point at topics Brain really covers.
+  `suggestions[]`, and the `query_log` row records `deflected=true` + the
  weak `top_score` + `fts_hits`. This is a feature, not a bug — the KB
  simply has no notes that close; the chips always point at topics Brain
  really covers.
 - **Answers deflect too often / too rarely** — tune
  `BOR_RELEVANCE_THRESHOLD` (lower = answers more, higher = more honest
-  deflection): `0.0` ⇒ every question gets answered, even unknown topics
+  deflection): `0.0` ⇒ the gate leans entirely on FTS hits; `1.0` ⇒
-  (expect confident-sounding guesses); `1.0` ⇒ everything deflects
+  everything deflects unless a chunk matches lexically. The `embed` model's
-  (nothing but a perfect 1.0 score counts as relevant). After changing
+  cosines cluster in a ~0.6–0.85 band on the live KB, so the default is
-  it, check the real scores:
+  `0.62`; after changing it, check the real scores:
-  `psql … -c 'SELECT question, top_score, deflected FROM query_log ORDER BY created_at DESC LIMIT 20'`
+  `psql … -c 'SELECT question, top_score, fts_hits, deflected FROM query_log ORDER BY created_at DESC LIMIT 20'`
 - **KB offline banner in the chat** — Postgres isn't running:
  `podman compose up -d db`.
 - **Stuck "Thinking…"** — the LLM is slow or down; a 120s client timeout
@@ -0,0 +1,40 @@
 """hybrid retrieval: generated FTS column on chunks + query_log.fts_hits
 Revision ID: 0002
 Revises: 0001
 Create Date: 2026-08-21
 A7/A8 (revised 2026-08-21, owner permission): retrieval becomes hybrid
 (cosine top-N + Postgres full-text top-N, RRF-fused). This adds:
 * ``chunks.tsv`` — generated ``TSVECTOR`` (``to_tsvector('english',
  content) STORED``) + GIN index for the lexical candidate list.
 * ``query_log.fts_hits`` — INT, nullable (pre-existing rows stay NULL:
  the column only carries meaning from hybrid retrieval onward).
 """
 from __future__ import annotations
 import sqlalchemy as sa
 from alembic import op
 revision = "0002"
 down_revision = "0001"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.execute(
        "ALTER TABLE chunks "
        "ADD COLUMN tsv tsvector "
        "GENERATED ALWAYS AS (to_tsvector('english', content)) STORED"
    )
    op.execute("CREATE INDEX ix_chunks_tsv ON chunks USING gin (tsv)")
    op.add_column("query_log", sa.Column("fts_hits", sa.Integer(), nullable=True))
 def downgrade() -> None:
    op.drop_column("query_log", "fts_hits")
    op.execute("DROP INDEX IF EXISTS ix_chunks_tsv")
    op.execute("ALTER TABLE chunks DROP COLUMN IF EXISTS tsv")
@@ -0,0 +1,39 @@
 """steering notes: owner tuning notes injected into every system prompt
 Revision ID: 0003
 Revises: 0002
 Create Date: 2026-08-22
 Phase 15 (steering-notes story): the owner can "tune" how Brain answers
 from the chat UI. Notes live in ``steering_notes`` and are read into the
 system prompt of every subsequent chat turn (``<tuning>`` section).
 """
 from __future__ import annotations
 import sqlalchemy as sa
 from sqlalchemy.dialects import postgresql
 from alembic import op
 revision = "0003"
 down_revision = "0002"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.create_table(
        "steering_notes",
        sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
        sa.Column("note", sa.Text(), nullable=False),
        sa.Column(
            "created_at",
            sa.DateTime(timezone=True),
            server_default=sa.func.now(),
            nullable=False,
        ),
    )
 def downgrade() -> None:
    op.drop_table("steering_notes")
@@ -0,0 +1,63 @@
 """Auth API — single-admin sign-in (phase 16; A10 revised 2026-08-22).
 * ``POST /api/login``   — 204 + signed session cookie on success; 401
  ``invalid password`` on any mismatch (constant-time, one generic
  message, no session set).
 * ``POST /api/logout``  — 204; clears the session and expires the cookie
  (idempotent for anonymous callers).
 * ``GET  /api/whoami``  — ``{"authenticated": bool, "role":
  "admin"|"anonymous"}``; the single source of truth for all UI gating.
 The public API otherwise stays stateless (A10): chat, the document
 content endpoint (soft rule — anonymous may open any document by direct
 URL), suggestions, and health never require the cookie.
 """
 from __future__ import annotations
 from fastapi import APIRouter, HTTPException, Request, Response
 from app.config import get_settings
 from app.core.auth import ADMIN_SESSION_KEY, check_password, sign_in, sign_out
 from app.schemas import LoginRequest, WhoamiResponse
 router = APIRouter(tags=["auth"])
@router.post("/login", status_code=204)
 def login(payload: LoginRequest, request: Request) -> Response:
    """Sign in the single admin.
    Success: 204 + the signed ``bor_session`` cookie (``same_site=lax``,
    12 h default lifetime). Failure: one generic 401 — the admin count is
    one, so there is nothing else to leak, and a wrong password must not
    set any session state.
    """
    settings = get_settings()
    if check_password(payload.password, settings.admin_password):
        sign_in(request.session)
        return Response(status_code=204)
    raise HTTPException(status_code=401, detail="invalid password")
@router.post("/logout", status_code=204)
 def logout(request: Request, response: Response) -> Response:
    """Sign out: clear the session AND expire the browser cookie.
    ``sign_out`` empties the session dict (which the middleware does not
    re-persist — an empty session has nothing to sign), so this route also
    sends ``delete_cookie`` to make the browser drop the signed cookie
    right now. Idempotent: an anonymous logout is still a 204.
    """
    sign_out(request.session)
    response.delete_cookie(get_settings().session_cookie, path="/")
    return Response(status_code=204)
@router.get("/whoami", response_model=WhoamiResponse)
 def whoami(request: Request) -> WhoamiResponse:
    """Who is the caller? Drives every UI gating decision (phase 16)."""
    authenticated = bool(request.session.get(ADMIN_SESSION_KEY))
    return WhoamiResponse(
        authenticated=authenticated,
        role="admin" if authenticated else "anonymous",
    )
@@ -1,18 +1,33 @@
 """POST /api/chat — a RAG chat turn streamed over SSE (PLAN §3/§4).
-Flow (LOCKED A7/A15): embed the question → pgvector cosine top-K chunks →
+Flow (LOCKED A7/A15): embed the question → hybrid retrieval (cosine
-the **honesty gate** (A8: best score < ``BOR_RELEVANCE_THRESHOLD`` ⇒
+top-N ∪ Postgres FTS top-N, RRF-fused) → the **honesty gate** → locked
-deflection) → locked persona prompt (PLAN §6) → ``turbo`` streamed as
+persona prompt (PLAN §6) → ``turbo`` streamed as ``delta`` events → final
-``delta`` events → final ``done`` event (``deflected``, ``sources``,
+``done`` event (``deflected``, ``sources``, ``suggestions``) +
-``suggestions``) + ``query_log`` row + the per-turn log line (PLAN §9).
+``query_log`` row + the per-turn log line (PLAN §9).
 Mid-stream failures become a structured ``error`` event; a pre-stream DB
 outage is a plain 503 JSON.
-Honesty gate: a weak retrieval (score strictly below the threshold — or
+Thinking (phase 17, PLAN §4 extension, owner permission 2026-08-23): the
-an empty KB) flips the turn to deflection mode: the LOW prompt carries
+model's reasoning arrives ahead of the answer and is streamed as
-weak-hit *titles only* (never document content) plus deterministic
+``thinking`` events before the ``delta`` events of the same turn. Each
-"Maybe try" chips, and the ``done`` event / ``query_log`` row record
+turn's thinking is counted in the per-turn log line
-``deflected=true`` with the weak score.
+(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
 ``thinking`` frames (the pieces are still counted).
 Honesty gate (A8, revised 2026-08-21): LOW — deflection — only when the
 best cosine is strictly below ``BOR_RELEVANCE_THRESHOLD`` **and** no
 candidate chunk FTS-matches the question (``fts_hits == 0``). A
 name-your-tool question with weak vector overlap but a lexical hit still
 gets a grounded answer. Deflection mode carries weak-hit *titles only*
 (never document content) plus deterministic "Maybe try" chips, and the
 ``done`` event / ``query_log`` row record ``deflected=true``, the weak
 score and the ``fts_hits`` count.
 Steering (phase 15): the owner's stored tuning notes are loaded per turn
 (oldest first) and injected into the system prompt as a ``<tuning>``
 section — both the HIGH and the LOW prompt carry it. The per-turn log
 line records ``tuning=N`` (the number of injected notes).
 """
 from __future__ import annotations
@@ -27,14 +42,26 @@ from fastapi import APIRouter, Depends
 from fastapi.responses import JSONResponse, StreamingResponse
 from sqlalchemy.orm import Session
 from app.api.steering import load_steering_notes
 from app.config import Settings, get_settings
 from app.db import db_available, get_db
 from app.models import Document, QueryLog
-from app.rag.llm import EmbeddingError, LLMClient, LLMError
+from app.rag.llm import (
    EmbeddingError,
    LLMClient,
    LLMError,
    StreamPiece,  # noqa: F401  (phase 17 typing: chat_stream yields StreamPiece)
 )
 from app.rag.prompts import build_deflect_prompt, build_high_prompt
 from app.rag.retriever import RetrievedChunk, retrieve, select_documents, weak_hit_titles
 from app.rag.suggestions import derive_suggestions
-from app.schemas import ChatDoneEvent, ChatErrorEvent, ChatRequest, SourceRef
+from app.schemas import (
    ChatDoneEvent,
    ChatErrorEvent,
    ChatRequest,
    ChatThinkingEvent,
    SourceRef,
 )
 logger = logging.getLogger("app.chat")
 router = APIRouter(tags=["chat"])
@@ -62,37 +89,64 @@ def sse_event(payload: dict[str, Any]) -> str:
 class TurnPlan:
    """What one chat turn sends to the LLM and reports on ``done``."""
-    top_score: float
+    top_score: float  # best cosine across candidates (query_log.top_score)
    fts_hits: int  # lexical (OR-tsquery) candidates matched
    deflected: bool
    system_prompt: str
    docs: list[Document]  # cited sources (weak hits when deflected)
    suggestions: list[str]  # "Maybe try" chips (deflected turns only)
    tuning_count: int = 0  # steering notes injected into the system prompt
-def plan_turn(chunks: Sequence[RetrievedChunk], settings: Settings) -> TurnPlan:
+def plan_turn(
-    """Apply the honesty gate (A8) and assemble prompt + context for a turn.
+    chunks: Sequence[RetrievedChunk],
    settings: Settings,
    notes: Sequence[str] | None = None,
 ) -> TurnPlan:
    """Apply the honesty gate (A8, revised) and assemble prompt + context.
-    * ``top_score >= threshold`` → grounded: HIGH prompt with the full
+    * **HIGH (grounded)** when ``best_cosine >= threshold`` **or**
-      top-N documents, no suggestions. A score exactly at the threshold
+      ``fts_hits > 0``: HIGH prompt with the full top-N documents, no
-      is an answer — the gate is strict (``score < threshold``).
+      suggestions. A cosine exactly at the threshold is an answer — the
-    * ``top_score < threshold`` (or no hits at all) → deflected: LOW
+      gate is strict (``< threshold``).
-      prompt (``DEFLECT_MODE``) with weak-hit titles only — never document
+    * **LOW (deflected)** only when ``best_cosine < threshold`` **and**
-      content — plus deterministic alternative-question chips derived
+      ``fts_hits == 0`` (or no hits at all): LOW prompt (``DEFLECT_MODE``)
-      from those titles.
+      with weak-hit titles only — never document content — plus
      deterministic alternative-question chips derived from those titles.
    ``top_score`` (stored in ``query_log``) is the best cosine, so the
    gate input is always a pure vector-similarity number; the lexical
    signal is recorded separately as ``fts_hits``.
    *notes* are the owner's steering notes (phase 15, oldest first):
    when non-empty, both the HIGH and the LOW prompt carry the
    ``<tuning>`` section; with no notes the prompts are unchanged.
    """
-    top_score = chunks[0].score if chunks else 0.0
+    steering = list(notes or [])
-    if top_score >= settings.relevance_threshold:
+    best_cosine = max((c.cosine for c in chunks), default=0.0)
    fts_hits = sum(1 for c in chunks if c.fts_hit)
    if best_cosine >= settings.relevance_threshold or fts_hits > 0:
        docs = select_documents(
            chunks, n=settings.top_n_docs, max_chars=settings.max_context_chars
        )
-        return TurnPlan(top_score, False, build_high_prompt(docs), docs, [])
+        return TurnPlan(
            best_cosine,
            fts_hits,
            False,
            build_high_prompt(docs, notes=steering),
            docs,
            [],
            len(steering),
        )
    titles = weak_hit_titles(chunks)
    return TurnPlan(
-        top_score,
+        best_cosine,
        fts_hits,
        True,
-        build_deflect_prompt(titles),
+        build_deflect_prompt(titles, notes=steering),
        select_documents(chunks, n=settings.top_n_docs, max_chars=settings.max_context_chars),
        derive_suggestions(titles, settings.suggestions),
        len(steering),
    )
@@ -139,12 +193,14 @@ async def chat(
            return
        embed_ms = int((time.monotonic() - t0) * 1000)
-        # 2. Retrieve top-K chunks, then the honesty gate (A8) picks the
+        # 2. Retrieve top-K chunks, load the owner's steering notes
-        #    HIGH (grounded) or LOW (deflected) prompt + context.
+        #    (phase 15), then the honesty gate (A8) picks the HIGH
        #    (grounded) or LOW (deflected) prompt + context.
        settings = get_settings()
        try:
-            chunks = retrieve(db, question_vec)
+            steering_notes = load_steering_notes(db)
-            plan = plan_turn(chunks, settings)
+            chunks = retrieve(db, request.message, question_vec)
            plan = plan_turn(chunks, settings, notes=steering_notes)
        except Exception:  # noqa: BLE001 — DB failure mid-turn
            logger.exception(
                "chat: retrieval failed question=%r total_ms=%d",
@@ -164,9 +220,19 @@ async def chat(
        ]
        # 3. Stream the answer (grounded, or an honest deflection).
        #    Phase 17: thinking pieces stream as ``thinking`` events
        #    ahead of the ``delta`` events (PLAN §4 extension); the
        #    kill-switch (``BOR_STREAM_THINKING=0``) suppresses the
        #    frames, not the counting.
        thinking_chars = 0
        try:
-            async for piece in llm.chat_stream(messages):
+            async for piece in llm.chat_stream(messages):  # StreamPiece (phase 17)
-                yield sse_event({"type": "delta", "text": piece})
+                if piece.kind == "thinking":
                    thinking_chars += len(piece.text)
                    if settings.stream_thinking:
                        yield sse_event(ChatThinkingEvent(text=piece.text).model_dump())
                else:
                    yield sse_event({"type": "delta", "text": piece.text})
        except LLMError as e:
            logger.error(
                "chat: LLM stream failed question=%r total_ms=%d — %s",
@@ -188,6 +254,7 @@ async def chat(
                QueryLog(
                    question=request.message,
                    top_score=plan.top_score,
                    fts_hits=plan.fts_hits,
                    chunk_hits=len(chunks),
                    deflected=plan.deflected,
                    sources=", ".join(source_paths),
@@ -199,14 +266,17 @@ async def chat(
            logger.exception("chat: failed to write query_log question=%r", request.message)
        logger.info(
-            "question=%r embed_ms=%d top_score=%.3f threshold=%.2f deflected=%s "
+            "question=%r embed_ms=%d top_score=%.3f fts_hits=%d tuning=%d threshold=%.2f "
-            "sources=%r total_ms=%d",
+            "deflected=%s sources=%r thinking_chars=%d total_ms=%d",
            request.message,
            embed_ms,
            plan.top_score,
            plan.fts_hits,
            plan.tuning_count,
            settings.relevance_threshold,
            plan.deflected,
            source_paths,
            thinking_chars,
            total_ms,
        )
        yield sse_event(
@@ -1,23 +1,45 @@
-"""GET /api/docs — the indexed document list (feeds the Sources page)."""
+"""GET /api/docs — the indexed document list (feeds the Sources page).
 GET /api/documents/content — one indexed document's full content (feeds the
 clickable document viewer, phase 10). DB-only by design: the (source, path)
 pair is looked up as a row, so there is no filesystem access and no
 path-traversal surface — ``../``-style values simply aren't rows (→ 404).
 """
 from __future__ import annotations
-from fastapi import APIRouter, Depends
+from pathlib import Path
 from fastapi import APIRouter, Depends, HTTPException
 from sqlalchemy import func, select
 from sqlalchemy.orm import Session
 from app.core.auth import require_admin
 from app.db import get_db
 from app.models import Chunk, Document
-from app.schemas import DocList, DocSummary
+from app.schemas import DocContent, DocList, DocSummary
 router = APIRouter(tags=["kb"])
 def doc_format(path: str) -> str:
    """Lowercased path suffix without its dot (``kubernetes.md`` → ``md``,
    ``notes/deep.Markdown`` → ``markdown``); ``text`` when the path has no
    suffix — the value shown in the viewer's format badge."""
    return Path(path).suffix.lower().lstrip(".") or "text"
@router.get("/docs", response_model=DocList)
-def list_documents(db: Session = Depends(get_db)) -> DocList:  # noqa: B008
+def list_documents(
    db: Session = Depends(get_db),  # noqa: B008
    _admin: None = Depends(require_admin),  # noqa: B008
 ) -> DocList:
    """All indexed documents with per-document chunk counts.
-    An empty list means the knowledge base has not been imported yet —
+    Admin-only (phase 16 — the catalog is what the sign-in gates; the
-    the Sources page renders its designed empty state in that case.
+    document viewer itself stays public, see below). Anonymous callers
    get 403 ``admin only`` and the Sources page renders its sign-in gate
    instead. An empty list means the knowledge base has not been imported
    yet — the Sources page renders its designed empty state in that case.
    """
    rows = db.execute(
        select(
@@ -45,3 +67,38 @@ def list_documents(db: Session = Depends(get_db)) -> DocList:  # noqa: B008
            for row in rows
        ]
    )
@router.get("/documents/content", response_model=DocContent)
 def get_document_content(
    source: str, path: str, db: Session = Depends(get_db)  # noqa: B008
 ) -> DocContent:
    """Full content of one indexed document, looked up by ``(source, path)``.
    Stateless (A10) and database-only: unknown pairs — including traversal
    strings such as ``../../etc/passwd`` — are just non-existent rows and
    map to 404 ``{detail: "document not found"}``.
    Deliberately PUBLIC for anonymous callers (phase 16 soft rule, owner
    decision 2026-08-22): the *catalog* (``GET /api/docs``) is what the
    sign-in gates, not the viewer — chat cites documents and anyone may
    open a cited document by direct URL.
    """
    row = db.execute(
        select(Document, func.count(Chunk.id).label("chunks"))
        .outerjoin(Chunk, Chunk.document_id == Document.id)
        .where(Document.source == source, Document.path == path)
        .group_by(Document.id)
    ).first()
    if row is None:
        raise HTTPException(status_code=404, detail="document not found")
    doc, chunks = row
    return DocContent(
        source=doc.source,
        path=doc.path,
        title=doc.title,
        format=doc_format(doc.path),
        content=doc.content,
        indexed_at=doc.indexed_at.isoformat(),
        chunks=chunks,
    )
@@ -0,0 +1,82 @@
 """Steering notes API — tune how Brain answers (phase 15, story
 ``steering-notes``).
 Admin-only CRUD under ``/api/steering`` (phase 16, A10 revised): notes
 are owner instructions stored in Postgres (``steering_notes``) and read
 into the system prompt of **every** chat turn as the ``<tuning>`` section
 (see :func:`app.rag.prompts.build_steering_section` and
 :func:`app.api.chat.chat`). The whole router sits behind
 :func:`app.core.auth.require_admin` — anonymous callers get 403 on every
 steering route (the chat turn itself reads the table in-process and
 stays public).
 """
 from __future__ import annotations
 import uuid
 from fastapi import APIRouter, Depends, HTTPException, Response
 from sqlalchemy import select
 from sqlalchemy.orm import Session
 from app.core.auth import require_admin
 from app.db import get_db
 from app.models import SteeringNote
 from app.schemas import SteeringNote as SteeringNoteOut
 from app.schemas import SteeringNoteIn, SteeringNoteList
 router = APIRouter(
    prefix="/steering",
    tags=["steering"],
    dependencies=[Depends(require_admin)],  # phase 16: tuning is admin-only
 )
 def load_steering_notes(db: Session) -> list[str]:
    """All steering notes, oldest first (the order they are numbered in the
    ``<tuning>`` prompt section). Used by the chat turn (``app.api.chat``)."""
    rows = db.scalars(
        select(SteeringNote).order_by(SteeringNote.created_at.asc(), SteeringNote.id.asc())
    ).all()
    return [row.note for row in rows]
@router.get("", response_model=SteeringNoteList)
 def list_steering_notes(
    db: Session = Depends(get_db),  # noqa: B008
 ) -> SteeringNoteList:
    """All notes, newest first (the UI panel's display order)."""
    rows = db.scalars(
        select(SteeringNote).order_by(SteeringNote.created_at.desc(), SteeringNote.id.desc())
    ).all()
    return SteeringNoteList(
        notes=[
            SteeringNoteOut(id=row.id, note=row.note, created_at=row.created_at) for row in rows
        ]
    )
@router.post("", response_model=SteeringNoteOut, status_code=201)
 def create_steering_note(
    payload: SteeringNoteIn,
    db: Session = Depends(get_db),  # noqa: B008
 ) -> SteeringNoteOut:
    """Store one tuning instruction (trimmed, 1–2000 chars — 422 otherwise)."""
    row = SteeringNote(note=payload.note)
    db.add(row)
    db.commit()
    db.refresh(row)
    return SteeringNoteOut(id=row.id, note=row.note, created_at=row.created_at)
@router.delete("/{note_id}", status_code=204)
 def delete_steering_note(
    note_id: uuid.UUID,
    db: Session = Depends(get_db),  # noqa: B008
 ) -> Response:
    """Remove a note; 404 when the id is unknown."""
    row = db.get(SteeringNote, note_id)
    if row is None:
        raise HTTPException(status_code=404, detail="steering note not found")
    db.delete(row)
    db.commit()
    return Response(status_code=204)
@@ -8,8 +8,15 @@ from __future__ import annotations
 import os
 from functools import lru_cache
 from pydantic import field_validator
 from pydantic_settings import BaseSettings, SettingsConfigDict
 #: The A9 import formats (PLAN anchor A9, revised 2026-08-21).
 #: ``BOR_IMPORT_EXTENSIONS`` may narrow — but never widen — this set.
 _ALLOWED_IMPORT_EXTENSIONS: frozenset[str] = frozenset(
    {"md", "markdown", "txt", "yaml", "yml", "json", "py"}
 )
 class Settings(BaseSettings):
    model_config = SettingsConfigDict(
@@ -34,16 +41,87 @@ class Settings(BaseSettings):
    llm_api_key: str = ""
    llm_chat_model: str = "turbo"
    llm_embed_model: str = "embed"
    #: Operator kill-switch for the ``thinking`` SSE events (phase 17,
    #: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
    #: pieces are still counted for the per-turn log line but never
    #: emitted — the answer stream itself is unchanged.
    stream_thinking: bool = True
    # --- RAG tuning ---
    embedding_dim: int = 768  # verified against aipi /v1 (embed model)
    top_k_chunks: int = 4
    top_n_docs: int = 2
-    relevance_threshold: float = 0.30
+    # Honesty gate (A8, re-tuned 2026-08-21): the ``embed`` model's cosine
    # scores compress into 0.41–0.84 on the real corpus, so the old 0.30
    # default never discriminated. LOW only fires when best cosine < this
    # AND no candidate chunk matches the question lexically (see A8).
    relevance_threshold: float = 0.62
    max_context_chars: int = 24_000
    #: Maximum output tokens a chat answer may use (owner instruction
    #: 2026-08-22: answers must run to their natural end — the old hard
    #: 700-token cap cut long answers off mid-sentence).
    max_output_tokens: int = 32_768
    chunk_target_chars: int = 2_000
    chunk_overlap_chars: int = 200
    embed_batch_size: int = 16
    #: Total char budget for the ``<tuning>`` section of the system prompt
    #: (phase 15, steering notes). The newest-fitting notes are kept and the
    #: overflow is replaced by the ``[…truncated…]`` marker.
    steering_max_chars: int = 8_000
    # --- Hybrid retrieval (A7, revised 2026-08-21) ---
    # cosine top-N ∪ Postgres FTS top-N, fused with Reciprocal Rank Fusion
    # (score = Σ 1/(rrf_k + rank) over the lists a chunk appears in).
    #
    # The vector window is deliberately wider than the lexical one: a
    # name-your-tool question's best *lexical* chunk (e.g. the "Install"
    # section of gitlab.md) can sit far down the vector ranking because the
    # question embeds close to generic templates. A 100-wide window is what
    # lets such chunks double-hit (one RRF term per list) and outrank a
    # template that owns vector rank 1 — measured 2026-08-22 against the
    # live 2774-chunk KB for "How did I install gitlab?" (gitlab.md:1 at
    # vrank 100 / lrank 3 → fused 0.0221 vs the template's 0.0164).
    hybrid_vector_candidates: int = 100
    hybrid_lexical_candidates: int = 30
    rrf_k: int = 60
    # --- Admin & sign-in (phase 16; A10 revised 2026-08-22) ---
    # Single-admin auth via a signed session cookie (Starlette
    # SessionMiddleware — no new services, no DB tables). Both secrets are
    # REQUIRED at startup: ``create_app()`` refuses to boot when either is
    # empty (``app.core.auth.ensure_admin_configured``). The password is
    # plaintext on purpose (homelab scope, owner decision 2026-08-22);
    # the session secret signs the cookie (``secrets.token_hex(32)``).
    admin_password: str = ""
    session_secret: str = ""
    #: Signed-cookie lifetime in seconds (default 12 h, refreshed on
    #: session writes — sliding for an active admin).
    session_max_age: int = 43_200
    session_cookie: str = "bor_session"
    # --- Import scope (A9, revised 2026-08-21) ---
    # Comma-separated list of lowercased file extensions (no dot) imported
    # by ``scripts/import_docs.py``. Hidden (dot) path components are always
    # skipped, plus the importer's exclusion list.
    # Stored as a raw CSV string (env-native — no JSON) and parsed on demand
    # via :py:meth:`import_extension_set`. ``mode="after"`` validation runs
    # against the raw string so a typo fails loudly at startup.
    import_extensions: str = "md,markdown,txt,yaml,yml,json,py"
    @field_validator("import_extensions")
    @classmethod
    def _import_extensions_known(cls, v: str) -> str:
        """Reject unknown/empty formats loudly instead of silently importing
        nothing (a typo like ``md,jsonn`` would otherwise walk zero files)."""
        exts = {part.strip().lstrip(".").lower() for part in v.split(",") if part.strip()}
        if not exts:
            raise ValueError("import_extensions must name at least one format")
        unknown = exts - _ALLOWED_IMPORT_EXTENSIONS
        if unknown:
            raise ValueError(
                f"unknown import extension(s): {', '.join(sorted(unknown))} — "
                f"allowed: {', '.join(sorted(_ALLOWED_IMPORT_EXTENSIONS))}"
            )
        return v
    # Suggested questions (onboarding + empty state).
    suggestions: list[str] = [
@@ -53,6 +131,15 @@ class Settings(BaseSettings):
        "What's currently running in the homelab?",
    ]
    @property
    def import_extension_set(self) -> frozenset[str]:
        """Lowercased, dotted extension set (``.md``) for path filtering."""
        return frozenset(
            f".{part.strip().lstrip('.').lower()}"
            for part in self.import_extensions.split(",")
            if part.strip()
        )
    @property
    def effective_api_key(self) -> str:
        """API key for aipi: explicit setting, then $AIPI_KEY, then a placeholder."""
@@ -0,0 +1,89 @@
 """Single-admin authentication (phase 16; LOCKED A10 revised 2026-08-22).
 One admin (the owner), one plaintext password, one signed cookie. The
 mechanism is Starlette's ``SessionMiddleware`` (itsdangerous-signed cookie
 — no server-side store, no new services, no DB tables): the public API
 stays stateless, the cookie is the *only* session state.
 Contract:
 * ``ensure_admin_configured`` — fail-loud startup gate: the app must name
  the missing ``BOR_`` variable(s) instead of serving anything.
 * ``check_password`` — constant-time compare; exactly one generic 401
  message (no user enumeration, there is no second user).
 * ``require_admin`` — FastAPI dependency; anonymous callers get 403
  ``{"detail": "admin only"}`` (used by ``GET /api/docs`` and the whole
  ``/api/steering`` router).
 * ``sign_in`` / ``sign_out`` — session-dict helpers for the API routes.
 """
 from __future__ import annotations
 import secrets
 from collections.abc import MutableMapping
 from typing import Any
 from fastapi import HTTPException, Request
 from app.config import Settings
 #: The session key the single admin is stored under.
 ADMIN_SESSION_KEY = "admin"
 def ensure_admin_configured(settings: Settings) -> None:
    """Refuse to boot without admin auth (fail-loud, A6 spirit).
    Raises :class:`RuntimeError` naming every missing ``BOR_`` variable so
    the startup log tells the operator exactly what to set.
    """
    missing = [
        env_name
        for env_name, value in (
            ("BOR_ADMIN_PASSWORD", settings.admin_password),
            ("BOR_SESSION_SECRET", settings.session_secret),
        )
        if not value.strip()
    ]
    if missing:
        raise RuntimeError(
            "Brain of Reese cannot start: admin auth is not configured. "
            f"Set the missing variable(s): {', '.join(missing)} "
            "(see .env.example and the README 'Admin & sign-in' section)."
        )
 def check_password(candidate: str, expected: str) -> bool:
    """Constant-time password check (``secrets.compare_digest``).
    One admin → one generic 401 on any mismatch: the response never reveals
    whether the password was *close*, empty, or not (no user enumeration).
    """
    return secrets.compare_digest(candidate.encode("utf-8"), expected.encode("utf-8"))
 def require_admin(request: Request) -> None:
    """FastAPI dependency: allow the signed-in admin, else 403.
    Reads the cookie-backed session installed by ``SessionMiddleware``;
    a request without a valid admin session gets 403 ``admin only``.
    """
    if not request.session.get(ADMIN_SESSION_KEY):
        raise HTTPException(status_code=403, detail="admin only")
 def sign_in(session: MutableMapping[str, Any]) -> None:
    """Mark the (cookie-backed) session as the single admin.
    Writing the key marks the session modified, so the middleware emits
    the signed ``bor_session`` cookie with the configured Max-Age.
    """
    session[ADMIN_SESSION_KEY] = True
 def sign_out(session: MutableMapping[str, Any]) -> None:
    """Clear the session state (the route additionally expires the cookie).
    An emptied dict is not re-persisted by the middleware, so the API
    route pairs this with ``response.delete_cookie`` to make the browser
    drop the signed cookie immediately.
    """
    session.clear()
@@ -4,6 +4,11 @@ Boots logging + conditional debugpy, then creates the FastAPI app:
 API routes first (so they win over the catch-all), and the static frontend
 mounted last. No CDN: everything the browser needs is served by this
 process from local files (see PLAN §UI/UX — No External Dependencies).
 Phase 16 (A10 revised): before anything is served, admin auth must be
 configured (fail-loud), and the app wraps every route in Starlette's
 SessionMiddleware — a signed ``bor_session`` cookie is the only session
 state in the system.
 """
 from __future__ import annotations
@@ -12,12 +17,16 @@ from pathlib import Path
 from fastapi import FastAPI
 from fastapi.staticfiles import StaticFiles
 from starlette.middleware.sessions import SessionMiddleware
 from app.api.auth import router as auth_router
 from app.api.chat import router as chat_router
 from app.api.docs import router as docs_router
 from app.api.health import router as health_router
 from app.api.steering import router as steering_router
 from app.api.suggestions import router as suggestions_router
 from app.config import get_settings
 from app.core.auth import ensure_admin_configured
 from app.core.debugging import configure_debugging
 from app.core.logging import configure_logging
@@ -29,13 +38,32 @@ logger = logging.getLogger("app")
 def create_app() -> FastAPI:
    # Fail loud BEFORE serving anything (phase 16): missing
    # BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET raises at boot, naming the
    # variable(s) — the app never starts in a half-authenticated state.
    ensure_admin_configured(settings)
    app = FastAPI(title=settings.app_name, version=settings.app_version)
    # Signed single-admin session cookie (Starlette middleware, itsdangerous
    # signer — no server-side store, no new services). Homelab HTTP: same_site
    # is "lax" and https_only stays off (documented in the README).
    app.add_middleware(
        SessionMiddleware,
        secret_key=settings.session_secret,
        session_cookie=settings.session_cookie,
        max_age=settings.session_max_age,
        same_site="lax",
        https_only=False,
    )
    # API routes first so they take precedence over the catch-all static mount.
    app.include_router(health_router, prefix="/api")
    app.include_router(auth_router, prefix="/api")
    app.include_router(suggestions_router, prefix="/api")
    app.include_router(docs_router, prefix="/api")
    app.include_router(chat_router, prefix="/api")
    app.include_router(steering_router, prefix="/api")
    static_dir = Path(settings.static_dir).resolve()
    if static_dir.is_dir():
@@ -2,12 +2,14 @@
 Data model — see ``.agent/PLAN.md`` §Data Model:
-* ``documents`` — one row per ``*.md`` file (full content, path, sha256 hash).
+* ``documents`` — one row per imported A9 file (full content, path, sha256 hash).
 * ``chunks``    — retrieval units; each chunk points at its parent document
                  via ``document_id``. This is how an embedding maps back to
                  a document path (the "feed the whole document" requirement).
-* ``query_log`` — observability: every question, its retrieval score, the
+* ``query_log``     — observability: every question, its retrieval score,
-                  deflection decision, and latency.
+                      the deflection decision, and latency.
 * ``steering_notes`` — owner tuning notes injected into the system prompt
                      of every chat turn (phase 15, ``<tuning>`` section).
 """
 from __future__ import annotations
@@ -74,8 +76,26 @@ class QueryLog(Base):
    id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
    question: Mapped[str] = mapped_column(Text)
    top_score: Mapped[float] = mapped_column(Float, default=0.0)  # best cosine similarity
    #: Lexical (FTS) candidates matched — the OR-tsquery hit count (A8). NULL
    #: for pre-hybrid rows (migration 0002).
    fts_hits: Mapped[int | None] = mapped_column(Integer)
    chunk_hits: Mapped[int] = mapped_column(Integer, default=0)
    deflected: Mapped[bool] = mapped_column(Boolean, default=False)  # True = honest "no idea"
    sources: Mapped[str] = mapped_column(Text, default="")  # comma-joined source paths
    latency_ms: Mapped[int] = mapped_column(Integer, default=0)
    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
 class SteeringNote(Base):
    """One owner tuning instruction (phase 15).
    Notes are read into the system prompt of **every** chat turn as the
    ``<tuning>`` section (oldest first, char-budgeted — see
    :func:`app.rag.prompts.build_steering_section`).
    """
    __tablename__ = "steering_notes"
    id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
    note: Mapped[str] = mapped_column(Text)  # trimmed, 1–2000 chars (API-enforced)
    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
@@ -1,25 +1,36 @@
-"""Markdown-aware chunker (PLAN §5 chunking policy).
+"""Format-aware chunker (PLAN §5 chunking policy).
-Pure functions, no I/O — fully unit-testable.
+Pure functions, no I/O — fully unit-testable. Stdlib only.
-Policy
+:func:`chunk_document` dispatches on the file's lowercased suffix;
------
+per-format policies:
-* **Sections** are split on ATX headings of level ≥ 2 (``## ``/``### ``/…).
+
-* A section that fits in ``target_chars`` becomes a single chunk.
+* **md / markdown** — sections are split on ATX headings of level ≥ 2
-* A longer section is sub-split at paragraph boundaries (blank lines outside
+  (``## ``/``### ``/…); a section that fits in ``target_chars`` becomes a
-  code fences); each chunk after the first starts with the trailing
+  single chunk, a longer one is sub-split at paragraph boundaries (blank
-  ``overlap_chars`` of the previous chunk so context survives the cut.
+  lines outside code fences) with ``overlap_chars`` carry-over, and every
-* Every chunk keeps its nearest preceding heading line (the section anchor),
+  chunk keeps its nearest preceding heading line (the section anchor). Code
-  so a retrieval hit is always readable in context.
+  fences are atomic (a boundary never falls inside one) except a fence
-* **Code fences** (``` / ~~~) are atomic: a chunk boundary never falls
+  larger than :data:`HARD_MAX_CHARS`, which is split by line.
-  inside one, and lines inside a fence are never mistaken for headings or
+* **yaml / yml** — blocks start at ``---`` document separators and at
-  paragraph breaks. One exception: a fence *larger than* :data:`HARD_MAX_CHARS`
+  top-level (indent-0) ``key:`` lines; every chunk keeps its key lines as
-  is split by line, because aipi's local embedding model rejects requests
+  anchors, so a hit is always readable in context.
-  over ~1024 input tokens and a single 5000-char code block would blow
+* **json** — pretty-printed (``json.dumps(obj, indent=2)``) and split on
-  past that on its own.
+  top-level keys (one ``{key: value}`` block per key); unparseable input
  falls back to paragraph packing.
 * **py** — split at top-level defs/classes via the stdlib ``ast`` (the
  module preamble — imports, constants — is its own block); an oversized
  definition falls back to line packing.
 * **txt** (and any unknown suffix) — paragraph packing.
 Every format honors :data:`HARD_MAX_CHARS` (1200 — the aipi ~1024-token
 request cap) and the target/overlap settings; oversized blocks are split
 by line so no chunk can exceed the cap.
 """
 from __future__ import annotations
 import ast
 import json
 import re
 from collections.abc import Sequence
@@ -203,3 +214,147 @@ def chunk_markdown(
    for start, end in _section_ranges(lines, flags):
        chunks.extend(_chunk_section(lines[start:end], flags[start:end], target, overlap))
    return [c for c in chunks if c.strip()]
 # ---------------------------------------------------------------------------
 # Non-markdown formats (A9, revised 2026-08-21): yaml/yml, json, py, txt
 # ---------------------------------------------------------------------------
 #: YAML document separator (column 0).
 _YAML_DOC_SEP_RE = re.compile(r"^-{3,}\s*$")
 #: Top-level YAML key (column 0, no leading whitespace) — the block anchor.
 _YAML_KEY_RE = re.compile(r"^[A-Za-z0-9_.\-]+\s*:")
 def _yaml_blocks(lines: Sequence[str]) -> list[str]:
    """Group YAML lines into blocks: ``---`` separators and indent-0
    ``key:`` lines each start a new block (the key line stays the anchor)."""
    blocks: list[str] = []
    cur: list[str] = []
    for line in lines:
        if cur and (_YAML_DOC_SEP_RE.match(line) or _YAML_KEY_RE.match(line)):
            blocks.append("\n".join(cur))
            cur = []
        cur.append(line)
    if cur:
        blocks.append("\n".join(cur))
    return [b for b in blocks if b.strip()]
 def chunk_yaml(content: str, target_chars: int = 2000, overlap_chars: int = 200) -> list[str]:
    """Split YAML on document separators + top-level keys (see module docstring)."""
    target, overlap = _normalize_target_overlap(target_chars, overlap_chars)
    return _pack_blocks(_yaml_blocks(content.splitlines()), target, overlap)
 def _json_blocks(content: str) -> list[str] | None:
    """Pretty-printed per-top-level-key blocks, or ``None`` if unparseable."""
    try:
        obj = json.loads(content)
    except (ValueError, TypeError):
        return None
    if isinstance(obj, dict):
        return [json.dumps({k: v}, indent=2) for k, v in obj.items()]
    # Top-level list/scalar: nothing to key on — one pretty-printed block.
    return [json.dumps(obj, indent=2)]
 def chunk_json(content: str, target_chars: int = 2000, overlap_chars: int = 200) -> list[str]:
    """Pretty-print JSON and split on top-level keys (unparseable → paragraphs)."""
    target, overlap = _normalize_target_overlap(target_chars, overlap_chars)
    blocks = _json_blocks(content)
    if blocks is None:
        return chunk_text(content, target, overlap)
    return _pack_blocks(blocks, target, overlap)
 def _python_blocks(content: str) -> list[str] | None:
    """Line blocks: module preamble, then one per top-level def/class.
    Returns ``None`` when the source does not parse (→ line/paragraph
    packing fallback).
    """
    try:
        tree = ast.parse(content)
    except (SyntaxError, ValueError):
        return None
    lines = content.splitlines()
    tops = [
        node
        for node in tree.body
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))
    ]
    tops.sort(key=lambda n: n.lineno)
    ranges: list[tuple[int, int]] = []
    for node in tops:
        start = node.lineno - 1
        for dec in node.decorator_list:
            start = min(start, dec.lineno - 1)
        end = node.end_lineno or node.lineno  # end_lineno is None on odd parses
        ranges.append((start, end))  # 0-based start, 1-based end
    blocks: list[str] = []
    cursor = 0
    for start, end in ranges:
        if start > cursor:
            blocks.append("\n".join(lines[cursor:start]))
        blocks.append("\n".join(lines[start:end]))
        cursor = end
    if cursor < len(lines):
        blocks.append("\n".join(lines[cursor:]))
    return [b for b in blocks if b.strip()]
 def chunk_python(content: str, target_chars: int = 2000, overlap_chars: int = 200) -> list[str]:
    """Split Python on top-level defs/classes (stdlib ``ast``; see module doc)."""
    target, overlap = _normalize_target_overlap(target_chars, overlap_chars)
    blocks = _python_blocks(content)
    if blocks is None:
        return chunk_text(content, target, overlap)
    return _pack_blocks(blocks, target, overlap)
 def chunk_text(content: str, target_chars: int = 2000, overlap_chars: int = 200) -> list[str]:
    """Plain-text paragraph packing (blank lines separate paragraphs)."""
    target, overlap = _normalize_target_overlap(target_chars, overlap_chars)
    lines = content.splitlines()
    return _pack_blocks(_paragraph_blocks(lines, [False] * len(lines)), target, overlap)
 def _normalize_target_overlap(target_chars: int, overlap_chars: int) -> tuple[int, int]:
    """Validate + clamp the size policy (shared by every format)."""
    if target_chars <= 0:
        raise ValueError("target_chars must be > 0")
    if overlap_chars < 0:
        raise ValueError("overlap_chars must be >= 0")
    # The endpoint's token cap is absolute — a larger target is unsafe.
    target = min(target_chars, HARD_MAX_CHARS)
    return target, min(overlap_chars, target - 1)
 #: suffix → chunker (A9, revised: md, markdown, txt, yaml, yml, json, py).
 _FORMAT_CHUNKERS = {
    ".md": chunk_markdown,
    ".markdown": chunk_markdown,
    ".txt": chunk_text,
    ".yaml": chunk_yaml,
    ".yml": chunk_yaml,
    ".json": chunk_json,
    ".py": chunk_python,
 }
 def chunk_document(
    content: str,
    path: str,
    target_chars: int = 2000,
    overlap_chars: int = 200,
 ) -> list[str]:
    """Chunk *content* according to *path*'s lowercased suffix.
    Unknown suffixes fall back to plain-text paragraph packing (the
    importer only passes A9-format files, so this is belt-and-braces).
    """
    name = path.rsplit("/", 1)[-1]
    suffix = "." + name.rsplit(".", 1)[-1].lower() if "." in name else ""
    chunker = _FORMAT_CHUNKERS.get(suffix, chunk_text)
    return chunker(content, target_chars, overlap_chars)
@@ -1,23 +1,30 @@
 """Knowledge-base importer (PLAN §5 / §9 / §11).
-Walks ``*.md`` files (A9 exclusion list), diffs by sha256 against
+Walks the A9-format files (``md, markdown, txt, yaml, yml, json, py`` by
-``documents.content_hash`` and, for every new or changed file, runs the
+default — ``BOR_IMPORT_EXTENSIONS``; case-insensitive), diffs by sha256
-two-phase upsert:
+against ``documents.content_hash`` and, for every new or changed file, runs
 the two-phase upsert:
 1. upsert the document row and replace its chunk rows (embeddings NULL)
 2. embed the new chunks in batches and attach the vectors
 3. commit — one transaction per file, so a failed embedding leaves the
   database untouched and the file is simply retried on the next run
 Scope (A9, revised 2026-08-21): any path containing a dot-prefixed
 component (hidden dirs — vendored caches like ``.esphome/.espressif/**`` —
 or hidden files) is skipped, plus the well-known exclusion list.
 ``prune=True`` deletes documents (of the imported sources only) whose files
-no longer exist. Per-file logging uses the verbs
+no longer exist **or no longer match the format filter** — this is how
-``added | updated | unchanged | pruned`` plus a summary line (PLAN §9).
+previously-imported junk (e.g. dot-dir READMEs) leaves the index. Per-file
 logging uses the verbs ``added | updated | unchanged | pruned`` plus a
 summary line with per-format counts (PLAN §9).
 """
 from __future__ import annotations
 import hashlib
 import logging
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from datetime import UTC, datetime
 from pathlib import Path
 from typing import Protocol
@@ -28,7 +35,7 @@ from sqlalchemy.orm import Session
 from app.config import Settings
 from app.db import SessionLocal
 from app.models import Chunk, Document
-from app.rag.chunker import chunk_markdown, extract_title
+from app.rag.chunker import chunk_document, extract_title
 from app.rag.llm import EmbeddingError
 logger = logging.getLogger("app.importer")
@@ -60,11 +67,20 @@ class ImportSummary:
    errors: int = 0
    chunks: int = 0
    embed_batches: int = 0
    #: Files walked, keyed by lowercased extension (``md``, ``yaml``, …).
    formats: dict[str, int] = field(default_factory=dict)
    def format_counts(self) -> str:
        """``md:203,yaml:267,py:14`` — highest count first (PLAN §9)."""
        if not self.formats:
            return "none"
        ordered = sorted(self.formats.items(), key=lambda kv: (-kv[1], kv[0]))
        return ",".join(f"{ext}:{count}" for ext, count in ordered)
    def log(self) -> None:
        logger.info(
            "import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
-            "errors=%d chunks=%d embed_batches=%d",
+            "errors=%d chunks=%d embed_batches=%d formats=%s",
            self.files,
            self.added,
            self.updated,
@@ -73,17 +89,32 @@ class ImportSummary:
            self.errors,
            self.chunks,
            self.embed_batches,
            self.format_counts(),
        )
-def iter_markdown_files(root: Path, excluded: frozenset[str] = EXCLUDED_DIRS) -> list[Path]:
+def iter_importable_files(
-    """All ``*.md`` files under *root* (sorted), skipping excluded dirs (A9)."""
+    root: Path,
    extensions: frozenset[str],
    excluded: frozenset[str] = EXCLUDED_DIRS,
 ) -> list[Path]:
    """All importable files under *root* (sorted), per the A9 scope rules.
    *extensions* is a set of lowercased dotted suffixes (``{'.md', '.py'}``).
    Skips: any path with a dot-prefixed component (hidden dirs/files —
    vendored caches like ``.esphome/.espressif/**``) and the well-known
    non-content directories in *excluded*.
    """
    if not root.is_dir():
        return []
    files: list[Path] = []
-    for path in sorted(root.rglob("*.md")):
+    for path in sorted(root.rglob("*")):
        if not path.is_file():
            continue
        rel = path.relative_to(root)
-        if any(part in excluded for part in rel.parts[:-1]):
+        if any(part.startswith(".") or part in excluded for part in rel.parts):
            continue
        if path.suffix.lower() not in extensions:
            continue
        files.append(path)
    return files
@@ -97,7 +128,7 @@ async def import_sources(
    limit: int | None = None,
    session: Session | None = None,
 ) -> ImportSummary:
-    """Import every ``*.md`` under *sources* (see module docstring).
+    """Import every A9-format file under *sources* (see module docstring).
    ``session`` may be supplied (tests); a private one is opened and closed
    otherwise. ``limit`` caps the number of files processed (debug only) and
@@ -120,12 +151,14 @@ async def import_sources(
                break
            source = root.name
            source_names.add(source)
-            for path in iter_markdown_files(root):
+            for path in iter_importable_files(root, llm.settings.import_extension_set):
                if limit is not None and summary.files >= limit:
                    break
                rel = path.relative_to(root).as_posix()
                seen.add((source, rel))
                summary.files += 1
                ext = path.suffix.lower().lstrip(".") or "unknown"
                summary.formats[ext] = summary.formats.get(ext, 0) + 1
                try:
                    await _index_file(
                        session, source=source, rel=rel, full_path=path, llm=llm,
@@ -172,7 +205,12 @@ async def _index_file(
        return
    verb = "updated" if doc is not None else "added"
    # A ``#`` line is a real heading in markdown but a comment in every
    # other format — titles for those come from the file stem.
    if full_path.suffix.lower() in (".md", ".markdown"):
        title = extract_title(content, fallback=full_path.stem)
    else:
        title = full_path.stem
    if doc is None:
        doc = Document(
            source=source,
@@ -200,7 +238,7 @@ async def _index_file(
    # policy stays intact for the rest of the KB.
    target = max(400, settings.chunk_target_chars)
    while True:
-        chunks_text = chunk_markdown(content, target, settings.chunk_overlap_chars)
+        chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
        doc.chunks = [
            Chunk(document_id=doc.id, position=i, content=c) for i, c in enumerate(chunks_text)
        ]
@@ -1,7 +1,12 @@
 """Async OpenAI-compatible client for the self-hosted aipi endpoint (PLAN A5).
 Provides the embeddings surface (importer, retrieval) and chat streaming
-(PLAN A15) for the RAG pipeline.
+(PLAN A15) for the RAG pipeline. Chat streaming yields typed
 :class:`StreamPiece` values (phase 17): aipi's ``turbo`` model streams
 its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
 wire convention, verified live 2026-08-23) **before** the answer's
 ``delta.content`` chunks, and reasoning counts against ``max_tokens``
 (an answer can in principle be empty).
 Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
 dimensions when the table is created, so a model that returns a different
@@ -12,7 +17,8 @@ from __future__ import annotations
 import logging
 from collections.abc import AsyncIterator
-from typing import cast
+from dataclasses import dataclass
 from typing import Literal, cast
 from openai import AsyncOpenAI
 from openai.types.chat import ChatCompletionMessageParam
@@ -34,6 +40,19 @@ class LLMError(RuntimeError):
    """The chat-completions endpoint failed (network, HTTP, or mid-stream)."""
@dataclass(frozen=True)
 class StreamPiece:
    """One piece of a streamed chat turn (phase 17, PLAN §4 extension).
    ``kind`` is ``"content"`` for answer text (an SSE ``delta`` frame)
    or ``"thinking"`` for the model's reasoning (an SSE ``thinking``
    frame). Frozen: pieces are immutable wire values, not accumulators.
    """
    kind: Literal["content", "thinking"]
    text: str
 # aipi's local embedding model rejects requests over ~1024 input tokens
 # ("input is too large to process"). Batch by estimated tokens, with a
 # safety margin under that cap — code-dense text can tokenize at ~3
@@ -168,13 +187,30 @@ class LLMClient:
        (vec,) = await self.embed([text])
        return vec
-    async def chat_stream(self, messages: list[dict[str, str]]) -> AsyncIterator[str]:
+    async def chat_stream(
-        """Stream assistant text deltas from the chat model (PLAN A5/A15).
+        self, messages: list[dict[str, str]]
    ) -> AsyncIterator[StreamPiece]:
        """Stream assistant pieces from the chat model (PLAN A5/A15, phase 17).
-        ``stream=True`` against the OpenAI-compatible endpoint; yields only
+        ``stream=True`` against the OpenAI-compatible endpoint, yielding
-        non-empty ``delta.content`` pieces. Any failure (network, HTTP,
+        typed :class:`StreamPiece` values. Wire convention (verified live
-        malformed stream) surfaces as :class:`LLMError` so the API layer can
+        against aipi's ``turbo`` on 2026-08-23): the model's reasoning
-        turn it into an SSE ``error`` event instead of a hung request.
+        arrives as ``delta.reasoning_content`` chunks (deepseek/litellm
        convention) **before** the first ``delta.content`` chunk, so in
        practice thinking pieces precede content pieces. The ``openai``
        SDK keeps unknown delta fields in ``model_extra``, so ``getattr``
        is the right accessor — no raw-HTTP parsing is needed. A chunk
        carrying both fields yields the thinking piece **first**.
        Reasoning counts against ``max_tokens``: an answer can in principle
        be empty (thinking with no content) — the UI handles that.
        Answers are allowed up to ``BOR_MAX_OUTPUT_TOKENS`` (default
        32 768) output tokens — the old hard 700-token cap cut long
        answers off mid-sentence (owner report 2026-08-22).
        Any failure (network, HTTP, malformed stream) surfaces as
        :class:`LLMError` so the API layer can turn it into an SSE
        ``error`` event instead of a hung request.
        """
        try:
            # ``{role, content}`` dicts are exactly what the message params
@@ -183,15 +219,23 @@ class LLMClient:
                model=self.settings.llm_chat_model,
                messages=cast("list[ChatCompletionMessageParam]", messages),
                temperature=0.4,
-                max_tokens=700,
+                max_tokens=self.settings.max_output_tokens,
                stream=True,
            )
            async for chunk in stream:
                if not chunk.choices:
                    continue
-                piece = chunk.choices[0].delta.content
+                delta = chunk.choices[0].delta
-                if piece:
+                reasoning = getattr(delta, "reasoning_content", None)
-                    yield piece
+                if not reasoning:
                    # Future-proofing: the same wire convention under a
                    # shorter field name.
                    reasoning = getattr(delta, "reasoning", None)
                if reasoning:
                    yield StreamPiece("thinking", reasoning)
                content = delta.content
                if content:
                    yield StreamPiece("content", content)
        except LLMError:
            raise
        except Exception as e:  # noqa: BLE001 — wrap transport-level failures
@@ -1,24 +1,35 @@
 """Locked system-prompt builder (PLAN §6).
 The persona + HONESTY GATE text is **locked verbatim** — change it through
-the plan, not here. Two modes:
+the plan, not here. (PLAN §6 revision, 2026-08-22: the owner's working-tree
 persona edits are preserved — no mandated ``"you've got this"`` tagline and
 no mandated deflection opening; the honesty gate itself is unchanged.)
 Two modes:
 * ``HIGH`` — grounded turn: full top-document texts under ``<documents>``.
 * ``LOW``  — deflection turn: weak-hit *titles only* plus the
  ``DEFLECT_MODE`` marker (the E2E mock LLM keys on that marker).
 Steering (phase 15): when the owner has stored tuning notes, both modes
 carry a ``<tuning>`` section between ``<relevance>…</relevance>`` and the
 mode body. With zero notes the prompt is byte-identical to the
 pre-steering text.
 """
 from __future__ import annotations
 from collections.abc import Sequence
 from app.config import get_settings
 from app.models import Document
 from app.rag.retriever import TRUNCATION_MARKER
 #: PLAN §6 verbatim (line wrapping included); ``{relevance}`` is filled by
 #: :func:`_base`.
 PERSONA: str = (
    'You are "Brain of Reese" — the digital brain of Reese, a self-hoster and\n'
    "homelab tinkerer. Personality: chippy, upbeat, warm, and genuinely\n"
-    'optimistic about the user\'s ability to do things ("you\'ve got this").\n'
+    "optimistic about the user's ability to do things.\n"
    "\n"
    "Rules:\n"
    "1. Answer ONLY from the provided document context. Cite which document(s)\n"
@@ -26,14 +37,20 @@ PERSONA: str = (
    "2. Be concrete: names, versions, ports, hosts, schedules — the specifics in\n"
    "   the docs are the value.\n"
    '3. HONESTY GATE: if <relevance> is "LOW", you must NOT pretend to know.\n'
-    '   Start your answer with a variant of: "I haven\'t done anything like that."\n'
+    "   Offer 2-3 alternative questions about things you DO have notes on.\n"
    "   Then offer 2-3 alternative questions about things you DO have notes on.\n"
    "4. Never invent facts, hosts, or steps that are not in the context.\n"
    "5. Keep answers tight: short paragraphs, bullets where helpful.\n"
    "\n"
    "<relevance>{relevance}</relevance>"
 )
 #: One-line intro of the ``<tuning>`` section (phase 15): the owner's notes
 #: steer the answer and win over the defaults when they conflict.
 _STEERING_INTRO = (
    "The owner of this brain asked you to steer your answers as follows. "
    "Where these instructions conflict with the defaults above, follow the owner:\n"
 )
 def _base(relevance: str) -> str:
    if relevance not in ("HIGH", "LOW"):
@@ -41,8 +58,46 @@ def _base(relevance: str) -> str:
    return PERSONA.replace("{relevance}", relevance)
-def build_high_prompt(documents: Sequence[Document]) -> str:
+def build_steering_section(notes: Sequence[str], max_chars: int | None = None) -> str:
-    """Grounded turn: locked persona + full texts of the top documents."""
+    """The ``<tuning>`` section of the system prompt (phase 15).
    * No notes (or only blank ones) → ``""`` — callers then build the
      prompt exactly as before, so a zero-note prompt is byte-identical to
      the pre-steering text.
    * Otherwise: numbered notes (in the given order — the chat turn passes
      them oldest-first, so #1 is the oldest note) capped at *max_chars*
      (default ``BOR_STEERING_MAX_CHARS``). When the budget cannot hold
      every note, the oldest-fitting prefix is kept and the overflow is
      replaced by the shared ``[…truncated…]`` marker.
    """
    cleaned = [str(n).strip() for n in notes]
    cleaned = [n for n in cleaned if n]
    if not cleaned:
        return ""
    limit = max_chars if max_chars is not None else get_settings().steering_max_chars
    if limit <= 0:
        return ""
    def render(count: int) -> str:
        lines = [f"{i}. {note}" for i, note in enumerate(cleaned[:count], start=1)]
        if count < len(cleaned):
            lines.append(TRUNCATION_MARKER)
        return f"<tuning>\n{_STEERING_INTRO}" + "\n".join(lines) + "\n</tuning>"
    for count in range(len(cleaned), 0, -1):
        rendered = render(count)
        if len(rendered) <= limit:
            return rendered
    # Pathological budget: not even the empty note list fits. The section
    # must still respect the cap — the bare marker when it fits, else none.
    if len(TRUNCATION_MARKER) <= limit:
        return TRUNCATION_MARKER
    return ""
 def build_high_prompt(documents: Sequence[Document], notes: Sequence[str] | None = None) -> str:
    """Grounded turn: locked persona (+ steering) + full texts of the top
    documents."""
    blocks = [
        f'<document source="{doc.source}" path="{doc.path}" title="{doc.title}">\n'
        f"{doc.content}\n"
@@ -52,15 +107,22 @@ def build_high_prompt(documents: Sequence[Document]) -> str:
    body = "\n\n".join(blocks) if blocks else (
        "(no documents matched — do not invent specifics)"
    )
-    return _base("HIGH") + "\n<documents>\n" + body + "\n</documents>"
+    section = build_steering_section(notes or [])
    prompt = _base("HIGH")
    if section:
        prompt += "\n" + section
    return prompt + "\n<documents>\n" + body + "\n</documents>"
-def build_deflect_prompt(titles: Sequence[str]) -> str:
+def build_deflect_prompt(titles: Sequence[str], notes: Sequence[str] | None = None) -> str:
    """Deflection turn: weak-hit titles only (no document content)."""
    weak = "\n".join(f"- {t}" for t in titles) if titles else "(nothing close at all)"
    section = build_steering_section(notes or [])
    mid = f"\n{section}\n" if section else "\n"
    return (
        _base("LOW")
-        + "\nDEFLECT_MODE: retrieval was weak — the titles below are the closest "
+        + mid
        + "DEFLECT_MODE: retrieval was weak — the titles below are the closest "
        "your notes come to the question. They are titles only; do not pretend "
        "they answer it. Use them to propose 2-3 alternative questions.\n"
        + weak
@@ -1,19 +1,31 @@
-"""pgvector cosine retrieval → parent-document mapping (PLAN §3/§6, A7).
+"""Hybrid retrieval: pgvector cosine ∪ Postgres FTS, RRF-fused (PLAN §6, A7).
-Retrieval returns the *chunks* closest to the question embedding (top-K by
+* **Vector list** — top-N chunks by cosine distance (``embedding <=> $1``),
-cosine distance). The product requirement is that the LLM receives the
+  each carrying its cosine ``1 − distance`` (the honesty-gate input).
-**entire relevant document**, not just the chunk (LOCKED A7) — so
+* **Lexical list** — top-N chunks matching an OR-``tsquery`` over the
-:meth:`select_documents` maps chunk hits back to their parent documents
+  question's tokens, ordered by ``ts_rank``. This is what finds
-(``chunks.document_id → documents``), dedupes, ranks by best chunk score,
+  name-your-tool questions ("gitlab") that vector similarity buries.
-and caps the combined context at ``BOR_MAX_CONTEXT_CHARS``.
+* **Fusion** — Reciprocal Rank Fusion (``score = Σ 1/(k + rank)`` over the
  lists a chunk appears in; chunks hit by both lists get both terms). The
  fused score ranks; :meth:`select_documents` and :func:`weak_hit_titles`
  keep working off ``score``.
 The product requirement is unchanged (LOCKED A7): the LLM receives the
 **entire relevant document**, not just the chunk — chunk hits map back to
 their parents, dedupe, rank by best fused score, and the combined context
 is capped at ``BOR_MAX_CONTEXT_CHARS``.
 Deterministic tie-break for equal fused scores:
 ``(−fused, −cosine, document.path, chunk.position)``.
 """
 from __future__ import annotations
 import re
 import uuid
 from collections.abc import Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
-from sqlalchemy import select
+from sqlalchemy import select, text
 from sqlalchemy.orm import Session
 from app.config import get_settings
@@ -22,50 +34,209 @@ from app.models import Chunk, Document
 #: Marker appended when the context budget is exceeded (PLAN §6).
 TRUNCATION_MARKER = "[…truncated…]"
 #: Alphanumeric tokens of a question (``to_tsquery`` input, OR-joined).
 _TOKEN_RE = re.compile(r"[a-z0-9]+")
 #: One row of the lexical candidate query (all fields needed to build a
 #: detached :class:`Document` plus the chunk fields and ``ts_rank``).
 _LEXICAL_SQL = text(
    """
    SELECT c.id            AS chunk_id,
           c.position      AS position,
           c.content       AS content,
           d.id            AS doc_id,
           d.source        AS source,
           d.path          AS path,
           d.full_path     AS full_path,
           d.title         AS title,
           d.content       AS doc_content,
           d.content_hash  AS content_hash,
           d.indexed_at    AS indexed_at,
           ts_rank(c.tsv, to_tsquery('english', :tsquery)) AS rank
    FROM chunks c
    JOIN documents d ON d.id = c.document_id
    WHERE c.tsv @@ to_tsquery('english', :tsquery)
    ORDER BY rank DESC, d.path ASC, c.position ASC
    LIMIT :limit
    """
 )
@dataclass
 class RetrievedChunk:
-    """One chunk hit: its cosine score plus the parent document row."""
+    """One retrieval candidate: fused rank score + parent document row.
    * ``score`` — RRF fused score (the ranking key for document selection
      and weak-hit titles).
    * ``cosine`` — vector similarity ``1 − distance`` (the honesty-gate
      input; ``0.0`` for lexical-only hits that have no vector rank).
    * ``fts_hit`` — the chunk matched the question's OR-tsquery.
    """
    chunk_id: uuid.UUID
    position: int
    content: str
-    score: float  # 1 − cosine_distance (higher is more similar)
+    score: float
    document: Document
    cosine: float = 0.0
    fts_hit: bool = False
-def retrieve(
+def lexical_tsquery(question: str) -> str | None:
-    db: Session, question_embedding: list[float], top_k: int | None = None
+    """OR-joined token string for ``to_tsquery('english', …)``, or ``None``.
 ) -> list[RetrievedChunk]:
    """Top-*top_k* chunks by pgvector cosine distance (``<=>``).
-    ``score = 1 − distance``. Results are ordered by ascending distance, so
+    Tokens are lowercased ``[a-z0-9]+`` runs, de-duplicated in order of
-    index 0 is the best hit. Chunks whose embedding is still NULL (two-phase
+    first appearance. Postgres does the lexing/stemming; a question whose
-    import in progress) are skipped.
+    tokens are all stopwords lexes to an *empty* tsquery (which matches
    nothing), so no special-casing is needed there. Pure-symbol questions
    ("???", "🔧") yield no tokens → ``None`` → no lexical query at all.
    """
    seen: set[str] = set()
    tokens: list[str] = []
    for tok in _TOKEN_RE.findall(question.lower()):
        if tok not in seen:
            seen.add(tok)
            tokens.append(tok)
    return " | ".join(tokens) if tokens else None
 def fuse(
    vector: Sequence[RetrievedChunk],
    lexical: Sequence[RetrievedChunk],
    k: int,
 ) -> list[RetrievedChunk]:
    """Reciprocal Rank Fusion over the two ranked candidate lists.
    ``score(chunk) = Σ 1/(k + rank)`` — one term per list the chunk appears
    in (ranks are 1-based; a chunk in both lists gets both terms). Returns
    the union ordered by ``(-score, -cosine, document.path, position)``.
    Lexical-only hits (no vector rank) enter with ``cosine=0.0`` and
    ``fts_hit=True``; vector chunks matched by the lexical list get
    ``fts_hit=True`` in place (the input objects are mutated — callers
    should not reuse them afterwards).
    """
    if k <= 0:
        raise ValueError("rrf k must be > 0")
    by_id: dict[uuid.UUID, RetrievedChunk] = {}
    fused: dict[uuid.UUID, float] = {}
    for rank, rc in enumerate(vector, start=1):
        by_id[rc.chunk_id] = rc
        fused[rc.chunk_id] = fused.get(rc.chunk_id, 0.0) + 1.0 / (k + rank)
    for rank, rc in enumerate(lexical, start=1):
        term = 1.0 / (k + rank)
        if rc.chunk_id in by_id:
            existing = by_id[rc.chunk_id]
            by_id[rc.chunk_id] = replace(existing, fts_hit=True)
            fused[rc.chunk_id] += term
        else:
            rc = replace(rc, fts_hit=True)
            by_id[rc.chunk_id] = rc
            fused[rc.chunk_id] = fused.get(rc.chunk_id, 0.0) + term
    out = [replace(rc, score=fused[rc.chunk_id]) for rc in by_id.values()]
    out.sort(key=lambda rc: (-rc.score, -rc.cosine, rc.document.path, rc.position))
    return out
 def _vector_candidates(
    db: Session, question_embedding: list[float], limit: int
 ) -> list[RetrievedChunk]:
    """Top-*limit* chunks by pgvector cosine distance (``<=>``).
    ``cosine = 1 − distance``. Chunks whose embedding is still NULL
    (two-phase import in progress) are skipped.
    """
    k = top_k if top_k is not None else get_settings().top_k_chunks
    distance = Chunk.embedding.cosine_distance(question_embedding)
    rows = db.execute(
        select(Chunk, distance.label("distance"), Document)
        .join(Document, Chunk.document_id == Document.id)
        .where(Chunk.embedding.is_not(None))
        .order_by(distance)
-        .limit(k)
+        .limit(limit)
    ).all()
    return [
        RetrievedChunk(
            chunk_id=chunk.id,
            position=chunk.position,
            content=chunk.content,
-            score=round(1.0 - float(dist), 6),
+            score=0.0,  # fused score is filled in by :func:`fuse`
            document=doc,
            cosine=round(1.0 - float(dist), 6),
        )
        for chunk, dist, doc in rows
    ]
 def _lexical_candidates(db: Session, question: str, limit: int) -> list[RetrievedChunk]:
    """Top-*limit* chunks matching the question's OR-tsquery (A7).
    Ordered by ``ts_rank`` (with ``d.path, c.position`` as the
    deterministic tie-break); an empty tsquery (stopword-only question)
    simply matches nothing.
    """
    tsquery = lexical_tsquery(question)
    if tsquery is None:
        return []
    rows = db.execute(
        _LEXICAL_SQL, {"tsquery": tsquery, "limit": limit}
    ).all()
    out: list[RetrievedChunk] = []
    for row in rows:
        doc = Document(
            id=row.doc_id,
            source=row.source,
            path=row.path,
            full_path=row.full_path,
            title=row.title,
            content=row.doc_content,
            content_hash=row.content_hash,
            indexed_at=row.indexed_at,
        )
        out.append(
            RetrievedChunk(
                chunk_id=row.chunk_id,
                position=row.position,
                content=row.content,
                score=0.0,  # filled in by :func:`fuse`
                document=doc,
                cosine=0.0,  # no vector rank — lexical-only hit
                fts_hit=True,
            )
        )
    return out
 def retrieve(
    db: Session,
    question: str,
    question_embedding: list[float],
    vector_candidates: int | None = None,
    lexical_candidates: int | None = None,
 ) -> list[RetrievedChunk]:
    """Hybrid retrieval (A7): vector top-N ∪ FTS top-N, RRF-fused.
    Returns the fused candidate list in rank order (best first). Each
    :class:`RetrievedChunk` carries the fused ``score`` (ranking), the
    ``cosine`` similarity (honesty gate) and the ``fts_hit`` flag.
    """
    settings = get_settings()
    v_n = (
        settings.hybrid_vector_candidates if vector_candidates is None else vector_candidates
    )
    l_n = (
        settings.hybrid_lexical_candidates if lexical_candidates is None else lexical_candidates
    )
    if v_n <= 0:
        raise ValueError("vector_candidates must be >= 1")
    if l_n <= 0:
        raise ValueError("lexical_candidates must be >= 1")
    vector = _vector_candidates(db, question_embedding, v_n)
    lexical = _lexical_candidates(db, question, l_n)
    return fuse(vector, lexical, settings.rrf_k)
 def weak_hit_titles(chunks: Sequence[RetrievedChunk]) -> list[str]:
-    """Distinct parent-document titles of *chunks*, best chunk score first.
+    """Distinct parent-document titles of *chunks*, best fused score first.
    Deflection mode (PLAN §6, A8) is built from these *titles only* — the
    LOW prompt and the "Maybe try" chips never see document content.
@@ -85,7 +256,7 @@ def select_documents(
    n: int | None = None,
    max_chars: int | None = None,
 ) -> list[Document]:
-    """Map chunk hits to distinct parent documents, ranked by best chunk score.
+    """Map chunk hits to distinct parent documents, ranked by best fused score.
    At most *n* documents are returned (default ``BOR_TOP_N_DOCS``). The
    returned rows carry the full document content; if the combined content
@@ -1,7 +1,10 @@
 """Pydantic request/response schemas (API contract)."""
 from __future__ import annotations
-from pydantic import BaseModel, Field
+import uuid
 from datetime import datetime
 from pydantic import BaseModel, Field, field_validator
 class HealthResponse(BaseModel):
@@ -19,12 +22,44 @@ class ChatRequest(BaseModel):
    message: str = Field(min_length=1, max_length=4000)
 class LoginRequest(BaseModel):
    """``POST /api/login`` body (phase 16): the single admin's password.
    An empty or wrong password is a 401 with one generic detail — never a
    422 that would hint at input-shape differences.
    """
    password: str = ""
 class WhoamiResponse(BaseModel):
    """``GET /api/whoami`` (phase 16) — drives all UI gating."""
    authenticated: bool
    role: str  # "admin" | "anonymous"
 class SourceRef(BaseModel):
    source: str
    path: str
    title: str
 class ChatThinkingEvent(BaseModel):
    """SSE thinking event: one chunk of the model's reasoning (phase 17).
    PLAN §4 extension (A15, owner permission 2026-08-23): frames of the
    shape ``{type: "thinking", text: str}`` stream ahead of the
    ``delta`` frames in practice (the model reasons before it answers). The
    client renders them in a collapsible "Thinking" block; the ``done``
    event shape is unchanged and thinking text never travels on it.
    Sibling of :class:`ChatErrorEvent`.
    """
    type: str = "thinking"
    text: str
 class ChatDoneEvent(BaseModel):
    """Final SSE event of a chat turn: metadata for the finished answer."""
@@ -61,3 +96,45 @@ class DocList(BaseModel):
    """Response of ``GET /api/docs`` (empty list → designed empty state)."""
    documents: list[DocSummary]
 class DocContent(BaseModel):
    """One indexed document's full content (feeds the viewer page, phase 10)."""
    source: str
    path: str
    title: str
    format: str
    content: str
    indexed_at: str
    chunks: int
 class SteeringNoteIn(BaseModel):
    """``POST /api/steering`` body: one tuning instruction (phase 15).
    The note is trimmed *before* the length constraints run, so a
    whitespace-only body is a 422 and a 2000-char note with surrounding
    spaces still passes.
    """
    note: str = Field(min_length=1, max_length=2000)
    @field_validator("note", mode="before")
    @classmethod
    def _trim_note(cls, v: object) -> object:
        return v.strip() if isinstance(v, str) else v
 class SteeringNote(BaseModel):
    """One stored steering note (API shape — ISO-8601 ``created_at``)."""
    id: uuid.UUID
    note: str
    created_at: datetime
 class SteeringNoteList(BaseModel):
    """``GET /api/steering`` response: all notes, newest first."""
    notes: list[SteeringNote]
@@ -14,11 +14,51 @@
 *   • thinking  — pre-token: typing dots + disabled "Thinking…" button;
 *                 after 10s the indicator's aria-label shows elapsed
 *                 seconds so screen-reader users are never left guessing.
 *                 Phase 17: while the model streams reasoning (`thinking`
 *                 SSE events), the live collapsible Thinking block IS the
 *                 visible feedback (it replaces the typing dots; the UI
 *                 state stays "thinking" — button still disabled,
 *                 "Thinking…") and the 120s guard clears on the first
 *                 thinking *or* delta event.
 *   • streaming — the first delta removes the dots and appends live into
- *                 the answer bubble; the button stays busy until `done`.
+ *                 the answer bubble (auto-collapsing the Thinking block,
 *                 phase 17); the button stays busy until `done`.
 *   • error     — red banner (role="alert") with an actionable retry hint;
 *                 the 120s guard (TURN_TIMEOUT_MS) catches hung pre-token
- *                 streams, so the button can never sit zombified.
+ *                 streams and the sawDone guard (phase 17) catches a
 *                 stream that dies after frames but before `done`, so the
 *                 button can never sit zombified.
 *
 * Conversation persistence (phase 14) makes the chat a durable LOCAL
 * session: the message list (raw text + turn metadata) lives in
 * localStorage under the versioned key `bor.chat.v1` and is re-rendered on
 * load — refresh, tab close, and a trip to Sources never lose it. Phase
 * 17: a brain record may carry an optional `thinking` field — the
 * collapsed Thinking block is restored with it; records without it (old
 * sessions) restore exactly as before, so no version bump. A10 is
 * untouched: the API stays stateless, nothing is stored server-side.
 * "New chat" (#new-chat-btn) clears the key + the list back to the empty
 * state.
 *
 * Steering notes (phase 15) let the owner tune how Brain answers: a
 * "Tune" button under every completed brain bubble (deflected included)
 * opens an inline form → POST /api/steering → the note is stored in
 * Postgres and injected into the system prompt of every subsequent turn
 * (the <tuning> section). Notes are listed newest-first in the header
 * "Tuning" panel (#steering-panel), where each can be deleted. Note text
 * is always rendered with textContent (XSS-safe), save/delete are
 * announced through a polite live region (#steering-announcer), and the
 * panel + count badge update on every change.
 *
 * Scroll (phase 18, owner choice 2026-08-23): the page auto-scrolls only
 * while the user is pinned to the bottom. NEAR_BOTTOM_PX (200px) covers
 * the composer zone — the textarea auto-grows to 192px plus the button
 * row — so "the composer is in view" counts as pinned: submitting from
 * the composer reveals your own message, and the answer follows token by
 * token while you stay pinned. Once you scroll up to read earlier
 * content, nothing drags the viewport back down for the rest of the turn
 * (thinking or answer). scrollReveal(wrap) is the single scroll gate;
 * `force` is reserved for the one-shot phase-14 restore landing.
 *
 * All DOM ids match frontend/index.html.
 */
@@ -59,52 +99,253 @@ const SEND_STATUS = Object.freeze({
 const TYPING_LABEL = "Brain of Reese is thinking";
 const ERROR_HINT = "Try again — if this persists, check the LLM is reachable.";
 /* A turn with no answer content (an empty stream, or reasoning that
   exhausted max_tokens — phase 17) still renders a bubble, and this exact
   text is what gets persisted: what the user saw is what is stored. */
 const EMPTY_ANSWER_FALLBACK = "Hmm — that came back empty. Ask me again?";
 /* Calm, don't remove: smooth scrolling is the one motion JS controls. */
 const reducedMotion =
  typeof matchMedia === "function" && matchMedia("(prefers-reduced-motion: reduce)").matches;
 const SCROLL = reducedMotion ? "auto" : "smooth";
-/* ---------- tiny, safe markdown renderer (no external libs, no CDN) ---------- */
+/* Follow-the-bottom scroll contract (phase 18, owner choice
-export function escapeHtml(s) {
+ * 2026-08-23): the page auto-scrolls only while the user is pinned
-  return s.replace(/[&<>"']/g, (c) => ({
+ * at the bottom — the 200px band covers the composer zone (the
-    "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;",
+ * textarea auto-grows to 192px + the button row), i.e. "the
-  }[c]));
+ * composer is in view". Exported so the band is unit-pinned (same
 * pattern as TURN_TIMEOUT_MS). */
 export const NEAR_BOTTOM_PX = 200;
 function isNearBottom() {
  const bottom =
    document.documentElement.scrollHeight - window.scrollY - window.innerHeight;
  return bottom <= NEAR_BOTTOM_PX;
 }
-export function renderMarkdown(md) {
+/* The ONE scroll call site in this file. `force` is used only by
-  // 1. Protect fenced code blocks.
+ * the phase-14 restore landing (one-shot, load-time). */
-  const codeBlocks = [];
+function scrollReveal(wrap, behavior = SCROLL, force = false) {
-  let text = md.replace(/```(\w*)\n([\s\S]*?)```/g, (_m, _lang, code) => {
+  if (force || isNearBottom()) {
-    codeBlocks.push(`<pre><code>${escapeHtml(code.replace(/\n$/, ""))}</code></pre>`);
+    wrap.scrollIntoView({ behavior, block: "end" });
-    return `\u0000CODE${codeBlocks.length - 1}\u0000`;
+  }
 }
 /* ---------- document viewer link (phase 10; phase 13 adds `back`) ----------
 * Every cited document opens in the viewer, in a NEW tab. All query
 * values are percent-encoded: real paths contain slashes and sometimes
 * spaces, which would otherwise corrupt the query string. `back` tells the
 * viewer which page to return to when its back button is clicked — the
 * chips live in the chat, so chat passes "/" (the viewer validates it:
 * only same-origin relative URLs are honored; Sources links omit it and
 * get the viewer's /sources.html default). (The renderer
 * renderMarkdown/escapeHtml now lives in assets/markdown.js — a classic
 * script loaded by index.html and document.html before these modules.) */
 export function documentUrl(source, path, back = "/") {
  let url =
    "/document.html?source=" + encodeURIComponent(source) + "&path=" + encodeURIComponent(path);
  if (back) url += "&back=" + encodeURIComponent(back);
  return url;
 }
 /* ---------- steering notes (phase 15) ----------
 *
 * The owner's tuning notes steer every future answer: they live in
 * Postgres (stateless API, A10) and the chat turn reads them into the
 * system prompt. UI contract: Tune button → inline form → save →
 * confirmation (or inline error, form kept); the header panel lists the
 * notes (newest first) with per-note delete.
 */
 const steeringToggle = document.querySelector("#steering-toggle");
 const steeringCount = document.querySelector("#steering-count");
 const steeringPanel = document.querySelector("#steering-panel");
 const steeringList = document.querySelector("#steering-list");
 const steeringEmpty = document.querySelector("#steering-empty");
 const steeringAnnouncer = document.querySelector("#steering-announcer");
 const TUNE_ICON =
  '<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M4 7h10M18 7h2M4 17h4M12 17h8"/><circle cx="15.5" cy="7" r="2.2"/><circle cx="9.5" cy="17" r="2.2"/></svg>';
 let tuneSeq = 0; // unique ids for one open tune form's inputs
 function announceSteering(message) {
  if (steeringAnnouncer) steeringAnnouncer.textContent = message;
 }
 /* "Tune" button in the meta row of a completed brain bubble. Reuses the
   sources' .msg-meta row when it exists (role=list → the button joins as
   a listitem so ARIA stays valid); otherwise creates a plain meta row.
   Phase 16: anonymous visitors never get the button — this single guard
   covers both fresh turns and the phase-14 restore path. */
 function appendTuneButton(wrap) {
  if (!isAdmin) return; // phase 16: tuning is admin-only
  const body = wrap.querySelector(".msg-body");
  if (!body) return;
  let meta = body.querySelector(".msg-meta");
  if (!meta) {
    meta = document.createElement("div");
    meta.className = "msg-meta";
    body.appendChild(meta);
  }
  if (meta.querySelector(".tune-btn")) return; // one per bubble
  const btn = document.createElement("button");
  btn.type = "button";
  btn.className = "tune-btn";
  if (meta.getAttribute("role") === "list") btn.setAttribute("role", "listitem");
  btn.innerHTML = TUNE_ICON + "<span>Tune</span>";
  btn.addEventListener("click", () => openTuneForm(wrap, btn));
  meta.appendChild(btn);
 }
 /* Inline tuning form under the bubble: labeled textarea (maxlength 2000)
   + Save / Cancel. Success replaces the form with the .tune-saved status
   (role=status); failure keeps the form and shows an inline error
   (role=alert) — the note is never lost on a failed save. */
 function openTuneForm(wrap, toggleBtn) {
  document.querySelectorAll(".tune-form").forEach((f) => f.remove()); // one at a time
  const body = wrap.querySelector(".msg-body");
  if (!body) return;
  tuneSeq += 1;
  const inputId = `tune-input-${tuneSeq}`;
  const form = document.createElement("form");
  form.className = "tune-form";
  form.noValidate = true;
  form.innerHTML =
    `<label for="${inputId}">Tuning note — how should Brain answer from now on?</label>` +
    `<textarea id="${inputId}" name="note" rows="2" maxlength="2000"
              placeholder="e.g. be more concise — or: assume I'm on NixOS"></textarea>`;
  const actions = document.createElement("div");
  actions.className = "tune-form-actions";
  const saveBtn = document.createElement("button");
  saveBtn.type = "submit";
  saveBtn.className = "tune-save";
  saveBtn.textContent = "Save";
  const cancelBtn = document.createElement("button");
  cancelBtn.type = "button";
  cancelBtn.className = "tune-cancel";
  cancelBtn.textContent = "Cancel";
  actions.append(saveBtn, cancelBtn);
  form.appendChild(actions);
  const status = document.createElement("p");
  status.className = "tune-error";
  status.setAttribute("role", "alert");
  status.hidden = true;
  form.appendChild(status);
  form.addEventListener("submit", async (e) => {
    e.preventDefault();
    saveBtn.disabled = true;
    status.hidden = true;
    try {
      const r = await fetch("/api/steering", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ note: form.querySelector("textarea").value }),
      });
      if (!r.ok) {
        let detail = "Could not save the note — try again.";
        try {
          const data = await r.json();
          if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {
            detail = String(data.detail[0].msg);
          } else if (typeof data.detail === "string" && data.detail) {
            detail = data.detail;
          }
        } catch { /* non-JSON error body */ }
        status.textContent = detail;
        status.hidden = false;
        saveBtn.disabled = false;
        return; // form kept on failure — the instruction survives
      }
      const saved = document.createElement("p");
      saved.className = "tune-saved";
      saved.setAttribute("role", "status");
      saved.textContent = "Saved — future answers will follow this.";
      form.replaceWith(saved);
      announceSteering("Tuning note saved. Future answers will follow it.");
      await loadSteering(); // panel + count badge update
    } catch {
      status.textContent = "Could not save the note — is the app reachable?";
      status.hidden = false;
      saveBtn.disabled = false;
    }
  });
  cancelBtn.addEventListener("click", () => {
    form.remove();
    toggleBtn.focus();
  });
  body.appendChild(form);
  form.querySelector("textarea").focus();
 }
-  // 2. Escape everything else, then apply inline + block transforms.
+/* Panel: newest-first list (textContent — XSS-safe), per-note delete,
-  text = escapeHtml(text)
+   empty text, and the header count badge. */
-    .replace(/`([^`\n]+)`/g, "<code>$1</code>")
+async function loadSteering() {
-    .replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>")
+  let notes = [];
-    .replace(/(^|[\s(])\*([^*\n]+)\*/g, "$1<em>$2</em>")
+  try {
-    .replace(/^### (.*)$/gm, "<h4>$1</h4>")
+    const r = await fetch("/api/steering");
-    .replace(/^## (.*)$/gm, "<h3>$1</h3>")
+    if (r.ok) notes = (await r.json()).notes || [];
-    .replace(/^# (.*)$/gm, "<h3>$1</h3>")
+  } catch { /* API unreachable: keep the last rendered list */ }
-    .replace(/^\s*[-*] (.*)$/gm, "<li>$1</li>")
+  renderSteeringPanel(notes);
-    .replace(/(<li>[\s\S]*?<\/li>)(?!\s*<li>)/g, "<ul>$1</ul>")
+  return notes;
-    .replace(/^\d+\. (.*)$/gm, "<li>$1</li>");
+}
-  // 3. Paragraphs (double newline separated).
+function renderSteeringPanel(notes) {
-  text = text
+  if (!steeringList) return;
-    .split(/\n{2,}/)
+  steeringList.textContent = "";
-    .map((block) => {
+  for (const n of notes) {
-      const b = block.trim();
+    const li = document.createElement("li");
-      if (!b) return "";
+    li.className = "steering-note";
-      if (/^<(h\d|ul|ol|pre|li)/.test(b)) return b;
+    const text = document.createElement("span");
-      return `<p>${b.replace(/\n/g, "<br>")}</p>`;
+    text.className = "steering-note-text";
-    })
+    text.textContent = n.note; // rendered as text, never as HTML
-    .join("");
+    li.appendChild(text);
    const del = document.createElement("button");
    del.type = "button";
    del.className = "steering-delete";
    del.setAttribute("aria-label", `Delete tuning note: ${n.note}`);
    del.innerHTML =
      '<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round"><path d="M5 7h14M10 7V5h4v2M8.5 7l.7 12h5.6l.7-12"/></svg>';
    del.addEventListener("click", () => deleteSteeringNote(n.id, del));
    li.appendChild(del);
    steeringList.appendChild(li);
  }
  if (steeringEmpty) steeringEmpty.hidden = notes.length > 0;
  if (steeringCount) steeringCount.textContent = String(notes.length);
 }
-  // 4. Restore code blocks.
+async function deleteSteeringNote(id, btn) {
-  return text.replace(/\u0000CODE(\d+)\u0000/g, (_m, i) => codeBlocks[Number(i)]);
+  btn.disabled = true;
  try {
    const r = await fetch(`/api/steering/${encodeURIComponent(id)}`, { method: "DELETE" });
    if (r.status === 404) {
      announceSteering("That note was already removed.");
      await loadSteering();
      return;
    }
    if (!r.ok) {
      announceSteering("Could not delete the note — try again.");
      btn.disabled = false;
      return;
    }
    await loadSteering();
    announceSteering("Tuning note deleted.");
  } catch {
    announceSteering("Could not delete the note — is the app reachable?");
    btn.disabled = false;
  }
 }
 function setSteeringPanel(open) {
  if (!steeringPanel || !steeringToggle) return;
  steeringPanel.hidden = !open;
  steeringToggle.setAttribute("aria-expanded", open ? "true" : "false");
 }
 if (steeringToggle && steeringPanel) {
  steeringToggle.addEventListener("click", () => {
    setSteeringPanel(steeringPanel.hidden);
    if (!steeringPanel.hidden) loadSteering(); // refresh when (re)opened
  });
 }
 /* ---------- avatar glyphs (phase 08: emoji-free chrome) ----------
@@ -118,8 +359,11 @@ const BRAIN_AVATAR =
 const USER_AVATAR =
  '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" aria-hidden="true"><circle cx="12" cy="8" r="3.6"/><path d="M4.8 20.2c.9-3.9 3.8-6 7.2-6s6.3 2.1 7.2 6"/></svg>';
-/* ---------- messages ---------- */
+/* ---------- messages ----------
-function addMessage(who, html) {
+ * Scroll is conditional (phase 18): addMessage reveals through
 * scrollReveal — only when the user is pinned to the bottom, or when
 * forced (the one-shot phase-14 restore landing). */
 function addMessage(who, html, scrollBehavior = SCROLL, force = false) {
  if (emptyState) emptyState.hidden = true;
  const wrap = document.createElement("div");
  wrap.className = `msg ${who}`;
@@ -129,7 +373,7 @@ function addMessage(who, html) {
      <div class="bubble">${html}</div>
    </div>`;
  messagesEl.appendChild(wrap);
-  wrap.scrollIntoView({ behavior: SCROLL, block: "end" });
+  scrollReveal(wrap, scrollBehavior, force);
  return wrap;
 }
@@ -147,13 +391,40 @@ function addTyping() {
      </div>
    </div>`;
  messagesEl.appendChild(wrap);
-  wrap.scrollIntoView({ behavior: SCROLL, block: "end" });
+  scrollReveal(wrap);
 }
 function removeTyping() {
  document.querySelector("#typing-indicator")?.remove();
 }
 /* ---------- thinking block (phase 17) ----------
 * The model's reasoning streams into a collapsible <details> block ABOVE
 * the answer bubble: created OPEN on the first `thinking` event,
 * auto-collapsed when the first answer token lands, and user-toggleable
 * afterwards (native <details>/<summary> — a real focusable control).
 * ensureThinkingBlock is idempotent (returns the existing block if any);
 * closeThinkingBlock never reopens a block once the answer has started,
 * so a late/interleaved `thinking` event only appends to the closed text. */
 function ensureThinkingBlock(wrap) {
  let block = wrap.querySelector(".thinking");
  if (!block) {
    block = document.createElement("details");
    block.className = "thinking";
    block.open = true;
    block.innerHTML =
      `<summary>Thinking</summary><div class="thinking-text"></div>`;
    const body = wrap.querySelector(".msg-body");
    body.insertBefore(block, body.querySelector(".bubble"));
  }
  return block;
 }
 function closeThinkingBlock(wrap) {
  const block = wrap?.querySelector?.(".thinking");
  if (block) block.open = false; // idempotent; no-op without a block
 }
 /* ---------- suggestions (shared chip component, phase 05) ----------
 *
 * One component, two homes: the onboarding row in the empty state and the
@@ -323,7 +594,9 @@ function appendSources(wrap, sources) {
    const chip = document.createElement("a");
    chip.className = "source-chip";
    chip.setAttribute("role", "listitem");
-    chip.href = "/sources.html";
+    chip.href = documentUrl(s.source, s.path, "/"); // back → the chat page
    chip.target = "_blank"; // open the full document in a new tab
    chip.rel = "noopener";
    chip.textContent = label;
    chip.title = label;
    meta.appendChild(chip);
@@ -353,6 +626,191 @@ function appendMaybeTry(wrap, suggestions) {
  body.appendChild(group);
 }
 /* ---------- conversation persistence (phase 14) ----------
 *
 * A durable LOCAL session (A10 unchanged: the API stays stateless — the
 * server stores nothing about the conversation). The whole conversation
 * lives in localStorage under a versioned key; a format bump = clean start:
 *
 *     bor.chat.v1 → { v: 1, messages: [{ who: "user"|"brain", text,
 *                                        sources?, deflected?, suggestions?,
 *                                        thinking? }] }
 *
 * Only RAW TEXT is stored — restore re-renders it through the escape-first
 * markdown renderer, so no HTML is ever persisted. Save points: the user
 * message on send (a failed turn keeps the question) and the brain message
 * on `done` (with sources/deflected/suggestions). Every localStorage access
 * is try/catch'd — private mode or quota exhaustion degrades silently to
 * in-memory-only chat. If the serialized state outgrows the budget (~700k
 * chars, far under the ~5MB quota) the oldest messages are dropped first.
 */
 const STORAGE_KEY = "bor.chat.v1";
 export const STORAGE_VERSION = 1;
 export const STORAGE_BUDGET_CHARS = 700_000;
 let conversation = []; // in-memory copy of the persisted messages
 function loadStoredConversation() {
  try {
    const raw = localStorage.getItem(STORAGE_KEY);
    if (!raw) return [];
    const data = JSON.parse(raw);
    if (!data || data.v !== STORAGE_VERSION || !Array.isArray(data.messages)) return [];
    // Legacy/corrupt shape → clean start; keep only well-formed raw-text
    // messages (nothing HTML-shaped can survive this filter).
    return data.messages.filter(
      (m) =>
        m &&
        (m.who === "user" || m.who === "brain") &&
        typeof m.text === "string" &&
        m.text.length > 0
    );
  } catch {
    return []; // unreadable storage: start clean, never throw
  }
 }
 function trimToBudget(messages) {
  let out = messages.slice();
  for (;;) {
    let size = Infinity;
    try {
      size = JSON.stringify({ v: STORAGE_VERSION, messages: out }).length;
    } catch {
      break; // even one message cannot serialize — keep it in memory only
    }
    if (size <= STORAGE_BUDGET_CHARS || out.length <= 1) return out;
    out = out.slice(1); // drop the oldest until it fits
  }
  return out;
 }
 function saveConversation() {
  try {
    localStorage.setItem(
      STORAGE_KEY,
      JSON.stringify({ v: STORAGE_VERSION, messages: trimToBudget(conversation) })
    );
  } catch {
    /* quota/private mode: chat keeps working with in-memory state only */
  }
 }
 export function clearStoredConversation() {
  try {
    localStorage.removeItem(STORAGE_KEY);
  } catch {
    /* nothing was stored */
  }
 }
 function renderStoredMessage(m) {
  // Phase 18: the restore landing is the only `force`d scroll — one-shot,
  // non-smooth, so a restored conversation lands on its latest message
  // (phase-14 behavior preserved) without smooth-scrolling through it.
  if (m.who === "user") {
    addMessage("user", renderMarkdown(m.text), "auto", true);
    return;
  }
  const wrap = addMessage("brain", renderMarkdown(m.text), "auto", true);
  if (m.thinking) {
    // Phase 17: restore the thinking block COLLAPSED above the bubble.
    const block = ensureThinkingBlock(wrap);
    block.open = false;
    block.querySelector(".thinking-text").innerHTML = renderMarkdown(m.thinking);
  }
  if (m.deflected) {
    wrap.classList.add("is-deflected");
    appendMaybeTry(wrap, m.suggestions);
  }
  appendSources(wrap, m.sources);
  appendTuneButton(wrap); // restored brain answers are tunable too
 }
 /* On load: re-render the stored conversation (markdown, source chips,
   deflected styling, maybe-try chips). addMessage hides the empty state,
   so a restored conversation starts right where it was left. */
 function restoreConversation() {
  conversation = loadStoredConversation();
  for (const m of conversation) renderStoredMessage(m);
 }
 /* Brain message save point (on `done`): raw accumulated text + metadata.
   Phase 17: meta.thinking is optional — `undefined` drops the key from
   the JSON, so turns without thinking persist exactly as before. An empty
   answer keeps the fallback/"…" text that was actually rendered — what
   the user saw is what is stored. */
 function rememberBrainTurn(rawText, meta) {
  conversation.push({ who: "brain", text: rawText || "…", ...meta });
  saveConversation();
 }
 /* ---------- new chat (phase 14) ----------
 * Clears the stored conversation + the rendered list and returns to the
 * empty state (suggestions included). Ignored while a turn is in flight —
 * a live stream must not be hijacked. Confirmation reuses the existing
 * #send-status live region (aria-live=polite). */
 /* ---------- single-admin auth (phase 16, A10 revised) ----------
 *
 * /api/whoami decides the header: anonymous → the Sign in link and NO
 * tuning surface at all — the Tuning toggle + panel are removed from the
 * DOM (the story says "absent", not just hidden), /api/steering is never
 * fetched, and appendTuneButton injects nothing (new or restored
 * messages). Admin → Sign out (POST /api/logout + reload) + the full
 * phase-15 UI. Whoami is awaited BEFORE the phase-14 restore, so restored
 * brain bubbles never flash a Tune button that should not be there.
 */
 const signInLink = document.querySelector("#sign-in-link");
 const signOutBtn = document.querySelector("#sign-out-btn");
 let isAdmin = false;
 function applyAuthState() {
  if (signInLink) signInLink.hidden = isAdmin;
  if (signOutBtn) signOutBtn.hidden = !isAdmin;
  if (!isAdmin && steeringToggle) {
    steeringToggle.remove();
    steeringPanel?.remove();
  }
 }
 async function loadAuthState() {
  try {
    const r = await fetch("/api/whoami");
    if (r.ok) isAdmin = (await r.json()).authenticated === true;
  } catch {
    isAdmin = false; // API unreachable: anonymous-safe defaults
  }
  applyAuthState();
  return isAdmin;
 }
 if (signOutBtn) {
  signOutBtn.addEventListener("click", async () => {
    signOutBtn.disabled = true;
    try {
      await fetch("/api/logout", { method: "POST" });
    } catch { /* the reload resets the UI either way */ }
    window.location.reload();
  });
 }
 const newChatBtn = document.querySelector("#new-chat-btn");
 function startNewChat() {
  if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;
  conversation = [];
  clearStoredConversation();
  removeTyping();
  messagesEl.querySelectorAll(".msg").forEach((el) => el.remove());
  if (emptyState) emptyState.hidden = false;
  clearErrorBanner();
  setUiState(UI_STATE.idle);
  input.value = "";
  autoGrow();
  input.focus();
  sendStatus.textContent = "New chat started — previous conversation cleared.";
 }
 if (newChatBtn) newChatBtn.addEventListener("click", startNewChat);
 function showErrorBanner(detail) {
  banner.hidden = false;
  banner.classList.add("is-error");
@@ -375,6 +833,10 @@ async function handleSend(e) {
  if (!text || sendBtn.disabled) return;
  addMessage("user", renderMarkdown(text));
  // Persistence save point 1: the question is stored the moment it is
  // sent, so a failed/interrupted turn never loses it.
  conversation.push({ who: "user", text });
  saveConversation();
  input.value = "";
  autoGrow();
  clearErrorBanner();
@@ -383,11 +845,16 @@ async function handleSend(e) {
  let acc = "";
  let res = null;
  let aborted = false; // the 120s guard already took the turn to error
  // Phase 17 (thinking display): turn-local reasoning state.
  let thinkingAcc = ""; // accumulated thinking text (persisted with the turn)
  let sawThinking = false; // did any `thinking` frame arrive this turn?
  let sawDone = false; // did the stream end with a `done` event?
  try {
    // thinking = pre-token: dots + busy button. The guard is armed so a
    // hung stream can never leave the button zombified; it clears on the
-    // first delta (entering "streaming") and on every terminal transition.
+    // first thinking OR delta event (phase 17) and on every terminal
    // transition.
    setUiState(UI_STATE.thinking);
    armTurnTimeout(() => {
      aborted = true;
@@ -410,16 +877,34 @@ async function handleSend(e) {
    }
    await readSSE(res, (ev) => {
      if (aborted) return;
-      if (ev.type === "delta") {
+      if (ev.type === "thinking") {
-        acc += ev.text || "";
+        // Phase 17: model reasoning — stream it live into the collapsible
-        if (!wrap) {
+        // Thinking block. No setUiState here: the UI state stays
-          // First token: dots out, live bubble in; the button stays busy.
+        // "thinking" (button still disabled with "Thinking…", #send-status
-          setUiState(UI_STATE.streaming);
+        // unchanged) — the live block simply replaces the typing dots as
-          wrap = addMessage("brain", "");
+        // the visible feedback.
        thinkingAcc += ev.text || "";
        sawThinking = true;
        clearTurnTimeout(); // the stream is alive — as the first delta says
        if (!wrap) wrap = addMessage("brain", "");
        removeTyping(); // the live block replaces the dots as feedback
        const block = ensureThinkingBlock(wrap);
        const textEl = block.querySelector(".thinking-text");
        textEl.innerHTML = renderMarkdown(thinkingAcc); // escape-first, XSS-safe
        if (block.open) {
          textEl.scrollTop = textEl.scrollHeight; // pin the stream to the bottom
          scrollReveal(wrap); // page follows only while pinned (phase 18)
        }
      } else if (ev.type === "delta") {
        acc += ev.text || "";
        if (uiState === UI_STATE.thinking) setUiState(UI_STATE.streaming);
        if (!wrap) wrap = addMessage("brain", ""); // first token: live bubble in
        closeThinkingBlock(wrap); // auto-collapse; idempotent, never reopens
        wrap.querySelector(".bubble").innerHTML = renderMarkdown(acc);
-        wrap.scrollIntoView({ behavior: SCROLL, block: "end" });
+        scrollReveal(wrap); // page follows only while pinned (phase 18)
      } else if (ev.type === "done") {
        sawDone = true;
        closeThinkingBlock(wrap); // the turn is over: settle the block closed
        if (!wrap) {
          setUiState(UI_STATE.streaming);
          wrap = addMessage("brain", "…");
@@ -429,12 +914,42 @@ async function handleSend(e) {
          appendMaybeTry(wrap, ev.suggestions);
        }
        appendSources(wrap, ev.sources);
        appendTuneButton(wrap); // every completed brain bubble is tunable
        // Thinking-without-answer (reasoning can exhaust max_tokens): the
        // bubble gets the empty-answer fallback — what the user saw is
        // what gets persisted.
        const finalText = acc || (sawThinking ? EMPTY_ANSWER_FALLBACK : "");
        if (!acc && sawThinking) {
          wrap.querySelector(".bubble").innerHTML = renderMarkdown(finalText);
        }
        // Persistence save point 2: the answer lands only when the turn is
        // complete (raw text + the done metadata; phase 17: + optional
        // thinking — `undefined` drops the key from the JSON).
        rememberBrainTurn(finalText || acc, {
          thinking: thinkingAcc || undefined,
          deflected: !!ev.deflected,
          sources: ev.sources,
          suggestions: ev.suggestions,
        });
      } else if (ev.type === "error") {
        throw new Error(ev.detail || "Something went wrong on my side.");
      }
    });
    // Stream-drop guard (phase 17): frames arrived but no `done` event —
    // the connection died mid-turn. Say so; never settle silently into
    // idle with a half bubble. The zero-frame case falls through to the
    // existing empty-answer fallback below.
    if (!sawDone && !aborted && (acc || thinkingAcc)) {
      setUiState(
        UI_STATE.error,
        "The stream ended before my answer finished — try again?"
      );
    }
    if (!aborted && !wrap) {
-      addMessage("brain", "Hmm — that came back empty. Ask me again?");
+      const fallback = EMPTY_ANSWER_FALLBACK;
      const fwrap = addMessage("brain", fallback);
      appendTuneButton(fwrap);
      rememberBrainTurn(fallback, {}); // persist what the user actually saw
    }
  } catch (err) {
    if (!aborted) {
@@ -451,7 +966,9 @@ async function handleSend(e) {
    stopThinkingClock();
    try { res?.body?.cancel(); } catch { /* stream already closed */ }
    if (uiState !== UI_STATE.idle) setUiState(UI_STATE.idle);
-    input.focus();
+    // Phase 18: focus back for the next question, but never move the
    // viewport — a user reading earlier content stays where they are.
    input.focus({ preventScroll: true });
  }
 }
@@ -464,5 +981,13 @@ input.addEventListener("keydown", (e) => {
 });
 composer.addEventListener("submit", handleSend);
 /* Boot: auth state FIRST — it decides whether the restored conversation
   gets Tune buttons and whether the steering UI exists at all (phase 16).
   Phase 14: the conversation then comes back exactly as left. */
 (async () => {
  await loadAuthState();
  restoreConversation();
  loadSuggestions();
  loadHealth();
  if (isAdmin) loadSteering(); // phase 15: panel + count badge (admin only)
 })();
@@ -0,0 +1,133 @@
 /* Brain of Reese — document viewer (phase 10).
 *
 * Reads `source`/`path` query params, fetches the stateless content
 * endpoint (GET /api/documents/content — database only, no filesystem),
 * and renders:
 *
 *   • md / markdown  → the shared escape-first renderer (markdown.js) in a
 *                      ≤46rem centered column;
 *   • any other      → the raw content as a text node inside
 *                      <pre class="doc-raw"> (mono, horizontal scroll).
 *
 * XSS-safe by construction: markdown is escaped before transform, raw
 * formats are set via textContent, and every document-derived string
 * (title, badges, path) is written with textContent — never innerHTML.
 *
 * A missing document (unknown pair, missing params, network error) shows
 * the designed not-found card with a link back to the Sources page.
 */
 const params = new URLSearchParams(window.location.search);
 const source = params.get("source") || "";
 const path = params.get("path") || "";
 const titleEl = document.querySelector("#doc-title");
 const metaEl = document.querySelector("#doc-meta");
 const contentEl = document.querySelector("#doc-content");
 const notFoundEl = document.querySelector("#doc-not-found");
 const mainEl = document.querySelector("#main");
 const backLink = document.querySelector("#doc-back");
 /* Back button (phase 13): the return target comes from the `back` query
 * param, not the browser history — both entry points (chat source chips
 * and the Sources table) open the viewer in a NEW tab, where there is no
 * history to go back to. The param is honored only for same-origin
 * relative URLs (starts with "/" but not "//"), so absolute (https://…),
 * protocol-relative (//…), and pseudo-protocol (javascript:…) values are
 * rejected; anything else falls back to the Sources page. The static
 * href="/sources.html" in document.html remains the no-JS fallback, and
 * with the href set the anchor's default click behavior IS the
 * deterministic navigation (no browser-history heuristics). */
 const backParam = params.get("back") || "";
 const backTarget =
  backParam.startsWith("/") && !backParam.startsWith("//")
    ? backParam
    : "/sources.html";
 backLink.href = backTarget;
 const backLabel = backLink.querySelector("span");
 if (backLabel) {
  backLabel.textContent =
    backTarget === "/"
      ? "Chat"
      : backTarget === "/sources.html"
        ? "Sources"
        : "Back";
 }
 function fmtDate(iso) {
  try {
    return new Date(iso).toLocaleString();
  } catch {
    return iso;
  }
 }
 function contentUrl(s, p) {
  return "/api/documents/content?source=" + encodeURIComponent(s) + "&path=" + encodeURIComponent(p);
 }
 function metaBadge(cls, text) {
  const el = document.createElement("span");
  el.className = cls;
  el.textContent = text;
  return el;
 }
 function render(doc) {
  titleEl.textContent = doc.title;
  document.title = `${doc.title} · Brain of Reese`;
  const pathCode = document.createElement("code");
  pathCode.className = "doc-path";
  pathCode.textContent = doc.path;
  metaEl.replaceChildren(
    metaBadge("doc-source-badge", doc.source),
    metaBadge("format-badge", doc.format),
    pathCode,
    metaBadge("doc-indexed", `Indexed ${fmtDate(doc.indexed_at)}`),
    metaBadge("doc-chunks", `${doc.chunks} chunk${doc.chunks === 1 ? "" : "s"}`),
  );
  notFoundEl.hidden = true;
  contentEl.replaceChildren();
  if (doc.format === "md" || doc.format === "markdown") {
    const wrap = document.createElement("div");
    wrap.className = "doc-md";
    wrap.innerHTML = renderMarkdown(doc.content); // escape-first: XSS-safe
    contentEl.appendChild(wrap);
  } else {
    const pre = document.createElement("pre");
    pre.className = "doc-raw";
    pre.textContent = doc.content; // text node: never parsed as HTML
    contentEl.appendChild(pre);
  }
 }
 function showNotFound() {
  titleEl.textContent = "Document not found";
  document.title = "Document not found · Brain of Reese";
  metaEl.replaceChildren();
  contentEl.replaceChildren();
  notFoundEl.hidden = false;
 }
 async function load() {
  try {
    if (!source || !path) {
      showNotFound();
      return;
    }
    const r = await fetch(contentUrl(source, path));
    if (!r.ok) {
      showNotFound();
      return;
    }
    render(await r.json());
  } catch {
    showNotFound();
  } finally {
    mainEl.focus(); // move focus to main on load (a11y)
  }
 }
 load();
@@ -0,0 +1,80 @@
 /* Brain of Reese — admin sign-in (phase 16, A10 revised).
 *
 * One admin, one password. On submit → POST /api/login: 204 sets the
 * signed session cookie and we redirect to `?next` (same-origin relative
 * URLs only — "/…" but never "//host" or an absolute URL; default
 * /sources.html). A 401 keeps the form and announces through the
 * role=alert error region. On load, /api/whoami already says admin →
 * straight to `next`, no form.
 *
 * No CDN, no state in this file: the signed cookie is the whole session.
 * All DOM ids match frontend/login.html.
 */
 const form = document.querySelector("#login-form");
 const passwordInput = document.querySelector("#login-password");
 const submitBtn = document.querySelector("#login-submit");
 const errorEl = document.querySelector("#login-error");
 const DEFAULT_NEXT = "/sources.html";
 /* Same-origin relative URLs only: honor `?next=/…`, reject anything that
   would leave the origin (protocol-relative "//…" or absolute). */
 function safeNext() {
  const next = new URLSearchParams(window.location.search).get("next") || DEFAULT_NEXT;
  return next.startsWith("/") && !next.startsWith("//") ? next : DEFAULT_NEXT;
 }
 function showError(message) {
  errorEl.textContent = message;
  errorEl.hidden = false;
  submitBtn.disabled = false;
  passwordInput.focus();
  passwordInput.select();
 }
 async function alreadySignedIn() {
  try {
    const r = await fetch("/api/whoami");
    if (!r.ok) return false;
    return (await r.json()).authenticated === true;
  } catch {
    return false; // API unreachable: stay on the form — submit will explain
  }
 }
 form.addEventListener("submit", async (e) => {
  e.preventDefault();
  errorEl.hidden = true;
  errorEl.textContent = "";
  submitBtn.disabled = true;
  try {
    const r = await fetch("/api/login", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ password: passwordInput.value }),
    });
    if (r.status === 204) {
      // Session cookie set — off to the requested page.
      window.location.replace(safeNext());
      return;
    }
    // One generic failure (401); anything else is a server-side surprise.
    const detail =
      r.status === 401
        ? "Invalid password — try again."
        : `Sign-in failed (HTTP ${r.status}) — try again.`;
    showError(detail);
  } catch {
    showError("Could not reach the server — try again.");
  }
 });
 /* Already the admin? Skip the form and go straight to the target. */
 (async () => {
  if (await alreadySignedIn()) {
    window.location.replace(safeNext());
    return;
  }
  passwordInput.focus();
 })();
@@ -0,0 +1,51 @@
 /* Brain of Reese — shared markdown renderer (no external libs, no CDN).
 *
 * Extracted from app.js (phase 10) so the chat page and the document
 * viewer share the exact same escape-first renderer: every character is
 * HTML-escaped before any markup transform runs, so document (or user)
 * content can never inject live HTML/XSS. Classic script on purpose:
 * index.html and document.html load it via a plain relative <script src>
 * and both module scripts (app.js / document.js) call the globals it
 * defines. Rendering behavior is unchanged from the original app.js copy.
 */
 function escapeHtml(s) {
  return s.replace(/[&<>"']/g, (c) => ({
    "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;",
  }[c]));
 }
 function renderMarkdown(md) {
  // 1. Protect fenced code blocks.
  const codeBlocks = [];
  let text = md.replace(/```(\w*)\n([\s\S]*?)```/g, (_m, _lang, code) => {
    codeBlocks.push(`<pre><code>${escapeHtml(code.replace(/\n$/, ""))}</code></pre>`);
    return `\u0000CODE${codeBlocks.length - 1}\u0000`;
  });
  // 2. Escape everything else, then apply inline + block transforms.
  text = escapeHtml(text)
    .replace(/`([^`\n]+)`/g, "<code>$1</code>")
    .replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>")
    .replace(/(^|[\s(])\*([^*\n]+)\*/g, "$1<em>$2</em>")
    .replace(/^### (.*)$/gm, "<h4>$1</h4>")
    .replace(/^## (.*)$/gm, "<h3>$1</h3>")
    .replace(/^# (.*)$/gm, "<h3>$1</h3>")
    .replace(/^\s*[-*] (.*)$/gm, "<li>$1</li>")
    .replace(/(<li>[\s\S]*?<\/li>)(?!\s*<li>)/g, "<ul>$1</ul>")
    .replace(/^\d+\. (.*)$/gm, "<li>$1</li>");
  // 3. Paragraphs (double newline separated).
  text = text
    .split(/\n{2,}/)
    .map((block) => {
      const b = block.trim();
      if (!b) return "";
      if (/^<(h\d|ul|ol|pre|li)/.test(b)) return b;
      return `<p>${b.replace(/\n/g, "<br>")}</p>`;
    })
    .join("");
  // 4. Restore code blocks.
  return text.replace(/\u0000CODE(\d+)\u0000/g, (_m, i) => codeBlocks[Number(i)]);
 }
@@ -9,10 +9,24 @@
 const tbody = document.querySelector("#docs-tbody");
 const emptyEl = document.querySelector("#sources-empty");
 const tableWrap = document.querySelector(".table-wrap");
 const statCards = document.querySelector("#stat-cards");
 const gateEl = document.querySelector("#sources-gate");
 const statDocs = document.querySelector("#stat-docs");
 const statChunks = document.querySelector("#stat-chunks");
 const statLast = document.querySelector("#stat-last");
 /* Phase 16: whoami BEFORE the docs fetch. Anonymous visitors get the
 * sign-in gate (stat cards + table hidden) and NO /api/docs call — the
 * catalog is admin-only. The document viewer itself stays public (the
 * soft rule), so the gate copy points at what keeps working. */
 async function isAdmin() {
  try {
    const r = await fetch("/api/whoami");
    if (r.ok) return (await r.json()).authenticated === true;
  } catch { /* API unreachable: anonymous-safe gate */ }
  return false;
 }
 function fmtDate(iso) {
  try {
    return new Date(iso).toLocaleString();
@@ -21,6 +35,12 @@ function fmtDate(iso) {
  }
 }
 /* Viewer link (phase 10) — same encoded URL the chat chips use; both query
 * values are percent-encoded (paths contain slashes, sometimes spaces). */
 export function documentUrl(source, path) {
  return "/document.html?source=" + encodeURIComponent(source) + "&path=" + encodeURIComponent(path);
 }
 async function loadDocs() {
  let r;
  try {
@@ -56,13 +76,30 @@ async function loadDocs() {
 function makeRow(d) {
  const tr = document.createElement("tr");
-  const cells = [d.source, d.path, d.title, String(d.chunks), fmtDate(d.indexed_at)];
+
-  for (const value of cells) {
+  const sourceTd = document.createElement("td");
  sourceTd.textContent = d.source; // document-derived text — never innerHTML
  tr.appendChild(sourceTd);
  // Path cell: a link to the document viewer (phase 10), full path as the
  // accessible/hover name (the column is ellipsized).
  const pathTd = document.createElement("td");
  pathTd.title = d.path; // full path on hover (column is ellipsized)
  const link = document.createElement("a");
  link.className = "doc-link";
  link.href = documentUrl(d.source, d.path);
  link.target = "_blank"; // open the full document in a new tab
  link.rel = "noopener";
  link.title = d.path; // full path as the link's hover/accessible name
  link.textContent = d.path;
  pathTd.appendChild(link);
  tr.appendChild(pathTd);
  for (const value of [d.title, String(d.chunks), fmtDate(d.indexed_at)]) {
    const td = document.createElement("td");
-    td.textContent = value; // document-derived text — never innerHTML
+    td.textContent = value;
    tr.appendChild(td);
  }
  tr.children[1].title = d.path; // full path on hover (column is ellipsized)
  return tr;
 }
@@ -74,4 +111,15 @@ function showEmpty() {
  if (tableWrap) tableWrap.hidden = true;
 }
 (async () => {
  if (!(await isAdmin())) {
    // Anonymous: gate in, catalog out, and no /api/docs request at all.
    if (statCards) statCards.hidden = true;
    if (tableWrap) tableWrap.hidden = true;
    if (emptyEl) emptyEl.hidden = true;
    if (gateEl) gateEl.hidden = false;
    return;
  }
  if (gateEl) gateEl.hidden = true;
  loadDocs();
 })();
@@ -152,9 +152,15 @@ body::after {
  position: sticky;
  top: 0;
  z-index: 20;
  /* Phase 12: body is a definite-height flex column; without this the
     header shrinks (flex-shrink:1) to its content minimum on any page
     whose content overflows the viewport (e.g. Sources at ≤640px). */
  flex-shrink: 0;
 }
-/* 2px brand→cyan gradient hairline under the sticky header (phase 08). */
+/* 2px brand→cyan gradient hairline under the sticky header (phase 08;
-.app-header::after {
+   shared by the app header and the document-viewer header, phase 10). */
 .app-header::after,
 .doc-header::after {
  content: "";
  position: absolute;
  inset-inline: 0;
@@ -168,11 +174,13 @@ body::after {
    rgb(34 211 238 / 0.05) 90%
  );
 }
 /* margin-left:auto on the nav (not justify-content:space-between) so the
   phase-14 "New chat" pill clusters with the nav on the right while the
   two-child Sources header keeps the exact same look. */
 .header-inner {
  height: 100%;
  display: flex;
  align-items: center;
  justify-content: space-between;
  gap: 1rem;
 }
 .brand {
@@ -192,7 +200,7 @@ body::after {
 .brand-mark { width: 22px; height: 22px; flex: 0 0 auto; display: block; }
 .brand-text strong { color: var(--brand-ink); font-weight: 700; }
-.app-nav { display: flex; gap: 0.25rem; }
+.app-nav { display: flex; gap: 0.25rem; margin-left: auto; }
 .nav-link {
  padding: 0.5rem 0.9rem;
  border-radius: 999px;
@@ -207,6 +215,101 @@ body::after {
 .nav-link:hover { background: var(--brand-soft); color: var(--brand-ink); }
 .nav-link.is-active { background: var(--brand); color: var(--bg); }
 /* "New chat" reset (phase 14): ghost pill in the chat header, hover like
   a nav link. ink-soft on surface ≈6.9:1; hover pair brand-ink/brand-soft
   ≈6.9:1 — both WCAG AA. Icon-only below 640px (aria-label keeps the
   accessible name); ≥44px touch target at every width. */
 .new-chat-btn {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  gap: 0.4rem;
  min-height: 44px;
  padding: 0.5rem 0.9rem;
  border-radius: 999px;
  border: 1px solid var(--line);
  background: transparent;
  color: var(--ink-soft);
  font: inherit;
  font-weight: 600;
  font-size: 0.95rem;
  white-space: nowrap;
  cursor: pointer;
 }
 .new-chat-btn:hover { background: var(--brand-soft); color: var(--brand-ink); }
 /* The plus mark is hidden on desktop (label carries the pill); it is the
   whole control below 640px. */
 .new-chat-btn svg { width: 16px; height: 16px; display: none; }
 /* Phase 16: header auth controls (Sign in link / Sign out button) — the
   same ghost pill as New chat, so the bar keeps one visual language.
   ink-soft on surface ≈6.9:1; hover pair brand-ink/brand-soft ≈6.9:1.
   Icon-only below 640px (aria-labels/labels keep the accessible names);
   ≥44px touch target at every width. Exactly one is ever visible. */
 .auth-link {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  gap: 0.4rem;
  min-height: 44px;
  padding: 0.5rem 0.9rem;
  border-radius: 999px;
  border: 1px solid var(--line);
  background: transparent;
  color: var(--ink-soft);
  font: inherit;
  font-weight: 600;
  font-size: 0.95rem;
  white-space: nowrap;
  text-decoration: none;
  cursor: pointer;
 }
 .auth-link:hover { background: var(--brand-soft); color: var(--brand-ink); }
 .auth-link:disabled { opacity: 0.6; cursor: wait; }
 .auth-link svg { width: 16px; height: 16px; display: none; }
 /* "Tuning" toggle (phase 15): ghost pill like New chat + a mono count
   badge (brand-ink on brand-soft ≈6.9:1). The label is visually-hidden
   (not removed) below 640px so the accessible name keeps the word.
   ≥44px touch target at every width. */
 .steering-toggle {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  gap: 0.4rem;
  min-height: 44px;
  padding: 0.5rem 0.9rem;
  border-radius: 999px;
  border: 1px solid var(--line);
  background: transparent;
  color: var(--ink-soft);
  font: inherit;
  font-weight: 600;
  font-size: 0.95rem;
  white-space: nowrap;
  cursor: pointer;
 }
 .steering-toggle:hover, .steering-toggle[aria-expanded="true"] {
  background: var(--brand-soft);
  color: var(--brand-ink);
 }
 .steering-toggle svg { width: 16px; height: 16px; display: block; }
 .steering-count {
  font-family: var(--mono);
  font-size: 0.78rem;
  font-weight: 700;
  min-width: 1.35rem;
  text-align: center;
  padding: 0.05rem 0.4rem;
  border-radius: 999px;
  background: var(--brand-soft);
  color: var(--brand-ink);
 }
 .steering-toggle[aria-expanded="true"] .steering-count {
  background: var(--brand);
  color: var(--bg); /* dark ink on brand: 5.2:1 */
 }
 /* ---------- Main frame ---------- */
 .app-main {
  flex: 1;
@@ -298,6 +401,58 @@ body::after {
  border-color: var(--accent-line);
 }
 /* Collapsible "Thinking" block (phase 17): the model's reasoning streams
   open ABOVE the answer bubble, auto-collapses when the answer starts, and
   stays user-toggleable (native <details>/<summary> — a real focusable
   control: >=44px target, :focus-visible via the global rule). Summary
   text is brand-ink on surface ≈8.7:1; the scratchpad body is ink-soft on
   surface ≈6.9:1 — both AA. */
 details.thinking {
  background: var(--surface);
  border: 1px solid var(--line);
  border-left: 3px solid var(--brand-soft);
  border-radius: var(--radius-sm);
  margin: 0 0 0.5rem;
  overflow: hidden;
 }
 details.thinking summary {
  display: flex;
  align-items: center;
  gap: 0.5rem;
  padding: 0.5rem 0.75rem;
  min-height: 44px;
  color: var(--brand-ink); /* 8.7:1 on --surface */
  font-size: 0.9rem;
  cursor: pointer;
  list-style: none;
 }
 details.thinking summary::-webkit-details-marker { display: none; }
 /* CSS chevron: ▸ rotates 90° when open (transition stills under
   prefers-reduced-motion — see the reduced-motion block below). */
 details.thinking summary::before {
  content: "▸";
  display: inline-block;
  transition: transform 0.15s ease;
 }
 details.thinking[open] summary::before { transform: rotate(90deg); }
 details.thinking summary:focus-visible {
  outline: 3px solid var(--brand);
  outline-offset: 2px;
 }
 /* The scratchpad is a scrollable, compact area (max-height keeps long
   reasoning from pushing the answer off-screen while open). */
 details.thinking .thinking-text {
  padding: 0 0.75rem 0.75rem;
  color: var(--ink-soft); /* 6.9:1 on --surface */
  font-size: 0.875rem;
  line-height: 1.55;
  max-height: 320px;
  overflow-y: auto;
 }
 /* The scratchpad is compact: tighten the renderer's paragraph/list margins. */
 details.thinking .thinking-text p,
 details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
 .msg-meta {
  font-size: 0.75rem;
  color: var(--ink-soft);
@@ -326,7 +481,7 @@ body::after {
  text-overflow: ellipsis;
  white-space: nowrap;
 }
-.source-chip:hover { background: #2a345f; }
+.source-chip:hover { background: #2a345f; text-decoration: underline; }
 /* "Maybe try" chips under a deflected bubble (phase 04). Unlike the
   onboarding row (which scrolls horizontally on mobile), this group wraps
@@ -347,6 +502,170 @@ body::after {
  max-width: 100%;
 }
 /* ---------- Steering notes (phase 15) ---------- */
 /* "Tune" button in the meta row of every completed brain bubble: ghost
   pill, ≥44px, right-aligned after the source chips. ink-soft on surface
   ≈6.9:1; hover pair brand-ink/brand-soft ≈6.9:1. */
 .tune-btn {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  gap: 0.35rem;
  min-height: 44px;
  margin-left: auto;
  padding: 0.35rem 0.8rem;
  border-radius: 999px;
  border: 1px solid var(--line);
  background: transparent;
  color: var(--ink-soft);
  font: inherit;
  font-weight: 600;
  font-size: 0.82rem;
  white-space: nowrap;
  cursor: pointer;
 }
 .tune-btn svg { width: 14px; height: 14px; display: block; }
 .tune-btn:hover { background: var(--brand-soft); color: var(--brand-ink); }
 /* Inline tuning form under the bubble: labeled textarea + Save/Cancel
   (both ≥44px). Save = brand button (dark ink 5.2:1), Cancel = ghost. */
 .tune-form {
  display: flex;
  flex-direction: column;
  gap: 0.5rem;
  background: var(--surface);
  border: 1px solid var(--brand-soft);
  border-radius: var(--radius-sm);
  padding: 0.75rem 0.85rem;
 }
 .tune-form label {
  font-size: 0.85rem;
  font-weight: 600;
  color: var(--ink-soft);
 }
 .tune-form textarea {
  font: inherit;
  font-size: 0.9rem;
  color: var(--ink);
  background: #0d1120;
  border: 1px solid var(--line);
  border-radius: var(--radius-sm);
  padding: 0.5rem 0.6rem;
  resize: vertical;
  min-height: 2.6rem;
 }
 .tune-form-actions { display: flex; gap: 0.5rem; }
 .tune-save {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  min-height: 44px;
  padding: 0.4rem 1.1rem;
  border: 0;
  border-radius: var(--radius-sm);
  background: var(--brand);
  color: var(--bg);
  font: inherit;
  font-weight: 700;
  cursor: pointer;
 }
 .tune-save:hover:not(:disabled) { background: #7d88f5; }
 .tune-save:disabled { opacity: 0.6; cursor: wait; }
 .tune-cancel {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  min-height: 44px;
  padding: 0.4rem 1rem;
  border-radius: var(--radius-sm);
  border: 1px solid var(--line);
  background: transparent;
  color: var(--ink-soft);
  font: inherit;
  font-weight: 600;
  cursor: pointer;
 }
 .tune-cancel:hover { background: var(--brand-soft); color: var(--brand-ink); }
 /* Save confirmation (role=status): ok pair ≈10.6:1. */
 .tune-saved {
  margin: 0.2rem 0 0 0.25rem;
  background: var(--ok-bg);
  color: var(--ok-ink);
  border: 1px solid rgb(110 231 168 / 0.35);
  border-radius: var(--radius-sm);
  padding: 0.45rem 0.8rem;
  font-size: 0.85rem;
  font-weight: 600;
 }
 /* Inline save failure (role=alert): err pair ≈9.1:1 — form is kept. */
 .tune-error {
  margin: 0 0 0 0.25rem;
  background: var(--err-bg);
  color: var(--err-ink);
  border: 1px solid var(--err-line);
  border-radius: var(--radius-sm);
  padding: 0.45rem 0.8rem;
  font-size: 0.85rem;
  font-weight: 600;
 }
 /* Header panel above the messages: notes newest-first, per-note delete,
   designed empty state. */
 .steering-panel {
  background: var(--surface);
  border: 1px solid var(--brand-soft);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 0.9rem 1.1rem 1rem;
 }
 .steering-panel-head { display: flex; flex-wrap: wrap; align-items: baseline; gap: 0.15rem 0.6rem; }
 .steering-panel-title { margin: 0; font-size: 1rem; font-weight: 700; color: var(--ink); }
 .steering-panel-sub { margin: 0; font-size: 0.82rem; color: var(--ink-soft); }
 .steering-list {
  list-style: none;
  margin: 0.65rem 0 0;
  padding: 0;
  display: flex;
  flex-direction: column;
  gap: 0.4rem;
 }
 .steering-note {
  display: flex;
  align-items: stretch;
  gap: 0.6rem;
  background: #0d1120;
  border: 1px solid var(--line);
  border-radius: var(--radius-sm);
  padding: 0.35rem 0.4rem 0.35rem 0.8rem;
 }
 .steering-note-text {
  color: var(--ink);
  font-size: 0.9rem;
  line-height: 1.45;
  /* Notes may be multi-line instructions — keep line breaks as typed. */
  white-space: pre-wrap;
  overflow-wrap: anywhere;
  flex: 1;
  min-width: 0;
 }
 .steering-delete {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  min-height: 44px;
  min-width: 44px;
  flex: 0 0 auto;
  border: 1px solid var(--line);
  border-radius: var(--radius-sm);
  background: transparent;
  color: var(--ink-soft);
  cursor: pointer;
 }
 .steering-delete svg { width: 16px; height: 16px; display: block; }
 .steering-delete:hover:not(:disabled) { background: var(--err-bg); color: var(--err-ink); border-color: var(--err-line); }
 .steering-delete:disabled { opacity: 0.5; cursor: wait; }
 .steering-empty { margin: 0.65rem 0 0; color: var(--ink-soft); font-size: 0.88rem; }
 /* typing indicator */
 .typing { display: inline-flex; gap: 5px; padding: 0.9rem 1rem; }
 .typing span {
@@ -364,6 +683,8 @@ body::after {
 }
@media (prefers-reduced-motion: reduce) {
  .typing span { animation: none; opacity: 0.7; }
  /* Phase 17 thinking block: the chevron stills (no rotation motion). */
  details.thinking summary::before { transition: none; }
 }
 /* ---------- Empty state & suggestions ---------- */
@@ -484,6 +805,65 @@ body::after {
 .kb-banner.is-error { background: var(--err-bg); color: var(--err-ink); border-color: var(--err-line); }
 .kb-banner svg { width: 18px; height: 18px; flex: 0 0 auto; display: block; }
 /* ---------- Login page (phase 16) ---------- */
 /* Centered card in the standard frame: one admin, one password. */
 .login-shell {
  display: flex;
  justify-content: center;
  flex: 1;
 }
 .login-card {
  width: 100%;
  max-width: 26rem;
  background: var(--surface);
  border: 1px solid var(--line);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 2rem 2rem 2.25rem;
 }
 .login-card h1 { margin: 0 0 0.4rem; font-size: 1.6rem; }
 .login-sub { margin: 0 0 1.5rem; color: var(--ink-soft); }
 #login-form { display: flex; flex-direction: column; gap: 0.75rem; }
 #login-password {
  font: inherit;
  font-size: 1rem;
  color: var(--ink);
  background: #0d1120;
  border: 1px solid var(--line);
  border-radius: var(--radius-sm);
  padding: 0.55rem 0.75rem;
  min-height: 44px;
 }
 #login-password:focus-visible { border-color: var(--brand); }
 /* Brand button: dark ink on brand 5.2:1 (never white on brand). */
 .login-submit {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  min-height: 44px;
  border: 0;
  border-radius: var(--radius-sm);
  background: var(--brand);
  color: var(--bg);
  font: inherit;
  font-weight: 700;
  cursor: pointer;
  padding-inline: 1rem;
 }
 .login-submit:hover:not(:disabled) { background: #7d88f5; }
 .login-submit:disabled { opacity: 0.6; cursor: wait; }
 /* Login failure (role=alert): err pair ≈9.1:1. */
 .login-error {
  margin: 0.9rem 0 0;
  background: var(--err-bg);
  color: var(--err-ink);
  border: 1px solid var(--err-line);
  border-radius: var(--radius-sm);
  padding: 0.5rem 0.8rem;
  font-size: 0.9rem;
  font-weight: 600;
 }
 /* ---------- Sources page ---------- */
 .sources-shell {
  display: flex;
@@ -515,6 +895,39 @@ body::after {
 .stat-value-sm { font-size: 1.15rem; font-weight: 700; }
 .stat-label { color: var(--ink-soft); font-size: 0.88rem; font-weight: 600; }
 /* Phase 16: anonymous sign-in gate — the designed replacement for the
   catalog (stat cards + table) until the admin signs in. */
 .sources-gate {
  display: flex;
  flex-direction: column;
  align-items: center;
  text-align: center;
  gap: 0.4rem;
  background: var(--surface);
  border: 1px solid var(--line);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 2.5rem 1.75rem;
 }
 .sources-gate-glyph { color: var(--brand-ink); width: 44px; height: 44px; }
 .sources-gate-glyph svg { width: 44px; height: 44px; display: block; }
 .sources-gate h2 { margin: 0.6rem 0 0.3rem; font-size: 1.4rem; }
 .sources-gate-sub { margin: 0 auto; max-width: 30rem; color: var(--ink-soft); }
 .sources-gate-link {
  display: inline-flex;
  align-items: center;
  justify-content: center;
  min-height: 44px;
  margin-top: 0.75rem;
  padding: 0.5rem 1.4rem;
  border-radius: 999px;
  background: var(--brand);
  color: var(--bg); /* dark ink on brand: 5.2:1 */
  font-weight: 700;
  text-decoration: none;
 }
 .sources-gate-link:hover { background: #7d88f5; }
 .table-wrap {
  background: var(--surface);
  border: 1px solid var(--line);
@@ -547,6 +960,188 @@ body::after {
 .docs-table tbody tr:hover { background: var(--bg); }
 .docs-table tbody tr:last-child td { border-bottom: 0; }
 /* ---------- Document viewer (phase 10) ---------- */
 /* Phase 12: the same fixed-height bar as .app-header (--header-h, 64px /
   58px mobile) — the header must never change size between chat,
   sources, and the document viewer (owner report 2026-08-22). */
 .doc-header {
  position: sticky;
  top: 0;
  z-index: 20;
  background: var(--surface);
  height: var(--header-h);
  /* Phase 12: same guard as .app-header — the bar never shrinks. */
  flex-shrink: 0;
 }
 .doc-header-inner {
  height: 100%;
  display: flex;
  align-items: center;
  gap: 0.9rem;
 }
 /* Back link: pill with an SVG arrow + "Sources" (>=44px touch target). */
 .doc-back {
  display: inline-flex;
  align-items: center;
  gap: 0.4rem;
  flex: 0 0 auto;
  min-height: 44px;
  padding: 0.45rem 0.9rem;
  border-radius: 999px;
  background: var(--brand-soft);
  border: 1px solid var(--line);
  color: var(--brand-ink); /* 6.9:1 on --brand-soft */
  font-weight: 600;
  font-size: 0.95rem;
  text-decoration: none;
 }
 .doc-back:hover { background: #2a345f; }
 .doc-back svg { width: 16px; height: 16px; display: block; }
 .doc-title-block { min-width: 0; }
 #doc-title {
  margin: 0;
  font-size: 1.3rem;
  line-height: 1.3;
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
 }
 /* Meta row: source badge · format badge · mono path · indexed · chunks.
   Phase 12: it may clip, but it must NEVER wrap — a wrapped meta row
   would grow the header past the shared --header-h bar. */
 .doc-meta {
  display: flex;
  flex-wrap: nowrap;
  align-items: center;
  gap: 0.4rem;
  margin-top: 0.35rem;
  font-size: 0.78rem;
  color: var(--ink-soft);
  /* Overflowing content (badges + path + dates) clips at the bar edge;
     nowrap keeps text on one line so the row can never grow the header. */
  white-space: nowrap;
  overflow: hidden;
 }
 .doc-source-badge {
  background: var(--brand-soft);
  border: 1px solid var(--line);
  color: var(--brand-ink);
  font-weight: 600;
  border-radius: 999px;
  padding: 0.02rem 0.55rem;
 }
 .format-badge {
  font-family: var(--mono);
  background: var(--brand-soft);
  border: 1px solid var(--line);
  color: var(--brand-ink);
  border-radius: var(--radius-sm);
  padding: 0.02rem 0.45rem;
 }
 .doc-path {
  font-family: var(--mono);
  font-size: 0.75rem;
  max-width: 26rem;
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
  /* The path is the meta row's designated ellipsis target: with a floor it
     keeps a visible box on narrow screens while the trailing badges clip. */
  min-width: 6rem;
 }
 .doc-chunks { font-family: var(--mono); }
 .doc-shell {
  display: flex;
  flex-direction: column;
  gap: 1rem;
  flex: 1;
 }
 #doc-content { display: flex; flex-direction: column; }
 .doc-loading { margin: 1.5rem auto; text-align: center; color: var(--ink-soft); }
 /* Markdown: the centered, ≤46rem reading column (PLAN §7.1). */
 .doc-md {
  width: 100%;
  max-width: 46rem;
  margin-inline: auto;
  background: var(--surface);
  border: 1px solid var(--line);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 1.5rem 1.75rem;
  overflow-wrap: anywhere;
 }
 .doc-md p { margin: 0.35rem 0; }
 .doc-md h3 { margin: 1.1rem 0 0.4rem; font-size: 1.15rem; }
 .doc-md h4 { margin: 0.9rem 0 0.35rem; font-size: 1rem; }
 .doc-md > :first-child { margin-top: 0; }
 .doc-md ul { margin: 0.4rem 0; padding-left: 1.3rem; }
 .doc-md pre {
  background: #0d1120;
  color: #e6e9f2;
  padding: 0.7rem 0.9rem;
  border: 1px solid var(--line);
  border-radius: var(--radius-sm);
  overflow-x: auto;
  font-size: 0.85rem;
  font-family: var(--mono);
 }
 .doc-md code { font-family: var(--mono); font-size: 0.88em; background: var(--brand-soft); padding: 0.08em 0.35em; border-radius: 5px; }
 .doc-md pre code { background: none; padding: 0; }
 /* Raw (non-markdown) formats: full-width mono pre, horizontal scroll. */
 .doc-raw {
  width: 100%;
  margin: 0;
  background: var(--surface);
  border: 1px solid var(--line);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 1.25rem 1.5rem;
  overflow-x: auto;
  white-space: pre;
  font-family: var(--mono);
  font-size: 0.85rem;
  line-height: 1.5;
 }
 /* Designed not-found state (no emoji — plain SVG mark, phase 08 rule). */
 .doc-not-found {
  width: 100%;
  max-width: 30rem;
  margin: 2rem auto;
  background: var(--surface);
  border: 1px solid var(--line);
  border-radius: var(--radius);
  box-shadow: var(--shadow);
  padding: 2.25rem 1.75rem;
  text-align: center;
 }
 .doc-not-found-glyph { color: var(--ink-soft); width: 44px; height: 44px; margin-inline: auto; }
 .doc-not-found-glyph svg { width: 44px; height: 44px; display: block; }
 .doc-not-found h2 { margin: 0.8rem 0 0.4rem; font-size: 1.3rem; }
 .doc-not-found-sub { margin: 0 0 1.25rem; color: var(--ink-soft); }
 .doc-open-sources {
  display: inline-flex;
  align-items: center;
  min-height: 44px;
  padding: 0.5rem 1.1rem;
  background: var(--brand);
  color: var(--bg); /* dark ink on brand: 5.2:1 */
  font-weight: 700;
  text-decoration: none;
  border-radius: var(--radius-sm);
 }
 .doc-open-sources:hover { background: #7d88f5; }
 /* Viewer links: Sources-table path cell + chat source chips (phase 10). */
 .doc-link {
  color: var(--brand-ink); /* 8.7:1 on --surface */
  text-decoration: none;
 }
 .doc-link:hover, .doc-link:focus-visible { text-decoration: underline; }
 /* ---------- Footer ---------- */
 .app-footer {
  border-top: 1px solid var(--line);
@@ -566,14 +1161,53 @@ body::after {
@media (max-width: 640px) {
  :root { --header-h: 58px; }
  .container { padding-inline: 0.9rem; }
-  .brand-text { font-size: 0.88rem; }
+  /* Phase 14: the New chat pill joins the header — tighten the bar so
-  .nav-link { padding: 0.45rem 0.7rem; font-size: 0.9rem; }
+     brand + nav + pill fit at 360px without horizontal overflow (the
     brand text may ellipsize as the designated squeeze target). */
  .header-inner { gap: 0.6rem; }
  .brand { min-width: 0; }
  .brand-text {
    font-size: 0.8rem;
    letter-spacing: 0.04em;
    min-width: 0;
    overflow: hidden;
    text-overflow: ellipsis;
    white-space: nowrap;
  }
  .nav-link { padding: 0.4rem 0.55rem; font-size: 0.88rem; }
  .new-chat-btn { padding: 0.4rem 0.55rem; }
  .new-chat-label { display: none; }
  .new-chat-btn svg { display: block; }
  /* Phase 16: the auth pill goes icon-only like New chat — brand text
     ellipsizes as the designated squeeze target, no bar overflow. */
  .auth-link { padding: 0.4rem 0.55rem; }
  .auth-label { display: none; }
  .auth-link svg { display: block; }
  .steering-toggle { padding: 0.4rem 0.55rem; }
  /* Visually hidden, NOT display:none — the accessible name keeps the
     word "Tuning" next to the count badge. */
  .steering-label {
    position: absolute !important;
    width: 1px; height: 1px;
    margin: -1px; padding: 0;
    overflow: hidden;
    clip: rect(0 0 0 0);
    white-space: nowrap;
    border: 0;
  }
  .steering-note { padding: 0.3rem 0.3rem 0.3rem 0.7rem; }
  .tune-btn { min-height: 44px; }
  .msg-body { max-width: 92%; }
  .empty-state { padding: 1.75rem 1.1rem; margin-top: 0.25rem; }
  .empty-state-title { font-size: 1.25rem; }
  .suggestions { flex-wrap: nowrap; overflow-x: auto; justify-content: flex-start;
    padding-bottom: 0.4rem; -webkit-overflow-scrolling: touch; scrollbar-width: thin; }
  .suggestion-chip { flex: 0 0 auto; }
  .doc-header-inner { flex-wrap: nowrap; gap: 0.5rem; }  /* phase 12: fixed-height bar — no wrap, no extra padding */
  #doc-title { font-size: 1.1rem; }
  .doc-path { max-width: 16rem; }
  .doc-md { padding: 1.1rem 1rem; }
  .doc-raw { padding: 1rem; font-size: 0.8rem; }
  .composer { padding: 0.5rem; }
  .footer-inner { flex-direction: column; gap: 0.2rem; text-align: center; }
  main { padding-bottom: env(safe-area-inset-bottom, 0); }
@@ -0,0 +1,58 @@
 <!doctype html>
 <html lang="en">
 <head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
  <meta name="description" content="Read a document indexed in Brain of Reese.">
  <title>Document · Brain of Reese</title>
  <link rel="icon" href="data:image/svg+xml,%3Csvg%20xmlns=%22http://www.w3.org/2000/svg%22%20viewBox=%220%200%2064%2064%22%3E%3Cpath%20d=%22M32%204%2055%2018v28L32%2060%209%2046V18Z%22%20fill=%22%23121a2e%22%20stroke=%22%236d78f2%22%20stroke-width=%224%22%20stroke-linejoin=%22round%22/%3E%3Ccircle%20cx=%2232%22%20cy=%2232%22%20r=%226.5%22%20fill=%22%236d78f2%22/%3E%3Cpath%20d=%22M32%2025.5V16M32%2048v-9.5M25.5%2032H16M48%2032h-9.5%22%20stroke=%22%2322d3ee%22%20stroke-width=%223%22%20stroke-linecap=%22round%22/%3E%3C/svg%3E">
  <link rel="stylesheet" href="/assets/styles.css">
 </head>
 <body>
  <a class="skip-link" href="#main">Skip to content</a>
  <header class="doc-header">
    <div class="container doc-header-inner">
      <a class="doc-back" id="doc-back" href="/sources.html">
        <svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M19 12H5"/><path d="m12 19-7-7 7-7"/></svg>
        <span>Sources</span>
      </a>
      <div class="doc-title-block">
        <h1 id="doc-title">Loading…</h1>
        <div id="doc-meta" class="doc-meta"></div>
      </div>
    </div>
  </header>
  <main id="main" class="app-main" tabindex="-1">
    <!-- aria-live wraps the load → content swap so screen readers hear the
         document land (phase 10 a11y contract). -->
    <div class="container doc-shell" aria-live="polite">
      <div id="doc-content">
        <p class="doc-loading" role="status">Loading document…</p>
      </div>
      <div class="doc-not-found" id="doc-not-found" hidden>
        <div class="doc-not-found-glyph" aria-hidden="true">
          <svg viewBox="0 0 48 48" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round"><path d="M12 4h16l8 8v28a4 4 0 0 1-4 4H12a4 4 0 0 1-4-4V8a4 4 0 0 1 4-4Z"/><path d="M28 4v8h8"/><path d="m19 22 10 10M29 22l-10 10"/></svg>
        </div>
        <h2>Document not found</h2>
        <p class="doc-not-found-sub">
          This document isn't in the knowledge base — it may have been removed
          from the index, or the notes were re-imported.
        </p>
        <a class="doc-open-sources" href="/sources.html">Open Sources</a>
      </div>
    </div>
  </main>
  <footer class="app-footer">
    <div class="container footer-inner">
      <span>Powered by Reese's self-hosted models</span>
    </div>
  </footer>
  <script src="assets/markdown.js"></script>
  <script type="module" src="assets/document.js"></script>
 </body>
 </html>
@@ -21,6 +21,30 @@
        <a href="/" class="nav-link is-active" aria-current="page">Chat</a>
        <a href="/sources.html" class="nav-link">Sources</a>
      </nav>
      <!-- Phase 15: open the tuning-notes panel (stored in Postgres, read
           into every system prompt) — chat page only. -->
      <button type="button" class="steering-toggle" id="steering-toggle"
              aria-expanded="false" aria-controls="steering-panel">
        <svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M4 7h10M18 7h2M4 17h4M12 17h8"/><circle cx="15.5" cy="7" r="2.2"/><circle cx="9.5" cy="17" r="2.2"/></svg>
        <span class="steering-label">Tuning</span>
        <span class="steering-count" id="steering-count">0</span>
      </button>
      <!-- Phase 14: reset the local (localStorage) conversation — chat page only. -->
      <button type="button" class="new-chat-btn" id="new-chat-btn" aria-label="New chat">
        <svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M12 5v14M5 12h14"/></svg>
        <span class="new-chat-label">New chat</span>
      </button>
      <!-- Phase 16: single-admin auth — exactly one of Sign in / Sign out
           is visible; /api/whoami decides at load (app.js). Icon-only
           below 640px (aria-labels keep the accessible names). -->
      <a href="/login.html?next=/sources.html" class="auth-link" id="sign-in-link" hidden>
        <svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M10 4h8a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2h-8"/><path d="M4 12h11"/><path d="m12 9 3 3-3 3"/></svg>
        <span class="auth-label">Sign in</span>
      </a>
      <button type="button" class="auth-link" id="sign-out-btn" aria-label="Sign out" hidden>
        <svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M14 4H6a2 2 0 0 0-2 2v12a2 2 0 0 0 2 2h8"/><path d="M9 12h11"/><path d="m17 9 3 3-3 3"/></svg>
        <span class="auth-label">Sign out</span>
      </button>
    </div>
  </header>
@@ -31,6 +55,18 @@
        <span id="kb-banner-text"></span>
      </div>
      <!-- Phase 15: tuning-notes panel (stored notes, newest first). -->
      <section class="steering-panel" id="steering-panel" role="region"
               aria-label="Tuning notes" hidden>
        <div class="steering-panel-head">
          <h2 class="steering-panel-title">Tuning notes</h2>
          <p class="steering-panel-sub">Every note below steers all future answers.</p>
        </div>
        <ul class="steering-list" id="steering-list"></ul>
        <p class="steering-empty" id="steering-empty">No tuning notes yet — press “Tune” under any answer to add one.</p>
      </section>
      <p class="visually-hidden" id="steering-announcer" role="status" aria-live="polite" aria-atomic="true"></p>
      <section class="messages" id="messages" aria-live="polite" aria-label="Conversation with Brain of Reese">
        <div class="empty-state" id="empty-state">
          <div class="empty-state-glyph" aria-hidden="true">
@@ -73,6 +109,7 @@
    </div>
  </footer>
  <script src="assets/markdown.js"></script>
  <script type="module" src="/assets/app.js"></script>
 </body>
 </html>
@@ -0,0 +1,62 @@
 <!doctype html>
 <html lang="en">
 <head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
  <meta name="description" content="Admin sign-in for Brain of Reese — unlocks the full Sources catalog and answer tuning.">
  <title>Sign in · Brain of Reese</title>
  <link rel="icon" href="data:image/svg+xml,%3Csvg%20xmlns=%22http://www.w3.org/2000/svg%22%20viewBox=%220%200%2064%2064%22%3E%3Cpath%20d=%22M32%204%2055%2018v28L32%2060%209%2046V18Z%22%20fill=%22%23121a2e%22%20stroke=%22%236d78f2%22%20stroke-width=%224%22%20stroke-linejoin=%22round%22/%3E%3Ccircle%20cx=%2232%22%20cy=%2232%22%20r=%226.5%22%20fill=%22%236d78f2%22/%3E%3Cpath%20d=%22M32%2025.5V16M32%2048v-9.5M25.5%2032H16M48%2032h-9.5%22%20stroke=%22%2322d3ee%22%20stroke-width=%223%22%20stroke-linecap=%22round%22/%3E%3C/svg%3E">
  <link rel="stylesheet" href="/assets/styles.css">
 </head>
 <body>
  <a class="skip-link" href="#main">Skip to content</a>
  <!-- Standard app frame + sticky header (phase 12 consistency). -->
  <header class="app-header">
    <div class="container header-inner">
      <span class="brand">
        <svg class="brand-mark" aria-hidden="true" viewBox="0 0 64 64"><path d="M32 4 55 18v28L32 60 9 46V18Z" fill="#121a2e" stroke="#6d78f2" stroke-width="4" stroke-linejoin="round"/><circle cx="32" cy="32" r="6.5" fill="#6d78f2"/><path d="M32 25.5V16M32 48v-9.5M25.5 32H16M48 32h-9.5" stroke="#22d3ee" stroke-width="3" stroke-linecap="round"/></svg>
        <span class="brand-text">Brain of <strong>Reese</strong></span>
      </span>
      <nav class="app-nav" aria-label="Primary">
        <a href="/" class="nav-link">Chat</a>
        <a href="/sources.html" class="nav-link">Sources</a>
      </nav>
    </div>
  </header>
  <main id="main" class="app-main" tabindex="-1">
    <div class="container login-shell">
      <!-- Centered sign-in card (phase 16): one admin, one password. -->
      <section class="login-card" aria-labelledby="login-title">
        <h1 id="login-title">Sign in</h1>
        <p class="login-sub">
          One admin account, one password. Signing in unlocks the full
          Sources catalog and the answer-tuning controls — chat stays open
          to everyone either way.
        </p>
        <form id="login-form">
          <label class="visually-hidden" for="login-password">Admin password</label>
          <input
            id="login-password"
            name="password"
            type="password"
            autocomplete="current-password"
            required
          >
          <button type="submit" class="login-submit" id="login-submit">Sign in</button>
        </form>
        <p id="login-error" class="login-error" role="alert" hidden></p>
      </section>
    </div>
  </main>
  <footer class="app-footer">
    <div class="container footer-inner">
      <span>Powered by Reese's self-hosted models</span>
    </div>
  </footer>
  <script type="module" src="/assets/login.js"></script>
 </body>
 </html>
@@ -34,6 +34,21 @@
        </p>
      </div>
      <!-- Phase 16: anonymous sign-in gate. The catalog is what the
           login locks — the document viewer itself stays public (soft
           rule), so the copy says what stays open. -->
      <section class="sources-gate" id="sources-gate" aria-labelledby="sources-gate-title" hidden>
        <div class="sources-gate-glyph" aria-hidden="true">
          <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"><rect x="4" y="10" width="16" height="10" rx="2"/><path d="M8 10V7a4 4 0 0 1 8 0v3"/><circle cx="12" cy="14.5" r="1.4" fill="currentColor" stroke="none"/><path d="M12 16v2"/></svg>
        </div>
        <h2 id="sources-gate-title">Sign in to view the full catalog</h2>
        <p class="sources-gate-sub">
          The complete list of indexed documents is admin-only. Chat — and
          any document an answer cites — stays open to everyone.
        </p>
        <a class="sources-gate-link" href="/login.html?next=/sources.html">Sign in</a>
      </section>
      <div class="stat-cards" id="stat-cards">
        <div class="stat-card" role="group" aria-label="Document statistics">
          <span class="stat-value" id="stat-docs">–</span>
@@ -19,6 +19,10 @@ dependencies = [
    # --- LLM client (OpenAI-compatible, self-hosted "aipi") ---
    "httpx>=0.27,<1.0",
    "openai>=1.40,<3.0",
    # Phase 16: starlette's SessionMiddleware signs the session cookie with
    # itsdangerous — an OPTIONAL starlette extra ("full") since starlette 1.x,
    # so the app declares it directly (narrower than starlette[full]).
    "itsdangerous>=2.2,<3.0",
 ]
 [dependency-groups]
@@ -0,0 +1,129 @@
 """Evaluate hybrid retrieval against the live knowledge base (phase 09).
 Embeds each question via aipi, runs the same hybrid search the chat API
 uses (cosine top-N + FTS top-N, RRF-fused), and prints the top-5 documents
 with their cosine / fts / fused scores plus the honesty-gate verdict:
    uv run python -m scripts.eval_retrieval "How did I install gitlab?"
    uv run python -m scripts.eval_retrieval --from-file questions.txt
 Requires ``AIPI_KEY`` in the environment (same convention as
 ``scripts/llm_probe.py``) and an imported knowledge base
 (``python -m scripts.import_docs``). Exit code 0 when all questions were
 scored (deflections are a normal result — the verdict column shows them).
 """
 from __future__ import annotations
 import argparse
 import asyncio
 import os
 import sys
 from dotenv import load_dotenv
 load_dotenv()
 def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(
        prog="python -m scripts.eval_retrieval",
        description="Rank hybrid retrieval results for one or more questions.",
    )
    p.add_argument(
        "questions",
        nargs="*",
        metavar="QUESTION",
        help="one or more questions to evaluate",
    )
    p.add_argument(
        "--from-file",
        type=str,
        default=None,
        metavar="PATH",
        help="read questions from a file (one per line, blanks/# skipped)",
    )
    p.add_argument(
        "--top",
        type=int,
        default=5,
        help="documents to print per question (default: 5)",
    )
    return p
 def read_questions(args: argparse.Namespace) -> list[str]:
    questions = list(args.questions)
    if args.from_file:
        with open(args.from_file, encoding="utf-8") as f:
            questions.extend(
                line.strip() for line in f if line.strip() and not line.lstrip().startswith("#")
            )
    return questions
 async def _embed_all(llm, questions: list[str]) -> list[list[float]]:
    return await llm.embed(questions)
 def main(argv: list[str] | None = None) -> int:
    args = build_parser().parse_args(argv)
    api_key = os.environ.get("BOR_LLM_API_KEY") or os.environ.get("AIPI_KEY", "")
    if not api_key or api_key == "not-needed":
        print(
            "eval_retrieval: AIPI_KEY is required in the environment "
            "(same convention as scripts/llm_probe.py).",
            file=sys.stderr,
        )
        return 1
    questions = read_questions(args)
    if not questions:
        print("eval_retrieval: no questions given (positional or --from-file).", file=sys.stderr)
        return 1
    from app.config import get_settings
    from app.db import SessionLocal, db_available
    from app.rag.llm import LLMClient
    from app.rag.retriever import RetrievedChunk, retrieve
    settings = get_settings()
    if not db_available():
        print("eval_retrieval: Postgres is down — run `podman compose up -d db`.", file=sys.stderr)
        return 1
    llm = LLMClient(settings)
    vectors = asyncio.run(_embed_all(llm, questions))
    print(
        f"eval: threshold={settings.relevance_threshold} "
        f"vector_candidates={settings.hybrid_vector_candidates} "
        f"lexical_candidates={settings.hybrid_lexical_candidates} rrf_k={settings.rrf_k}"
    )
    with SessionLocal() as db:
        for question, vec in zip(questions, vectors, strict=True):
            chunks = retrieve(db, question, vec)
            best_cosine = max((c.cosine for c in chunks), default=0.0)
            fts_hits = sum(1 for c in chunks if c.fts_hit)
            verdict = (
                "LOW (deflect)"
                if best_cosine < settings.relevance_threshold and fts_hits == 0
                else "HIGH (answer)"
            )
            print(f"\nquestion: {question!r}")
            print(f"  gate: best_cosine={best_cosine:.4f} fts_hits={fts_hits} -> {verdict}")
            # Best chunk per document, in fused rank order.
            best_by_doc: dict[str, RetrievedChunk] = {}
            for c in chunks:
                key = f"{c.document.source}/{c.document.path}"
                if key not in best_by_doc:
                    best_by_doc[key] = c
            for i, c in enumerate(list(best_by_doc.values())[: args.top], start=1):
                print(
                    f"  {i}. {c.document.source}/{c.document.path}  "
                    f"cosine={c.cosine:.4f} fts={int(c.fts_hit)} fused={c.score:.5f}  "
                    f"({c.document.title})"
                )
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -1,16 +1,20 @@
-"""Import markdown directories into the Brain of Reese knowledge base.
+"""Import A9-format directories into the Brain of Reese knowledge base.
 Examples::
    uv run python -m scripts.import_docs                      # ~/Homelab + ~/Deployments
    uv run python -m scripts.import_docs --source ~/OtherDocs # extra dir (repeatable)
-    uv run python -m scripts.import_docs --prune              # also drop deleted files
+    uv run python -m scripts.import_docs --prune              # also drop deleted/out-of-scope files
    uv run python -m scripts.import_docs --limit 5            # debug: first 5 files only
-Only ``*.md`` files are imported; non-content dirs (``.venv``,
+Imported formats (PLAN anchor A9, revised): ``md, markdown, txt, yaml,
 yml, json, py`` (case-insensitive; narrow with ``BOR_IMPORT_EXTENSIONS``).
 Any path with a dot-prefixed component (hidden files/dirs — vendored
 caches) is skipped, along with non-content dirs (``.venv``,
 ``node_modules``, ``.git``, ``__pycache__``, ``.pytest_cache``, ``dist``,
-``build``) are skipped (PLAN anchor A9). Re-runs are cheap: files are
+``build``). Re-runs are cheap: files are diffed by sha256 and unchanged
-diffed by sha256 and unchanged ones are not re-embedded.
+ones are not re-embedded; ``--prune`` also drops documents whose files no
 longer match the format filter.
 """
 from __future__ import annotations
@@ -31,7 +35,7 @@ DEFAULT_SOURCES: list[Path] = [Path("~/Homelab"), Path("~/Deployments")]
 def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(
        prog="python -m scripts.import_docs",
-        description="Import *.md files into the Brain of Reese knowledge base.",
+        description="Import A9-format files (md/txt/yaml/json/py) into the knowledge base.",
    )
    p.add_argument(
        "--source",
@@ -43,7 +47,7 @@ def build_parser() -> argparse.ArgumentParser:
    p.add_argument(
        "--prune",
        action="store_true",
-        help="also delete documents whose files no longer exist",
+        help="also delete documents whose files no longer exist or match the format filter",
    )
    p.add_argument(
        "--limit",
@@ -74,7 +78,7 @@ def main(argv: list[str] | None = None) -> int:
        f"import_docs: files={summary.files} added={summary.added} "
        f"updated={summary.updated} unchanged={summary.unchanged} "
        f"pruned={summary.pruned} errors={summary.errors} chunks={summary.chunks} "
-        f"embed_batches={summary.embed_batches}"
+        f"embed_batches={summary.embed_batches} formats={summary.format_counts()}"
    )
    # Non-zero if any file failed, so cron/CI notice — the rest of the KB
    # was imported and the failed files are retried on the next run.
@@ -1,14 +1,30 @@
 """Shared fixtures for unit + integration tests."""
 from __future__ import annotations
 import os
 from collections.abc import Iterator
 import pytest
 from fastapi.testclient import TestClient
 from sqlalchemy.orm import Session
-from app.db import SessionLocal, db_available
+# In-process integration/E2E tests drive the app with *mock* embeddings
-from app.main import app as fastapi_app
+# (bag-of-words, cosine ~0.1–0.8), not the live aipi model — so the honesty
 # gate is calibrated to the mock's distribution, mirroring tests/e2e/
 # conftest.py. Must be set before ``app.main`` (below) caches settings.
 # The production default stays 0.62 (app/config.py, A8 revised).
 os.environ.setdefault("BOR_RELEVANCE_THRESHOLD", "0.30")
 # Phase 16: single-admin auth is fail-loud — create_app() refuses to boot
 # without both vars, and app.main (imported below) builds the app at
 # import time. Set known test values first, same pattern as the threshold.
 ADMIN_PASSWORD = "test-admin-password"
 SESSION_SECRET = "test-session-secret-0123456789abcdef0123456789abcdef"
 os.environ.setdefault("BOR_ADMIN_PASSWORD", ADMIN_PASSWORD)
 os.environ.setdefault("BOR_SESSION_SECRET", SESSION_SECRET)
 from app.db import SessionLocal, db_available  # noqa: E402
 from app.main import app as fastapi_app  # noqa: E402
@pytest.fixture()
@@ -16,6 +32,19 @@ def client() -> TestClient:
    return TestClient(fastapi_app)
@pytest.fixture()
 def admin_client(client: TestClient) -> TestClient:
    """A client signed in as the single admin (phase 16).
    TestClient keeps its cookie jar across requests, so one login covers
    every subsequent request of the test. Use it for the admin-only
    surface (``GET /api/docs``, ``/api/steering``).
    """
    r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
    assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
    return client
@pytest.fixture()
 def db() -> Iterator[Session]:
    """Real Postgres session (``podman compose up -d db``).
@@ -0,0 +1,38 @@
 """Shared Playwright auth helper (phase 16).
 ``login`` drives the REAL form login on /login.html (fill → submit →
 redirect) so every story that needs the admin does exactly what a human
 would — no cookie surgery. ``password=None`` uses the shared E2E admin
 password (success path); pass a wrong value to drive the error state
 (no redirect, ``#login-error`` role=alert visible, still anonymous).
 """
 from __future__ import annotations
 from playwright.sync_api import Page, expect
 from e2e.conftest import ADMIN_PASSWORD  # noqa: F401  (re-exported for tests)
 DEFAULT_NEXT = "/sources.html"
 def login(page: Page, app_url: str, password: str | None = None, next: str | None = None) -> None:
    """Perform the real form login and wait for its outcome.
    * correct password (or ``password=None`` → the shared admin password)
      → redirects to ``next`` (default ``/sources.html``);
    * wrong password → ``#login-error`` (role=alert) is visible, the URL
      never changes, and the visitor is still anonymous.
    """
    attempt = ADMIN_PASSWORD if password is None else password
    url = f"{app_url}/login.html"
    if next is not None:
        url += f"?next={next}"
    page.goto(url)
    expect(page.locator("#login-password")).to_be_visible()
    page.fill("#login-password", attempt)
    page.click("#login-form button[type=submit]")
    if attempt != ADMIN_PASSWORD:
        expect(page.locator("#login-error")).to_be_visible(timeout=15_000)
        expect(page).to_have_url(url)  # no redirect on failure
        return
    expect(page).to_have_url(app_url + (next or DEFAULT_NEXT), timeout=30_000)
@@ -31,6 +31,13 @@ MOCK_PORT = int(os.environ.get("E2E_MOCK_PORT", "8901"))
 APP_URL = f"http://127.0.0.1:{APP_PORT}"
 USE_REAL_LLM = os.environ.get("E2E_REAL_LLM") == "1"
 # Phase 16: the app under test boots with single-admin auth configured
 # (fail-loud otherwise). Known E2E values — the shared form-login helper
 # (tests/e2e/auth_helpers.py) uses ADMIN_PASSWORD; the secret is fixed so
 # session cookies stay valid across a session-scoped app restart.
 ADMIN_PASSWORD = "e2e-admin-password"
 SESSION_SECRET = "e2e-session-secret-0123456789abcdef0123456789abcdef"
 def _wait_http(url: str, timeout: float = 40.0) -> None:
    deadline = time.monotonic() + timeout
@@ -82,7 +89,16 @@ def app_server(mock_llm: int) -> Iterator[str]:
        if USE_REAL_LLM
        else f"http://127.0.0.1:{mock_llm}/v1"
    )
    # The E2E mock's token-overlap embeddings have their own score
    # distribution (phase 09) — the app under test gets the mock-calibrated
    # threshold so every story suite keeps its deterministic gate behavior.
    # The production default stays 0.62 (re-tuned against the real
    # `embed` model's 0.41–0.84 cosine range, PLAN A8).
    env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
    env.setdefault("BOR_DATABASE_URL", "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese")
    # Phase 16: admin auth must be set or create_app() refuses to boot.
    env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
    env["BOR_SESSION_SECRET"] = SESSION_SECRET
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "app.main:app",
         "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
@@ -9,10 +9,22 @@ Implements just enough of the aipi surface:
  unrelated ones score low and trigger honest deflection.
 * ``POST /v1/chat/completions`` — streaming (SSE) or not. The content keys
  off markers in the system prompt:
  - user message containing ``write a long answer`` -> a ~900-word
    deterministic numbered answer (long-answers story, phase 11)
  - ``DEFLECT_MODE``  -> honest "I haven't done anything like that" answer
  - otherwise         -> upbeat answer quoting the provided document context
  - user message containing ``pretend to think slowly`` -> 3s warm-up delay
    (used by the loading-feedback story).
  - user message containing ``think out loud`` -> the answer is preceded by
    ~800 chars of deterministic ``reasoning_content`` chunks (the
    thinking-display story, phase 17).
  - system prompt containing ``<tuning>`` (phase 15, steering notes) ->
    the composed answer ends with `` (tuning: <first note line>)`` —
    makes prompt injection observable in the UI deterministically.
  ``max_tokens`` is honored deterministically (token ≈ whitespace word),
  like a real endpoint: an answer longer than the cap is truncated. This
  is what makes the phase-11 truncation regression observable.
 """
 from __future__ import annotations
@@ -63,23 +75,107 @@ def _context(body: dict[str, Any]) -> str:
    return max((m.get("content", "") for m in msgs), key=len)
 LONG_ANSWER_TRIGGER = "write a long answer"
 #: ~920 words — comfortably past the old hard 700-token cap (where the
 #: tail would be cut) yet short enough to stream in ~8s at the mock's
 #: per-chunk pacing.
 LONG_ANSWER_LINES = 40
 LONG_ANSWER_END = "LONG-ANSWER-END"
 #: Phase 17 (thinking-display story): a user message containing this
 #: substring (case-insensitive) is answered with a deterministic
 #: ``reasoning_content`` stream ahead of the content — same convention as
 #: the other user-message triggers above. Existing E2E questions do not
 #: contain the substring, so every other suite is unaffected.
 THINKING_TRIGGER = "think out loud"
 def long_answer() -> str:
    """~900-word deterministic walkthrough (phase 11): numbered steps plus
    a unique final line that must survive the stream untruncated."""
    lines = [
        f"{i}. Step {i}: configure node-{i} with the homelab defaults and "
        f"verify that step {i} of the long walkthrough is complete before moving on."
        for i in range(1, LONG_ANSWER_LINES + 1)
    ]
    lines.append(LONG_ANSWER_END)
    return "\n".join(lines)
 #: First numbered note line of a ``<tuning>`` section (phase 15).
 _TUNING_BLOCK_RE = re.compile(r"<tuning>\n(.*?)\n</tuning>", re.S)
 _NOTE_LINE_RE = re.compile(r"^\d+\.\s*(.+)$")
 def first_tuning_note(system: str) -> str | None:
    """The first steering note in the system prompt, or ``None``.
    The prompt numbers notes 1..N oldest-first (see
    ``app.rag.prompts.build_steering_section``); the mock echoes the first
    one into its answer so prompt injection is observable in the UI.
    """
    block = _TUNING_BLOCK_RE.search(system)
    if not block:
        return None
    for line in block.group(1).splitlines():
        m = _NOTE_LINE_RE.match(line.strip())
        if m:
            return m.group(1).strip()
    return None
 def compose_answer(body: dict[str, Any]) -> str:
    system = _system(body)
    user = _user(body)
-    if "DEFLECT_MODE" in system:
+    if LONG_ANSWER_TRIGGER in user.lower():
-        return (
+        answer = long_answer()
    elif "DEFLECT_MODE" in system:
        answer = (
            "Ah — I haven't done anything like that, so I don't want to make stuff up! "
            "You're thinking bigger than my notes for a second. Try asking about "
            "kubernetes, backups, or deploying a new service — I know those inside out. "
            "You've got this!"
        )
    else:
        ctx = _context(body)
        snippet = ctx[:220].replace("\n", " ").strip()
-    return (
+        answer = (
            f"Great question — you've absolutely got this! Here's what my notes say about "
            f"“{user.strip()[:80]}”: {snippet}… That's the gist from the docs; happy to "
            "dig into any of it. (Deterministic mock answer for E2E.)"
        )
    # Steering (phase 15): when the system prompt carries <tuning>, the
    # answer ends with the first note — deterministically observable.
    note = first_tuning_note(system)
    if note:
        answer = f"{answer} (tuning: {note})"
    return answer
 def compose_thinking(body: dict[str, Any]) -> str:
    """Deterministic reasoning scratchpad (thinking-display story, phase 17).
    A fixed 4-line "Step 1… Step 4" template quoting the first ~60 chars
    of the user question: unique per question, byte-stable across runs,
    ~700–900 chars total (≈ 60–75 frames at the mock's 12-char/0.02s
    pacing). The ``Step 2: Check my notes`` line fragment is what the E2E
    assertions key off.
    """
    q = _user(body).strip()[:60]
    return (
        f"Step 1: Read the question carefully — “{q}” — and figure out what kind of "
        "answer it wants (a how-to, a lookup, or a design decision) before touching "
        "the docs, so I don't over- or under-answer.\n"
        "Step 2: Check my notes for the closest match. The homelab kubernetes file "
        "is the obvious candidate, but I should also consider whether a deployments "
        "note covers the same ground better.\n"
        "Step 3: Re-read the relevant sections top to bottom so every specific — "
        "hosts, versions, ports, schedules — is exact as written rather than "
        "remembered, and note which document each fact comes from.\n"
        "Step 4: Draft the answer around those specifics, keep it tight with short "
        "paragraphs and bullets where it helps, cite the documents by path, and "
        "double-check that nothing is invented."
    )
@app.post("/__shutdown__")
@@ -126,11 +222,31 @@ def embeddings(body: dict[str, Any]) -> dict[str, Any]:
    }
-def _sse_stream(answer: str, delay: float) -> Any:
+def _sse_stream(answer: str, delay: float, thinking: str = "") -> Any:
    """SSE frames for one chat completion (phase 17: + reasoning).
    When ``thinking`` is non-empty its 12-char slices go out FIRST as
    ``delta.reasoning_content`` frames — same 0.02s cadence and envelope
    as the content frames, the aipi wire convention (reasoning before
    content). Without ``thinking`` the output is byte-identical to the
    content-only stream, so the other story suites are unaffected.
    """
    model = "turbo"
    chunk_id = f"chatcmpl-{uuid.uuid4()}"
    if delay:
        time.sleep(delay)
    for piece in re.findall(r".{1,12}", thinking, re.S):
        payload = {
            "id": chunk_id,
            "object": "chat.completion.chunk",
            "created": int(time.time()),
            "model": model,
            "choices": [
                {"index": 0, "delta": {"reasoning_content": piece}, "finish_reason": None}
            ],
        }
        yield f"data: {json_dumps(payload)}\n\n"
        time.sleep(0.02)
    for piece in re.findall(r".{1,12}", answer, re.S):
        payload = {
            "id": chunk_id,
@@ -163,29 +279,43 @@ def json_dumps(obj: dict[str, Any]) -> str:
    return json.dumps(obj)
 def _apply_max_tokens(answer: str, max_tokens: Any) -> str:
    """Deterministic stand-in for the endpoint's output cap: one token ≈
    one whitespace-separated word. Answers within the cap pass through
    byte-identical, so existing (short) answers are unaffected."""
    if not isinstance(max_tokens, int) or max_tokens <= 0:
        return answer
    words = answer.split()
    if len(words) <= max_tokens:
        return answer
    return " ".join(words[:max_tokens])
@app.post("/v1/chat/completions")
 def chat_completions(body: dict[str, Any]) -> Any:
-    answer = compose_answer(body)
+    answer = _apply_max_tokens(compose_answer(body), body.get("max_tokens"))
    delay = 3.0 if "pretend to think slowly" in _user(body) else 0.0
    thinking = compose_thinking(body) if THINKING_TRIGGER in _user(body).lower() else ""
    if not body.get("stream"):
        message: dict[str, Any] = {"role": "assistant", "content": answer}
        if thinking:
            # Harmless future-proofing: the app only uses streaming, but a
            # non-streaming client that reads the field gets the reasoning.
            message["reasoning_content"] = thinking
        return {
            "id": f"chatcmpl-{uuid.uuid4()}",
            "object": "chat.completion",
            "created": int(time.time()),
            "model": body.get("model", "turbo"),
            "choices": [
-                {
+                {"index": 0, "message": message, "finish_reason": "stop"}
                    "index": 0,
                    "message": {"role": "assistant", "content": answer},
                    "finish_reason": "stop",
                }
            ],
            "usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150},
        }
    return StreamingResponse(
-        _sse_stream(answer, delay),
+        _sse_stream(answer, delay, thinking=thinking),
        media_type="text/event-stream",
        headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
    )
@@ -0,0 +1,326 @@
 """Phase 16 E2E (Playwright): single-admin sign-in (A10 revised).
 Story: ``.agent/user_stories/admin-auth.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_admin_auth.py -v --no-cov
 The E2E app server boots with ``BOR_ADMIN_PASSWORD``/``BOR_SESSION_SECRET``
 set (``tests/e2e/conftest.py``); the shared ``tests/e2e/auth_helpers.py::login``
 performs the real form login on /login.html.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_anonymous_chat_without_tuning``
 2. ``test_anonymous_sources_gated_viewer_open``
 3. ``test_login_wrong_password_shows_error``
 4. ``test_admin_login_unlocks_sources_and_tuning``
 5. ``test_logout_returns_to_anonymous``
 6. ``test_login_page_a11y``
 """
 from __future__ import annotations
 import asyncio
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import ADMIN_PASSWORD, login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
 DOC_TITLE = "Kubernetes Homelab Cluster"
 DOC_VIEWER_URL = "/document.html?source=docs&path=homelab%2Fkubernetes.md"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread (Playwright owns the test loop)."""
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log + steering notes), optionally re-seed."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 def _ask(page: Page, question: str) -> None:
    """Send one turn and wait until the grounded answer has fully landed."""
    page.fill("#message-input", question)
    page.click("#send-btn")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(
        MOCK_ANSWER_MARKER, timeout=30_000
    )
    expect(page.locator("#send-btn")).to_be_enabled()
    expect(page.locator("#send-label")).to_have_text("Send")
 # ---------------------------------------------------------------------------
 # 1. Anonymous: chat works, the tuning UI is gone, Sign in is offered
 # ---------------------------------------------------------------------------
 def test_anonymous_chat_without_tuning(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    # Header: Sign in offered, Sign out not.
    expect(page.locator("#sign-in-link")).to_be_visible()
    expect(page.locator("#sign-in-link")).to_have_attribute(
        "href", "/login.html?next=/sources.html"
    )
    expect(page.locator("#sign-out-btn")).to_be_hidden()
    # Chat still streams a grounded answer (with source chips) for
    # anonymous visitors…
    _ask(page, QUESTION)
    expect(page.locator(".msg.brain .source-chip", has_text="kubernetes.md")).to_have_count(1)
    # …but the tuning UI is completely gone: no Tune button (new or
    # restored), no Tuning toggle or panel in the DOM at all.
    expect(page.locator(".msg.brain .tune-btn")).to_have_count(0)
    expect(page.locator("#steering-toggle")).to_have_count(0)
    expect(page.locator("#steering-panel")).to_have_count(0)
    # A reload (the phase-14 restore path) must not bring it back.
    page.reload()
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(MOCK_ANSWER_MARKER)
    expect(page.locator(".msg.brain .tune-btn")).to_have_count(0)
    expect(page.locator("#steering-toggle")).to_have_count(0)
    expect(page.locator("#sign-in-link")).to_be_visible()
 # ---------------------------------------------------------------------------
 # 2. Anonymous: Sources gated, the document viewer stays open (soft rule)
 # ---------------------------------------------------------------------------
 def test_anonymous_sources_gated_viewer_open(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    api_docs_calls: list[str] = []
    page.on(
        "request",
        lambda r: api_docs_calls.append(r.url) if "/api/docs" in r.url else None,
    )
    page.goto(f"{app_url}/sources.html")
    # The gate, with its sign-in link (≥44px) — not a redirect.
    gate = page.locator("#sources-gate")
    expect(gate).to_be_visible()
    expect(gate).to_contain_text("Sign in to view the full catalog")
    link = gate.locator("a[href='/login.html?next=/sources.html']")
    expect(link).to_have_count(1)
    box = link.bounding_box()
    assert box is not None and box["height"] >= 44
    # Stat cards + table hidden…
    expect(page.locator("#stat-cards")).to_be_hidden()
    expect(page.locator("#docs-table")).to_be_hidden()
    expect(page.locator("#sources-empty")).to_be_hidden()
    # …and NO /api/docs call was ever made.
    assert api_docs_calls == [], f"anonymous sources page called /api/docs: {api_docs_calls}"
    # The soft rule: any seeded document still opens by direct URL.
    page.goto(app_url + DOC_VIEWER_URL)
    expect(page.locator("#doc-title")).to_have_text(DOC_TITLE, timeout=15_000)
    expect(page.locator("#doc-content")).not_to_be_empty()
 # ---------------------------------------------------------------------------
 # 3. Wrong password → role=alert error, no redirect, still anonymous
 # ---------------------------------------------------------------------------
 def test_login_wrong_password_shows_error(page: Page, app_url: str, db_ready: None) -> None:
    _reset_db(mock_port=0, seed=False)
    page.set_default_timeout(30_000)
    login(page, app_url, password="definitely-not-the-password")
    error = page.locator("#login-error")
    expect(error).to_be_visible()
    assert error.get_attribute("role") == "alert"
    expect(error).not_to_be_empty()
    # No redirect happened…
    expect(page).to_have_url(app_url + "/login.html")
    # …and the server agrees: still anonymous, no session cookie set.
    who = page.evaluate("() => fetch('/api/whoami').then((r) => r.json())")
    assert who == {"authenticated": False, "role": "anonymous"}
    # The form stays usable: the correct password now succeeds.
    page.fill("#login-password", ADMIN_PASSWORD)
    page.click("#login-form button[type=submit]")
    expect(page).to_have_url(app_url + "/sources.html", timeout=30_000)
 # ---------------------------------------------------------------------------
 # 4. Correct password → Sources + tuning unlocked, Sign out offered
 # ---------------------------------------------------------------------------
 def test_admin_login_unlocks_sources_and_tuning(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    # Real form login (default password + next) lands on the catalog.
    login(page, app_url)
    expect(page).to_have_url(app_url + "/sources.html")
    expect(page.locator("#sources-gate")).to_be_hidden()
    expect(page.locator("#stat-docs")).to_have_text("8")
    expect(page.locator("#stat-chunks")).not_to_have_text("–")
    expect(page.locator("#docs-table")).to_be_visible()
    expect(page.locator("#docs-tbody tr")).to_have_count(8)
    # Chat: the tuning UI is back — header toggle with count badge,
    # Sign out instead of Sign in, Tune under the answer.
    page.goto(app_url)
    expect(page.locator("#sign-out-btn")).to_be_visible()
    expect(page.locator("#sign-in-link")).to_be_hidden()
    toggle = page.locator("#steering-toggle")
    expect(toggle).to_be_visible()
    expect(page.locator("#steering-count")).to_have_text("0")
    _ask(page, QUESTION)
    tune = page.locator(".msg.brain .tune-btn").last
    expect(tune).to_be_visible()
    box = tune.bounding_box()
    assert box is not None and box["height"] >= 44
    # The API agrees: admin, and the gated endpoints answer now.
    who = page.evaluate("() => fetch('/api/whoami').then((r) => r.json())")
    assert who == {"authenticated": True, "role": "admin"}
    docs_status = page.evaluate("() => fetch('/api/docs').then((r) => r.status)")
    assert docs_status == 200
 # ---------------------------------------------------------------------------
 # 5. Sign out → anonymous again (gate back, tuning gone, restore untunable)
 # ---------------------------------------------------------------------------
 def test_logout_returns_to_anonymous(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    login(page, app_url, next="/")  # straight into the chat
    expect(page).to_have_url(app_url + "/")
    expect(page.locator("#sign-out-btn")).to_be_visible()
    expect(page.locator("#steering-toggle")).to_be_visible()
    # One grounded turn as admin (persisted to localStorage by phase 14).
    _ask(page, QUESTION)
    expect(page.locator(".msg.brain .tune-btn").last).to_be_visible()
    # Sign out: POST /api/logout + reload → anonymous again.
    page.click("#sign-out-btn")
    expect(page.locator("#sign-in-link")).to_be_visible(timeout=30_000)
    expect(page.locator("#sign-out-btn")).to_be_hidden()
    expect(page.locator("#steering-toggle")).to_have_count(0)
    expect(page.locator("#steering-panel")).to_have_count(0)
    # The restored conversation came back… without any Tune button.
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(MOCK_ANSWER_MARKER)
    expect(page.locator(".msg.brain .tune-btn")).to_have_count(0)
    # The server agrees, and Sources is gated again.
    who = page.evaluate("() => fetch('/api/whoami').then((r) => r.json())")
    assert who == {"authenticated": False, "role": "anonymous"}
    page.goto(f"{app_url}/sources.html")
    expect(page.locator("#sources-gate")).to_be_visible()
    expect(page.locator("#docs-table")).to_be_hidden()
 # ---------------------------------------------------------------------------
 # 6. Login page accessibility (WCAG 2.1 AA basics)
 # ---------------------------------------------------------------------------
 def test_login_page_a11y(page: Page, app_url: str, db_ready: None) -> None:
    _reset_db(mock_port=0, seed=False)
    page.set_default_timeout(30_000)
    page.goto(f"{app_url}/login.html")
    # Standard app frame: landmarks + skip link, no CDN tags.
    expect(page.locator("header.app-header")).to_have_count(1)
    expect(page.locator("nav[aria-label='Primary']")).to_have_count(1)
    expect(page.locator("main#main")).to_have_count(1)
    expect(page.locator("footer.app-footer")).to_have_count(1)
    expect(page.locator(".skip-link")).to_have_count(1)
    html = page.content()
    assert 'src="https://' not in html and 'href="https://' not in html
    # The password field is labeled (visually-hidden <label for=…>).
    pw = page.get_by_label("Admin password")
    expect(pw).to_have_count(1)
    expect(pw.first).to_have_attribute("type", "password")
    expect(pw.first).to_have_attribute("autocomplete", "current-password")
    # Touch targets ≥44px (field + submit).
    for el in (pw.first, page.locator("#login-form button[type=submit]")):
        box = el.bounding_box()
        assert box is not None and box["height"] >= 44, f"target too small: {box}"
    # Keyboard focus draws the 3px focus-visible outline.
    page.focus("#login-password")
    outline = page.evaluate(
        "() => getComputedStyle(document.querySelector('#login-password')).outlineWidth"
    )
    assert outline == "3px", f"focus-visible outline missing: {outline!r}"
    # Errors are announced through the role=alert region.
    error = page.locator("#login-error")
    assert error.get_attribute("role") == "alert"
    expect(error).to_be_hidden()
    page.fill("#login-password", "wrong")
    page.click("#login-form button[type=submit]")
    expect(error).to_be_visible(timeout=15_000)
    # A signed-in visit to /login.html?next=/ redirects immediately.
    page.fill("#login-password", ADMIN_PASSWORD)
    page.click("#login-form button[type=submit]")
    expect(page).to_have_url(app_url + "/sources.html", timeout=30_000)
    page.goto(f"{app_url}/login.html?next=/")
    expect(page).to_have_url(app_url + "/", timeout=30_000)
@@ -0,0 +1,297 @@
 """Phase 14 E2E (Playwright): the chat conversation survives a refresh.
 Story: ``.agent/user_stories/chat-persistence.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_chat_persistence.py -v --no-cov
 The conversation is a durable LOCAL session (localStorage key
 ``bor.chat.v1`` — A10 keeps the API stateless). Each test gets a fresh
 browser context (the shared conftest's ``page`` fixture calls
 ``browser.new_page``), so localStorage is clean by construction: the
 fresh-context tests start with the empty state exactly as before phase 14.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_conversation_survives_reload``
 2. ``test_deflected_turn_restores_styling``
 3. ``test_new_chat_clears_conversation``
 4. ``test_persists_across_page_navigation``
 """
 from __future__ import annotations
 import asyncio
 import json
 import re
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 OFF_TOPIC = "How do I bake sourdough bread?"
 MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
 DEFLECT_PHRASE = r"haven't done anything like that"
 STORAGE_KEY = "bor.chat.v1"
 #: Phase 10 viewer URL + phase 13 back=/ (the restored chip must be
 #: byte-identical to the live-rendered one).
 CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 def _stored(page: Page) -> str | None:
    """Raw localStorage payload for the chat (None when the key is absent)."""
    return page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
 def _stored_parsed(page: Page) -> dict[str, Any]:
    raw = _stored(page)
    assert raw is not None, "the conversation key must exist in localStorage"
    return json.loads(raw)
 def _ask(page: Page, question: str) -> None:
    """Send one turn and wait until the grounded answer has fully landed."""
    page.fill("#message-input", question)
    page.click("#send-btn")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(
        MOCK_ANSWER_MARKER, timeout=30_000
    )
    expect(page.locator("#send-btn")).to_be_enabled()
    expect(page.locator("#send-label")).to_have_text("Send")
 def _ask_deflected(page: Page, question: str) -> None:
    """Send an off-topic turn and wait until the deflected answer landed."""
    page.fill("#message-input", question)
    page.click("#send-btn")
    bubble = page.locator(".msg.brain.is-deflected .bubble").first
    bubble.wait_for(state="visible", timeout=30_000)
    expect(bubble).to_contain_text(re.compile(DEFLECT_PHRASE, re.IGNORECASE), timeout=30_000)
    expect(page.locator("#send-btn")).to_be_enabled()
    expect(page.locator("#send-label")).to_have_text("Send")
 # ---------------------------------------------------------------------------
 # 1. Refresh: the whole conversation comes back exactly as left
 # ---------------------------------------------------------------------------
 def test_conversation_survives_reload(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    summary = _reset_db(mock_llm, seed=True)
    assert summary is not None and summary.added == 8  # A9 formats
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, QUESTION)
    # The turn is persisted: versioned payload, RAW text (no HTML), and the
    # brain message carries the done metadata.
    stored = _stored_parsed(page)
    assert stored["v"] == 1
    assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
    assert stored["messages"][0]["text"] == QUESTION
    brain = stored["messages"][1]
    assert MOCK_ANSWER_MARKER in brain["text"]
    assert "<" not in brain["text"], "persisted brain text must be raw, not rendered HTML"
    assert brain["deflected"] is False
    assert any(s["path"] == "homelab/kubernetes.md" for s in brain["sources"])
    # Refresh — the same context keeps its localStorage.
    page.reload()
    expect(page.locator("#empty-state")).to_be_hidden()
    # Both bubbles restored: text + the source chip with the exact viewer URL.
    expect(page.locator(".msg.user .bubble")).to_have_count(1)
    expect(page.locator(".msg.user .bubble")).to_contain_text(QUESTION)
    bubble = page.locator(".msg.brain .bubble")
    expect(bubble).to_have_count(1)
    expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip).to_have_count(1)
    expect(chip.first).to_have_attribute("href", CHIP_HREF)
    expect(chip.first).to_have_attribute("target", "_blank")
    # The restore is read-only: storage still holds the same two messages.
    assert [m["who"] for m in _stored_parsed(page)["messages"]] == ["user", "brain"]
    # And the restored chat is live: a follow-up turn extends it.
    _ask(page, "What about the nodes?")
    expect(page.locator(".msg.user .bubble")).to_have_count(2)
    assert len(_stored_parsed(page)["messages"]) == 4
 # ---------------------------------------------------------------------------
 # 2. Deflected turn: amber styling + "Maybe try" chips survive a refresh
 # ---------------------------------------------------------------------------
 def test_deflected_turn_restores_styling(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask_deflected(page, OFF_TOPIC)
    # The deflected metadata (suggestions) is persisted with the answer.
    stored = _stored_parsed(page)
    brain = stored["messages"][-1]
    assert brain["who"] == "brain"
    assert brain["deflected"] is True
    assert len(brain["suggestions"]) >= 2
    page.reload()
    # Amber deflected bubble + "Maybe try" chips come back, styled.
    expect(page.locator("#empty-state")).to_be_hidden()
    expect(page.locator(".msg.user .bubble")).to_contain_text(OFF_TOPIC)
    restored = page.locator(".msg.brain.is-deflected .bubble")
    expect(restored).to_have_count(1)
    expect(restored.first).to_contain_text(re.compile(DEFLECT_PHRASE, re.IGNORECASE))
    style = restored.first.evaluate("el => getComputedStyle(el)")
    assert style["backgroundColor"] == "rgb(43, 33, 16)"  # --accent-bg (dark theme)
    assert style["borderTopColor"] == "rgb(245, 158, 11)"  # --accent-line
    # The chips are restored from the stored suggestions — same texts, order,
    # and still one-tap-submittable (the shared chip component).
    chips = page.locator(".msg.brain.is-deflected .maybe-try .suggestion-chip")
    expect(chips.first).to_be_visible()
    restored_texts = [chips.nth(i).inner_text() for i in range(chips.count())]
    assert restored_texts == [s.strip() for s in brain["suggestions"] if s.strip()]
    chips.first.click()
    expect(page.locator("#message-input")).to_have_value("")
    expect(page.locator(".msg.user .bubble")).to_have_count(2, timeout=30_000)
    expect(page.locator(".msg.brain .bubble").nth(1)).to_contain_text(
        MOCK_ANSWER_MARKER, timeout=30_000
    )
 # ---------------------------------------------------------------------------
 # 3. "New chat": clear the conversation, back to the empty state
 # ---------------------------------------------------------------------------
 def test_new_chat_clears_conversation(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, QUESTION)
    expect(page.locator(".msg")).to_have_count(2)
    assert _stored(page) is not None
    # The reset control: ghost pill in the chat header, ≥44px, accessible name.
    btn = page.locator("#new-chat-btn")
    expect(btn).to_be_visible()
    expect(btn).to_have_attribute("type", "button")
    expect(btn).to_have_attribute("aria-label", "New chat")
    box = btn.bounding_box()
    assert box is not None and box["height"] >= 44
    btn.click()
    # Conversation gone, empty state + suggestions back, storage key cleared.
    expect(page.locator(".msg")).to_have_count(0)
    expect(page.locator("#empty-state")).to_be_visible()
    expect(page.locator("#suggestions .suggestion-chip").first).to_be_visible(timeout=15_000)
    assert _stored(page) is None, "New chat must clear the localStorage key"
    # Confirmation via the existing live region (#send-status, aria-live=polite).
    expect(page.locator("#send-status")).to_contain_text("New chat started")
    # And it is a clean slate: a fresh turn starts a fresh conversation.
    _ask(page, QUESTION)
    stored = _stored_parsed(page)
    assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
    assert stored["messages"][0]["text"] == QUESTION
 # ---------------------------------------------------------------------------
 # 4. Navigation: a trip to Sources and back keeps the conversation
 # ---------------------------------------------------------------------------
 def test_persists_across_page_navigation(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, QUESTION)
    _ask_deflected(page, OFF_TOPIC)  # mixed conversation: grounded + deflected
    # A trip to Sources — the New chat control is chat-page-only.
    # (Phase 16: the catalog is admin-only — the trip starts with a
    # real form login.)
    login(page, app_url, next="/sources.html")
    expect(page.locator("#docs-tbody tr").first).to_be_visible(timeout=15_000)
    expect(page.locator("#new-chat-btn")).to_have_count(0)
    # Back to the chat: the conversation is exactly as left — both turns,
    # the source chip, and the amber deflected bubble with its chips.
    page.goto(app_url + "/")
    expect(page.locator("#empty-state")).to_be_hidden()
    expect(page.locator(".msg.user .bubble")).to_have_count(2)
    expect(page.locator(".msg.user .bubble").first).to_contain_text(QUESTION)
    expect(page.locator(".msg.user .bubble").nth(1)).to_contain_text(OFF_TOPIC)
    expect(page.locator(".msg.brain .bubble").first).to_contain_text(MOCK_ANSWER_MARKER)
    expect(page.locator(".msg.brain .source-chip", has_text="kubernetes.md")).to_have_count(1)
    deflected = page.locator(".msg.brain.is-deflected .bubble")
    expect(deflected).to_have_count(1)
    expect(deflected.first).to_contain_text(re.compile(DEFLECT_PHRASE, re.IGNORECASE))
    maybe_chip = page.locator(".msg.brain.is-deflected .maybe-try .suggestion-chip")
    expect(maybe_chip.first).to_be_visible()
    # The New chat control is back on the chat page.
    expect(page.locator("#new-chat-btn")).to_be_visible()
@@ -76,7 +76,7 @@ def test_on_topic_question_streams_grounded_answer(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    summary = _reset_db(mock_llm, seed=True)
-    assert summary is not None and summary.added == 3
+    assert summary is not None and summary.added == 8  # A9 formats
    page.set_default_timeout(30_000)
    page.goto(app_url)
@@ -98,10 +98,16 @@ def test_on_topic_question_streams_grounded_answer(
    # Grounded: a kubernetes.md source chip renders under the bubble
    # (top-N docs can add more chips; the question's doc must be among them).
    # Phase 10: chips open the document viewer in a new tab (encoded URL);
    # phase 13 appends back=/ so the viewer's back button returns to chat.
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip).to_have_count(1)
    expect(chip.first).to_contain_text("kubernetes.md")
-    expect(chip.first).to_have_attribute("href", "/sources.html")
+    expect(chip.first).to_have_attribute(
        "href", "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
    )
    expect(chip.first).to_have_attribute("target", "_blank")
    expect(chip.first).to_have_attribute("rel", "noopener")
    # Button recovers: enabled + "Send" (never stale).
    expect(page.locator("#send-btn")).to_be_enabled()
@@ -42,6 +42,7 @@ from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
@@ -111,7 +112,7 @@ def _seed_kb(mock_port: int) -> ImportSummary:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    summary = _run_in_thread(_import_fixtures(mock_port))
-    assert summary is not None and summary.added == 3
+    assert summary is not None and summary.added == 8  # A9 formats
    return summary
@@ -212,8 +213,9 @@ def test_dark_palette_and_contrast(
    _assert_aa(pairs["button"], "dark ink on brand (send button)")
    _assert_aa(pairs["chip"], "chip ink on chip bg (chat)")
-    # Sources page pairs.
+    # Sources page pairs. (Phase 16: the stat cards are admin-only —
-    page.goto(f"{app_url}/sources.html")
+    # a real form login first.)
    login(page, app_url, next="/sources.html")
    page.locator(".stat-card").first.wait_for(state="visible", timeout=10_000)
    pairs = page.evaluate(
        """() => {
@@ -0,0 +1,207 @@
 """Phase 13 E2E (Playwright): the viewer's back button returns to the
 page the document was opened from.
 Story: ``.agent/user_stories/document-back-navigation.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_document_back_navigation.py -v --no-cov
 Both entry points (chat source chips, Sources table links) open the viewer
 in a NEW tab, where there is no browser history — so the return target is
 carried in the viewer URL: chat chips append ``&back=%2F`` (resolves to
 "Chat"), Sources links omit the param (the viewer's default
 ``/sources.html`` applies → "Sources"). The viewer only honors
 same-origin relative ``back`` values; everything else falls back to
 ``/sources.html``.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_back_from_chat_returns_to_chat`` — question → source chip →
   new tab with ``&back=%2F`` → back link href ``/`` labeled "Chat" →
   click → the chat page.
 2. ``test_back_from_sources_returns_to_sources`` — Sources table link →
   new tab without a ``back`` param → back link href ``/sources.html``
   labeled "Sources" → click → the Sources page.
 3. ``test_malicious_back_param_is_rejected`` — absolute,
   protocol-relative, and ``javascript:`` ``back`` values all fall back
   to ``/sources.html`` (labeled "Sources", navigable).
 """
 from __future__ import annotations
 import asyncio
 import re
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 # Seeded fixture doc (source=docs) shared by every test in this file.
 DOC_SOURCE = "docs"
 DOC_PATH = "homelab%2Fkubernetes.md"
 DOC_TITLE = "Kubernetes Homelab Cluster"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 # ---------------------------------------------------------------------------
 # 1. Chat source chip → viewer with back=/ → back returns to the chat
 # ---------------------------------------------------------------------------
 def test_back_from_chat_returns_to_chat(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    page.fill("#message-input", QUESTION)
    page.click("#send-btn")
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip).to_have_count(1, timeout=30_000)
    # Chat chips carry back=/ (encoded %2F) so the viewer knows where home is.
    expect(chip.first).to_have_attribute(
        "href", f"/document.html?source={DOC_SOURCE}&path={DOC_PATH}&back=%2F"
    )
    with page.expect_popup() as popup_info:
        chip.first.click()
    viewer = popup_info.value
    expect(viewer).to_have_url(
        re.compile(
            re.escape(f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}&back=%2F")
        )
    )
    # The cited document actually rendered (this is the viewer, not an error).
    expect(viewer.locator("#doc-title")).to_have_text(DOC_TITLE)
    # Back link resolved to the chat page, labeled "Chat".
    back = viewer.locator("#doc-back")
    expect(back).to_have_attribute("href", "/")
    expect(back).to_have_text("Chat")
    # Click: deterministic anchor navigation back to the chat page.
    back.click()
    expect(viewer).to_have_url(f"{app_url}/")
    expect(viewer.locator("#composer")).to_be_visible()
 # ---------------------------------------------------------------------------
 # 2. Sources table link → viewer without back param → back returns to Sources
 # ---------------------------------------------------------------------------
 def test_back_from_sources_returns_to_sources(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    login(page, app_url)  # phase 16: the Sources table is admin-only
    row = page.locator("#docs-tbody tr", has_text="kubernetes.md")
    expect(row).to_have_count(1)
    link = row.locator("td:nth-child(2) a.doc-link")
    expect(link).to_have_count(1)
    # Sources links carry NO back param — the viewer's default target
    # (/sources.html) applies.
    expect(link).to_have_attribute(
        "href", f"/document.html?source={DOC_SOURCE}&path={DOC_PATH}"
    )
    with page.expect_popup() as popup_info:
        link.click()
    viewer = popup_info.value
    assert "back=" not in viewer.url, f"unexpected back param: {viewer.url}"
    expect(viewer.locator("#doc-title")).to_have_text(DOC_TITLE)
    # Back link kept the default target, labeled "Sources".
    back = viewer.locator("#doc-back")
    expect(back).to_have_attribute("href", "/sources.html")
    expect(back).to_have_text("Sources")
    back.click()
    expect(viewer).to_have_url(f"{app_url}/sources.html")
    expect(viewer.locator("#docs-table")).to_be_visible()
 # ---------------------------------------------------------------------------
 # 3. Hostile back values (absolute, protocol-relative, pseudo-protocol)
 #    are all rejected in favor of the same-origin default
 # ---------------------------------------------------------------------------
 def test_malicious_back_param_is_rejected(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    errors: list[str] = []
    dialogs: list[str] = []
    page.on("pageerror", lambda e: errors.append(str(e)))
    def _catch_dialog(d) -> None:  # a fired dialog == executed script
        dialogs.append(d.message)
        d.dismiss()
    page.on("dialog", _catch_dialog)
    viewer_base = f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}"
    # Anything that is not a same-origin relative URL must be rejected:
    # an absolute https URL, a protocol-relative URL, and a javascript:
    # pseudo-protocol.
    for evil in ("https%3A%2F%2Fevil.com", "%2F%2Fevil.com", "javascript%3Aalert(1)"):
        page.goto(f"{viewer_base}&back={evil}")
        expect(page.locator("#doc-title")).to_have_text(DOC_TITLE)
        back = page.locator("#doc-back")
        expect(back).to_have_attribute("href", "/sources.html")
        expect(back).to_have_text("Sources")
    # And the fallback is really navigable: clicking lands on Sources.
    page.click("#doc-back")
    expect(page).to_have_url(f"{app_url}/sources.html")
    assert dialogs == [], f"dialog fired — a back param escaped validation: {dialogs}"
    assert errors == [], f"console crashes: {errors}"
@@ -0,0 +1,274 @@
 """Phase 10 E2E (Playwright): the clickable document viewer.
 Story: ``.agent/user_stories/document-viewer.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_document_viewer.py -v --no-cov
 Seeding reuses the real importer against ``tests/fixtures/docs/`` with the
 deterministic mock embeddings (same pattern as the earlier story suites).
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_source_chip_opens_document``       — chip → NEW TAB → viewer with
   title + known content string + format badge.
 2. ``test_sources_row_links_to_viewer``      — Sources path link (yaml
   fixture) → viewer with raw content in a ``pre``.
 3. ``test_markdown_renders_and_stays_xss_safe`` — md fixture containing
   ``<script>alert(1)</script>`` renders as visible escaped text (no
   execution).
 4. ``test_missing_doc_shows_not_found``      — unknown doc → not-found
   state + Sources link; no console crash.
 5. ``test_viewer_theme_and_no_cdn``          — dark theme + every
   ``script[src]`` / ``link[href]`` local or ``data:`` + a11y frame.
 """
 from __future__ import annotations
 import asyncio
 import re
 from datetime import UTC, datetime
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.models import Document
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 # ---------------------------------------------------------------------------
 # 1. Chat source chip → new tab → full document
 # ---------------------------------------------------------------------------
 def test_source_chip_opens_document(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    page.fill("#message-input", QUESTION)
    page.click("#send-btn")
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip).to_have_count(1, timeout=30_000)
    # New-tab contract: same-origin viewer URL, all query values encoded
    # (the path's slashes come out as %2F — exactly why encoding matters),
    # plus back=/ (phase 13) so the viewer's back button returns to chat.
    expect(chip.first).to_have_attribute(
        "href", "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
    )
    expect(chip.first).to_have_attribute("target", "_blank")
    expect(chip.first).to_have_attribute("rel", "noopener")
    with page.expect_popup() as popup_info:
        chip.first.click()
    viewer = popup_info.value
    expect(viewer).to_have_url(
        re.compile(
            re.escape(
                f"{app_url}/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
            )
        )
    )
    expect(viewer.locator("#doc-title")).to_have_text("Kubernetes Homelab Cluster")
    # Meta row: source badge · format badge · mono path · indexed · chunks.
    expect(viewer.locator("#doc-meta .doc-source-badge")).to_have_text("docs")
    expect(viewer.locator("#doc-meta .format-badge")).to_have_text("md")
    expect(viewer.locator("#doc-meta .doc-path")).to_have_text("homelab/kubernetes.md")
    expect(viewer.locator("#doc-meta .doc-indexed")).to_contain_text("Indexed")
    assert re.fullmatch(r"\d+ chunks?", viewer.locator("#doc-meta .doc-chunks").inner_text())
    # Full document, rendered markdown in the centered column (not a pre).
    expect(viewer.locator("#doc-content .doc-md")).to_have_count(1)
    expect(viewer.locator("#doc-content")).to_contain_text("Talos Linux on three nodes")
 # ---------------------------------------------------------------------------
 # 2. Sources table path link → viewer (yaml → raw pre)
 # ---------------------------------------------------------------------------
 def test_sources_row_links_to_viewer(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    login(page, app_url)  # phase 16: the Sources catalog is admin-only
    row = page.locator("#docs-tbody tr", has_text="gitlab-compose.yaml")
    expect(row).to_have_count(1)
    link = row.locator("td:nth-child(2) a.doc-link")
    expect(link).to_have_count(1)
    # Encoded URL: the slashes in the path value come out as %2F.
    expect(link).to_have_attribute(
        "href",
        "/document.html?source=docs&path=homelab%2Fcontainer_gitlab%2Fgitlab-compose.yaml",
    )
    expect(link).to_have_attribute("target", "_blank")
    expect(link).to_have_attribute("rel", "noopener")
    expect(link).to_have_attribute("title", "homelab/container_gitlab/gitlab-compose.yaml")
    with page.expect_popup() as popup_info:
        link.click()
    viewer = popup_info.value
    expect(viewer.locator("#doc-title")).to_have_text("gitlab-compose")
    expect(viewer.locator("#doc-meta .format-badge")).to_have_text("yaml")
    # Non-markdown formats render as escaped monospace text in a pre.
    pre = viewer.locator("#doc-content pre.doc-raw")
    expect(pre).to_have_count(1)
    expect(pre).to_contain_text("gitlab/gitlab-ce:17.2.1-ce.0")
    font = pre.evaluate("el => getComputedStyle(el).fontFamily")
    assert "mono" in font
 # ---------------------------------------------------------------------------
 # 3. Markdown renders through the shared renderer and stays XSS-safe
 # ---------------------------------------------------------------------------
 def test_markdown_renders_and_stays_xss_safe(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    # A document whose content carries a hostile <script> line. The viewer
    # is database-only, so it can be seeded straight into the KB.
    with SessionLocal() as db:
        db.add(
            Document(
                source="docs",
                path="notes/xss-fixture.md",
                full_path="/tmp/xss-fixture.md",
                title="Xss Fixture",
                content="# Xss Fixture\n\n<script>alert(1)</script>\n\nXSS-FIXTURE-MARKER",
                content_hash="c" * 64,
                indexed_at=datetime.now(UTC),
            )
        )
        db.commit()
    dialogs: list[str] = []
    def _catch_dialog(d) -> None:  # a fired dialog == executed script
        dialogs.append(d.message)
        d.dismiss()
    page.on("dialog", _catch_dialog)
    page.goto(f"{app_url}/document.html?source=docs&path=notes%2Fxss-fixture.md")
    expect(page.locator("#doc-title")).to_have_text("Xss Fixture")
    # The tag shows up as VISIBLE, ESCAPED text — rendered, never executed.
    expect(page.locator("#doc-content")).to_contain_text("<script>alert(1)</script>")
    expect(page.locator("#doc-content")).to_contain_text("XSS-FIXTURE-MARKER")
    assert page.locator("#doc-content script").count() == 0, "hostile script became live HTML"
    assert dialogs == [], f"dialog fired — script executed: {dialogs}"
 # ---------------------------------------------------------------------------
 # 4. Missing document → designed not-found state, no console crash
 # ---------------------------------------------------------------------------
 def test_missing_doc_shows_not_found(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    errors: list[str] = []
    page.on("pageerror", lambda e: errors.append(str(e)))
    page.goto(f"{app_url}/document.html?source=docs&path=definitely/not/here.md")
    expect(page.locator("#doc-title")).to_have_text("Document not found")
    card = page.locator("#doc-not-found")
    expect(card).to_be_visible()
    expect(card).to_contain_text("Document not found")
    expect(card.locator("a.doc-open-sources")).to_have_attribute("href", "/sources.html")
    expect(page.locator("#doc-content")).to_be_empty()
    # Missing params → the same designed state (no fetch, no crash).
    page.goto(f"{app_url}/document.html")
    expect(page.locator("#doc-not-found")).to_be_visible()
    assert errors == [], f"console crashes: {errors}"
 # ---------------------------------------------------------------------------
 # 5. Dark theme + all assets local + a11y frame
 # ---------------------------------------------------------------------------
 def test_viewer_theme_and_no_cdn(page: Page, app_url: str, mock_llm: int, db_ready: None) -> None:
    _reset_db(mock_llm, seed=True)
    page.goto(f"{app_url}/document.html?source=docs&path=homelab%2Fkubernetes.md")
    expect(page.locator("#doc-content .doc-md")).not_to_be_empty()
    # Dark theme inherited from phase 08 (same sampling as that story).
    bg = page.evaluate("() => getComputedStyle(document.documentElement).backgroundColor")
    assert bg == "rgb(10, 14, 23)"
    # No-CDN: every script/link reference is same-origin or a data: URI.
    refs = page.evaluate(
        """() => [...document.querySelectorAll("script[src], link[href]")]
            .map((el) => el.src || el.href)"""
    )
    assert refs, "expected local asset references on /document.html"
    for ref in refs:
        assert ref.startswith(app_url) or ref.startswith("data:"), f"non-local: {ref}"
    # A11y frame: landmarks, skip link, aria-live around the load→content
    # swap, and focus moved to main on load.
    expect(page.locator("header.doc-header")).to_have_count(1)
    expect(page.locator("main#main")).to_have_count(1)
    expect(page.locator("footer.app-footer")).to_have_count(1)
    expect(page.locator(".skip-link")).to_have_count(1)
    expect(page.locator(".doc-shell")).to_have_attribute("aria-live", "polite")
    assert page.evaluate("() => document.activeElement && document.activeElement.id") == "main"
    # Markdown column centered and capped at 46rem (736px at 16px root).
    box = page.locator("#doc-content .doc-md").bounding_box()
    assert box is not None and box["width"] <= 736 + 1
@@ -0,0 +1,355 @@
 """Phase 18 E2E (Playwright, mock-only): the chat follows the bottom.
 Story: ``.agent/user_stories/follow-bottom-scroll.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov
 The follow-the-bottom contract (owner choice 2026-08-23, option 1 — no
 "↓ new content" pill): the page auto-scrolls *only while the user is
 already pinned at the bottom*; submitting a question reveals the user's
 own message; once the user scrolls up, nothing auto-scrolls for the rest
 of the turn (thinking or answer); a restored conversation still lands on
 the latest message.
 MOCK-ONLY suite: the scenarios key off the deterministic mock's
 ``write a long answer`` trigger (~900 words ≈ 8s of streaming — a wide,
 reliable window to scroll away in) and, for scenario 4, the phase-17
 ``think out loud`` trigger (both fire independently). ``E2E_REAL_LLM=1``
 would make the scroll-away windows unpredictable, so it is not supported
 here.
 Measurement convention: the scroller is the DOCUMENT — there is no inner
 scroll container (``body`` is ``min-height: 100dvh``; the page scrolls on
 the window). Scroll position is read via ``page.evaluate`` as
 ``{ y: window.scrollY, sh: document.documentElement.scrollHeight,
 ch: window.innerHeight }``; "near bottom" = ``sh - y - ch <= 200``
 (mirrors the frontend's ``NEAR_BOTTOM_PX``); scrolling to the top is
 ``page.evaluate("() => window.scrollTo(0, 0)")``.
 Real-user flow: the user submits from the composer — i.e. pinned at the
 bottom (a normal ``fill`` + ``Enter``/click) — and only *after* the
 stream starts do they scroll up to read earlier messages. The no-yank
 scenarios follow exactly that sequence, so no off-screen input
 manipulation is needed (and Playwright's own click/fill auto-scroll
 never fires, because the composer is already in view).
 Determinism note: the mock paces every SSE frame at 0.02s, so the long
 answer streams for several seconds — "mid-stream" assertions land
 comfortably inside the window on headless Chromium. Every "held still"
 assertion compares against the exact ``scrollTo(0, 0)`` position
 (tolerance 5px for rounding).
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_submit_reveals_new_message``
 2. ``test_stream_follows_while_pinned_at_bottom``
 3. ``test_no_yank_while_scrolled_up_during_answer_stream``
 4. ``test_no_yank_while_scrolled_up_during_thinking``
 5. ``test_restore_lands_on_latest_message``
 """
 from __future__ import annotations
 import asyncio
 import time
 from collections.abc import Iterator
 from pathlib import Path
 from threading import Thread
 from typing import Any
 import pytest
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 #: Mirror of app.js's exported ``NEAR_BOTTOM_PX`` (the 200px composer-zone
 #: band that counts as "pinned to the bottom").
 NEAR_BOTTOM_PX = 200
 #: Mock long-answer trigger (~900 words ≈ 8s of streaming at the mock's
 #: 0.02s/frame pace) — the wide, deterministic window to scroll away in.
 LONG_QUESTION = "write a long answer about my kubernetes cluster"
 #: Phase-17 thinking prefix + the long-answer trigger: both mock triggers
 #: fire independently (a ~1.3s reasoning stream, then the long answer).
 THINK_LONG_QUESTION = "think out loud — write a long answer about my kubernetes cluster"
 #: The mock long answer's unique final line (mock_llm.LONG_ANSWER_END) —
 #: proves the whole stream landed even while the viewport was at the top.
 LONG_ANSWER_END = "LONG-ANSWER-END"
 #: Line fragment the mock's deterministic scratchpad carries
 #: (mock_llm.compose_thinking) — same key phase 17's suite uses.
 THINKING_FRAGMENT = "Step 2: Check my notes"
 #: Tolerance for "the viewport held still at the top" (rounding).
 HOLD_TOLERANCE_PX = 5
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
@pytest.fixture()
 def seeded_kb(mock_llm: int, db_ready: None) -> Iterator[None]:
    """A fresh KB seeded from ``tests/fixtures/docs`` (8 docs, A9 formats),
    truncated again on teardown. ``db_ready`` (conftest) skips with clear
    instructions when Postgres is down."""
    summary = _reset_db(mock_llm, seed=True)
    assert summary is not None and summary.added == 8
    yield
    _reset_db(mock_llm, seed=False)
 # ---------------------------------------------------------------------------
 # Measurement + flow helpers
 # ---------------------------------------------------------------------------
 def scroll_state(page: Page) -> dict[str, float]:
    """The document scroller's state (there is no inner scroll container)."""
    return page.evaluate(
        "() => ({ y: window.scrollY, "
        "sh: document.documentElement.scrollHeight, "
        "ch: window.innerHeight })"
    )
 def near_bottom(state: dict[str, float]) -> bool:
    """Mirror of app.js's ``isNearBottom`` — the NEAR_BOTTOM_PX band."""
    return state["sh"] - state["y"] - state["ch"] <= NEAR_BOTTOM_PX
 def held_at_top(page: Page) -> bool:
    """The viewport has not moved from ``window.scrollTo(0, 0)`` (±5px)."""
    return scroll_state(page)["y"] <= HOLD_TOLERANCE_PX
 def wait_settled(page: Page) -> None:
    """The turn is over: the never-stale contract re-enabled the button."""
    expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
    expect(page.locator("#send-label")).to_have_text("Send")
 def submit(page: Page, question: str) -> None:
    """Submit from the composer — the real-user flow (pinned at the
    bottom, so Playwright's click/fill auto-scroll never kicks in)."""
    page.fill("#message-input", question)
    page.click("#send-btn")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
 def brain_bubble_longer_than(n: int) -> str:
    """JS predicate: the LAST brain bubble's rendered text is > n chars
    (i.e. that far into the stream)."""
    return (
        "() => { const els = document.querySelectorAll('.msg.brain .bubble');"
        f" const el = els[els.length - 1]; return !!el && el.innerText.length > {n}; }}"
    )
 def brain_message_in_view(page: Page) -> bool:
    """The last brain message intersects the viewport vertically. Partial
    visibility counts: a long answer is taller than the window, and the
    contract is that it is revealed (its lower edge in view), not that it
    fits."""
    box = page.locator(".msg.brain").last.bounding_box()
    if box is None:
        return False
    ch = scroll_state(page)["ch"]
    return box["y"] < ch and box["y"] + box["height"] > 0
 # ---------------------------------------------------------------------------
 # 1. Submit: the user's message and the answer reveal into view
 # ---------------------------------------------------------------------------
 def test_submit_reveals_new_message(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    # A fresh chat page starts pinned at the bottom (short conversation —
    # the composer, i.e. the user, sits in the band).
    assert near_bottom(scroll_state(page))
    submit(page, LONG_QUESTION)
    wait_settled(page)
    # The last brain message is inside the viewport ...
    assert brain_message_in_view(page), (
        "the answer must be revealed — the last brain message is not in view"
    )
    # ... and the page is still pinned at the bottom.
    assert near_bottom(scroll_state(page))
 # ---------------------------------------------------------------------------
 # 2. Follow: while pinned, the page keeps up with the stream
 # ---------------------------------------------------------------------------
 def test_stream_follows_while_pinned_at_bottom(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    submit(page, LONG_QUESTION)
    # ~2s into the stream: the answer bubble already carries >200 chars
    # (the mock paces frames at 0.02s).
    page.wait_for_function(brain_bubble_longer_than(200), timeout=30_000)
    # Let the smooth follow scroll settle before measuring.
    time.sleep(0.3)
    # The follow behavior is alive — not accidentally removed.
    assert near_bottom(scroll_state(page)), (
        "the page must follow the stream while the user is pinned at the bottom"
    )
    wait_settled(page)
    assert near_bottom(scroll_state(page))
 # ---------------------------------------------------------------------------
 # 3. No yank: scrolled up mid-ANSWER — the viewport holds for the rest
 #    of the turn (the answer finishes off-screen below, by design)
 # ---------------------------------------------------------------------------
 def test_no_yank_while_scrolled_up_during_answer_stream(
    page: Page, app_url: str, seeded_kb: None
 ) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    # Turn 1 (settled) makes the document overflow the 800px viewport.
    submit(page, LONG_QUESTION)
    wait_settled(page)
    state = scroll_state(page)
    assert state["sh"] > state["ch"], "a long answer must make the document scrollable"
    assert near_bottom(state), "follow was active: the settled turn ends pinned"
    # Turn 2: submit from the composer (pinned — normal flow), then let
    # the new answer stream a bit.
    submit(page, LONG_QUESTION)
    page.wait_for_function(brain_bubble_longer_than(200), timeout=30_000)
    # The user goes up to read while the stream is running.
    page.evaluate("() => window.scrollTo(0, 0)")
    # The stream kept running at the top ...
    page.wait_for_function(brain_bubble_longer_than(600), timeout=30_000)
    assert held_at_top(page), "the viewport must hold still while scrolled up"
    # ... and nothing scrolls for the rest of the turn — the answer
    # finishes off-screen below, by design.
    wait_settled(page)
    assert held_at_top(page)
 # ---------------------------------------------------------------------------
 # 4. No yank: scrolled up during THINKING — the whole reasoning stream
 #    plus the answer's start happen at the top
 # ---------------------------------------------------------------------------
 def test_no_yank_while_scrolled_up_during_thinking(
    page: Page, app_url: str, seeded_kb: None
 ) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    # One settled turn first, so the document overflows (scrollable).
    submit(page, LONG_QUESTION)
    wait_settled(page)
    # The thinking turn: submit pinned (normal flow) ...
    submit(page, THINK_LONG_QUESTION)
    details = page.locator(".msg.brain").last.locator("details.thinking")
    details.wait_for(state="attached", timeout=10_000)
    # ... and, while the reasoning stream is still open (phase-17
    # behavior: created open, ~1.3s before the first answer token) ...
    expect(details).to_have_attribute("open", "")
    # ... the user goes up to read.
    page.evaluate("() => window.scrollTo(0, 0)")
    # The whole thinking stream plus the answer's start happen at the top.
    bubble = page.locator(".msg.brain").last.locator(".bubble")
    expect(bubble).not_to_have_text("", timeout=30_000)
    assert held_at_top(page), "the viewport must hold still during thinking"
    # Settled: still at the top, and everything landed (off-screen,
    # which is the point of the story).
    wait_settled(page)
    assert held_at_top(page)
    expect(details.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
    expect(bubble).to_contain_text(LONG_ANSWER_END)
 # ---------------------------------------------------------------------------
 # 5. Restore: the one-shot landing still puts the latest message in view
 #    (phase 14 behavior preserved — pinned so a future "remove all
 #    scrolling" change fails loudly instead of silently)
 # ---------------------------------------------------------------------------
 def test_restore_lands_on_latest_message(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    # Two settled turns (user + brain × 2) — the document overflows.
    submit(page, LONG_QUESTION)
    wait_settled(page)
    submit(page, LONG_QUESTION)
    wait_settled(page)
    page.reload()
    # Restore re-renders from localStorage; wait until the last restored
    # brain answer is fully back.
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(
        LONG_ANSWER_END, timeout=30_000
    )
    wait_settled(page)
    expect(page.locator(".msg.user .bubble")).to_have_count(2)
    state = scroll_state(page)
    assert state["sh"] > state["ch"]
    # The forced one-shot landing (the only `force`d scrolls) puts the
    # last brain message back in view ...
    assert brain_message_in_view(page), (
        "a restored conversation must land on its latest message"
    )
    # ... and the page sits at the bottom.
    assert near_bottom(state)
@@ -0,0 +1,185 @@
 """Phase 12 E2E (Playwright): one header, same size on every page.
 Story: ``.agent/user_stories/header-consistency.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_header_consistency.py -v --no-cov
 The sticky top bar must be exactly ``--header-h`` tall on Chat, Sources,
 and the document viewer (whose back button says "Sources" — the page
 users compare against chat): 64px desktop, 58px at ≤640px.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_header_height_identical_across_pages_desktop``
 2. ``test_header_height_identical_across_pages_mobile``
 3. ``test_viewer_header_content_still_fits`` (phase-10 regression guard)
 Phase 16 adaptation: the auth control (Sign in / Sign out) joins the chat
 header's ``.header-inner`` — the desktop test verifies its presence in
 both auth states without the bar's height moving (height assertions
 unchanged).
 """
 from __future__ import annotations
 import asyncio
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 VIEWER_URL = "/document.html?source=docs&path=homelab%2Fkubernetes.md"
 SOURCES_URL = "/sources.html"
 #: The shared header-bar token values (frontend/assets/styles.css :root
 #: and the ≤640px media query).
 DESKTOP_HEADER_H = 64
 MOBILE_HEADER_H = 58
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread (Playwright owns the test loop)."""
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _seed_db(mock_port: int) -> None:
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    _run_in_thread(_import_fixtures(mock_port))
 def _box_height(page: Page, selector: str) -> float:
    box = page.locator(selector).bounding_box()
    assert box is not None, f"{selector} not rendered"
    return box["height"]
 def _header_heights(page: Page, app_url: str) -> dict[str, float]:
    """Measured heights of the sticky top bar on the three pages."""
    heights: dict[str, float] = {}
    page.goto(app_url + "/")
    heights["chat"] = _box_height(page, ".app-header")
    page.goto(app_url + SOURCES_URL)
    heights["sources"] = _box_height(page, ".app-header")
    page.goto(app_url + VIEWER_URL)
    expect(page.locator("#doc-title")).to_have_text("Kubernetes Homelab Cluster", timeout=15_000)
    heights["document"] = _box_height(page, ".doc-header")
    return heights
 # ---------------------------------------------------------------------------
 # 1. Desktop: all three pages, one identical 64px bar
 # ---------------------------------------------------------------------------
 def test_header_height_identical_across_pages_desktop(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    page.set_viewport_size({"width": 1280, "height": 800})
    _seed_db(mock_llm)
    heights = _header_heights(page, app_url)
    assert heights["chat"] == DESKTOP_HEADER_H, f"chat header {heights['chat']}px"
    assert heights["sources"] == DESKTOP_HEADER_H, f"sources header {heights['sources']}px"
    assert heights["document"] == DESKTOP_HEADER_H, (
        f"document header {heights['document']}px (was content-sized)"
    )
    # Phase 16: the auth control lives in the same bar — anonymous sees
    # "Sign in", signed-in sees "Sign out", and neither state moves the
    # height.
    page.goto(app_url + "/")
    expect(page.locator("#sign-in-link")).to_be_visible()
    expect(page.locator("#sign-out-btn")).to_be_hidden()
    assert _box_height(page, ".app-header") == DESKTOP_HEADER_H
    login(page, app_url, next="/")
    expect(page.locator("#sign-out-btn")).to_be_visible()
    expect(page.locator("#sign-in-link")).to_be_hidden()
    assert _box_height(page, ".app-header") == DESKTOP_HEADER_H
 # ---------------------------------------------------------------------------
 # 2. Mobile (≤640px): all three pages, one identical 58px bar
 # ---------------------------------------------------------------------------
 def test_header_height_identical_across_pages_mobile(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    page.set_viewport_size({"width": 375, "height": 812})
    _seed_db(mock_llm)
    heights = _header_heights(page, app_url)
    assert heights["chat"] == MOBILE_HEADER_H, f"chat header {heights['chat']}px"
    assert heights["sources"] == MOBILE_HEADER_H, f"sources header {heights['sources']}px"
    assert heights["document"] == MOBILE_HEADER_H, (
        f"document header {heights['document']}px (meta row must clip, not wrap)"
    )
 # ---------------------------------------------------------------------------
 # 3. Regression: the phase-10 viewer header still shows title, badges,
 #    back link — single-line on both desktop and mobile
 # ---------------------------------------------------------------------------
 def test_viewer_header_content_still_fits(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _seed_db(mock_llm)
    for width, expected_h in ((1280, DESKTOP_HEADER_H), (375, MOBILE_HEADER_H)):
        page.set_viewport_size({"width": width, "height": 800})
        page.goto(app_url + VIEWER_URL)
        expect(
            page.locator("#doc-title")
        ).to_have_text("Kubernetes Homelab Cluster", timeout=15_000)
        # Title on a single line (ellipsis, no wrap growth).
        assert _box_height(page, "#doc-title") < 30  # one line at either size
        # Meta row: badges + path present and visible, single line.
        assert _box_height(page, ".doc-meta") < 26
        expect(page.locator(".doc-source-badge", has_text="docs")).to_be_visible()
        expect(page.locator(".format-badge", has_text="md")).to_be_visible()
        expect(page.locator(".doc-path", has_text="homelab/kubernetes.md")).to_be_visible()
        # Back link still there, ≥44px touch target, in the shared bar.
        back = page.locator("#doc-back")
        expect(back).to_be_visible()
        assert _box_height(page, "#doc-back") >= 44
        expect(page.locator(".doc-header")).to_have_css(
            "height", f"{expected_h}px"
        )
@@ -84,7 +84,7 @@ def test_off_topic_question_deflects_honestly(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    summary = _reset_db(mock_llm, seed=True)
-    assert summary is not None and summary.added == 3
+    assert summary is not None and summary.added == 8  # A9 formats
    page.set_default_timeout(30_000)
    page.goto(app_url)
    expect(page.locator("#kb-banner")).to_be_hidden()
@@ -8,6 +8,9 @@ Run in isolation (DB must be up: ``podman compose up -d db``):
 Seeding runs the real import function in-process against
 ``tests/fixtures/docs/`` with the deterministic mock embeddings — it is a
 fixture, not the subject of the tests.
 Phase 16 adaptation: the Sources catalog is admin-only — every test
 performs the real form login (``e2e.auth_helpers.login``) first.
 """
 from __future__ import annotations
@@ -23,6 +26,7 @@ from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
@@ -31,6 +35,11 @@ EXPECTED_ROWS = (
    "homelab/kubernetes.md",
    "homelab/backups.md",
    "deployments/new-service.md",
    "homelab/container_gitlab/gitlab.md",
    "homelab/container_gitlab/gitlab-compose.yaml",
    "homelab/networking/static-dns.json",
    "homelab/scripts/uptime_probe.py",
    "homelab/ssh/ssh_aliases.txt",
 )
@@ -76,16 +85,21 @@ def test_sources_page_lists_indexed_docs(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    summary = _reset_db(mock_llm, seed=True)
-    assert summary is not None and summary.added == 3
+    # Eight A9-format files are imported; .hidden/junk.md is out of scope
    # (A9 revised — hidden path components are never walked).
    assert summary is not None and summary.added == 8
    assert summary.formats == {"md": 4, "yaml": 1, "json": 1, "py": 1, "txt": 1}
-    page.goto(f"{app_url}/sources.html")
+    login(page, app_url)  # phase 16: the catalog is admin-only
-    expect(page.locator("#stat-docs")).to_have_text("3")
+    expect(page.locator("#stat-docs")).to_have_text("8")
    expect(page.locator("#stat-chunks")).to_have_text(str(summary.chunks))
    expect(page.locator("#stat-last")).not_to_have_text("–")
    expect(page.locator("#sources-empty")).to_be_hidden()
    for row_path in EXPECTED_ROWS:
        expect(page.locator("#docs-tbody tr", has_text=row_path)).to_have_count(1)
    # The hidden junk was never indexed (A9 scope).
    expect(page.locator("#docs-tbody tr", has_text=".hidden")).to_have_count(0)
    # The path column carries the full path for hover (ellipsis is visual only).
    expect(page.locator("#docs-tbody tr", has_text="homelab/kubernetes.md")
           .get_by_role("cell").nth(1)).to_have_attribute("title", "homelab/kubernetes.md")
@@ -95,7 +109,7 @@ def test_sources_table_layout(
    page: Page, browser: Browser, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
-    page.goto(f"{app_url}/sources.html")
+    login(page, app_url)  # phase 16: the catalog is admin-only
    page.locator("#docs-tbody tr").first.wait_for(state="visible")
    wrap = page.locator(".table-wrap")
@@ -114,7 +128,7 @@ def test_sources_table_layout(
    # scrolls horizontally instead of squeezing into a hairline.
    mobile = browser.new_page(viewport={"width": 375, "height": 812})
    try:
-        mobile.goto(f"{app_url}/sources.html")
+        login(mobile, app_url)  # phase 16: the catalog is admin-only
        mobile.locator("#docs-tbody tr").first.wait_for(state="visible")
        scroll_width, client_width = mobile.evaluate(
            "() => { const el = document.querySelector('.table-wrap');"
@@ -128,7 +142,7 @@ def test_sources_table_layout(
 def test_empty_state_when_no_docs(page: Page, app_url: str, mock_llm: int, db_ready: None) -> None:
    _reset_db(mock_llm, seed=False)
-    page.goto(f"{app_url}/sources.html")
+    login(page, app_url)  # phase 16: the (empty-state) catalog is admin-only
    expect(page.locator("#sources-empty")).to_be_visible()
    expect(page.locator("#sources-empty")).to_contain_text("Nothing indexed yet")
    expect(page.locator("#sources-empty code")).to_have_text(
@@ -176,7 +176,7 @@ def test_typing_indicator_during_slow_think(
    """AC1/AC5: the 3s mock warm-up must show the typing indicator for
    >=2s before any text appears, then it is gone once the answer lands."""
    summary = _reset_db(mock_llm, seed=True)
-    assert summary is not None and summary.added == 3
+    assert summary is not None and summary.added == 8  # A9 formats
    page.set_default_timeout(30_000)
    page.goto(app_url)
@@ -0,0 +1,133 @@
 """Phase 11 E2E (Playwright): long answers stream to completion.
 Story: ``.agent/user_stories/long-answers.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_long_answers.py -v --no-cov
 The mock LLM honors ``max_tokens`` (token ≈ word) like a real endpoint,
 and emits a ~900-word deterministic answer for the "write a long answer"
 trigger. Under the old hard 700-token cap the answer loses its tail
 (the final line never arrives); with ``BOR_MAX_OUTPUT_TOKENS`` defaulting
 to 32 768 the full answer streams to completion.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_long_answer_streams_to_completion`` — trigger question →
   full ~900-word answer, final line intact, >700 words rendered.
 2. ``test_normal_answer_unaffected`` — a regular question still streams
   a complete short answer.
 """
 from __future__ import annotations
 import asyncio
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 LONG_QUESTION = "How is my Kubernetes cluster set up? write a long answer"
 NORMAL_QUESTION = "How is my Kubernetes cluster set up?"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread (Playwright owns the test loop)."""
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int) -> ImportSummary:
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    return _run_in_thread(_import_fixtures(mock_port))
 def _last_brain_text(page: Page) -> str:
    return page.locator(".msg.brain .bubble").last.inner_text()
 # ---------------------------------------------------------------------------
 # 1. Long answer: the final line must survive the stream
 # ---------------------------------------------------------------------------
 def test_long_answer_streams_to_completion(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm)
    page.set_default_timeout(45_000)
    page.goto(app_url)
    page.fill("#message-input", LONG_QUESTION)
    page.click("#send-btn")
    # The mock streams ~900 words in ~8s; wait for the unique final line —
    # under the old 700-token cap it was cut off and never arrived.
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(
        "LONG-ANSWER-END", timeout=60_000
    )
    full = _last_brain_text(page)
    # The old cap would have stopped the answer at 700 words — prove the
    # rendered answer ran well past it.
    assert len(full.split()) > 700, (
        f"answer looks truncated at {len(full.split())} words"
    )
    # First and last step both rendered (the markdown list strips the
    # "1." prefix — no mid-sentence cut between them either).
    assert "Step 1:" in full
    assert "Step 40:" in full
    # Turn settled: send button re-enabled (never-stale contract).
    expect(page.locator("#send-btn")).to_be_enabled()
 # ---------------------------------------------------------------------------
 # 2. Normal (short) answers are unaffected by the raised cap
 # ---------------------------------------------------------------------------
 def test_normal_answer_unaffected(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    page.fill("#message-input", NORMAL_QUESTION)
    page.click("#send-btn")
    bubble = page.locator(".msg.brain .bubble").last
    expect(bubble).to_contain_text("Deterministic mock answer for E2E", timeout=30_000)
    expect(bubble).not_to_contain_text("LONG-ANSWER-END")
    # Grounded: the question's own document is cited as a chip.
    expect(
        page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    ).to_have_count(1, timeout=30_000)
    expect(page.locator("#send-btn")).to_be_enabled()
@@ -45,6 +45,7 @@ from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
@@ -206,7 +207,7 @@ def test_no_horizontal_overflow_at_viewports(
            )
            _assert_no_doc_overflow(page, f"chat @ {width}px")
-            page.goto(f"{app_url}/sources.html")
+            login(page, app_url, next="/sources.html")  # phase 16: admin-only
            page.locator("#docs-tbody tr").first.wait_for(state="visible", timeout=10_000)
            _assert_no_doc_overflow(page, f"sources @ {width}px")
        finally:
@@ -254,7 +255,7 @@ def test_sources_table_full_width(
    _reset_db(mock_llm, seed=True)
    page = browser.new_page(viewport={"width": 1280, "height": 800})
    try:
-        page.goto(f"{app_url}/sources.html")
+        login(page, app_url, next="/sources.html")  # phase 16: admin-only
        page.locator("#docs-tbody tr").first.wait_for(state="visible", timeout=10_000)
        wrap_box = page.locator(".table-wrap").bounding_box()
        shell_box = page.locator(".sources-shell").bounding_box()
@@ -268,7 +269,7 @@ def test_sources_table_full_width(
    mobile = browser.new_page(viewport={"width": 375, "height": 812})
    try:
-        mobile.goto(f"{app_url}/sources.html")
+        login(mobile, app_url, next="/sources.html")  # phase 16: admin-only
        mobile.locator("#docs-tbody tr").first.wait_for(state="visible", timeout=10_000)
        scroll, client = mobile.evaluate(
            "() => { const el = document.querySelector('.table-wrap');"
@@ -389,7 +390,8 @@ def test_contrast_pairs_pass_aa(
    _assert_aa(pairs["deflection"], "deflection ink on deflection bg")
    # Sources page: ink-soft/surface, white/brand (active nav).
-    page.goto(f"{app_url}/sources.html")
+    # (Phase 16: the stat cards are admin-only — sign in first.)
    login(page, app_url, next="/sources.html")
    page.locator(".stat-card").first.wait_for(state="visible", timeout=10_000)
    pairs = page.evaluate(
        """() => {
@@ -498,7 +500,7 @@ def test_long_content_wraps_without_overflow(
    # Sources @ 360px: the long path ellipsizes, full path stays in `title`.
    phone = browser.new_page(viewport={"width": 360, "height": 740})
    try:
-        phone.goto(f"{app_url}/sources.html")
+        login(phone, app_url, next="/sources.html")  # phase 16: admin-only
        row = phone.locator("#docs-tbody tr", has_text="backup_rotation").first
        row.wait_for(state="visible", timeout=10_000)
        cell = row.get_by_role("cell").nth(1)
@@ -0,0 +1,205 @@
 """Phase 09 E2E (Playwright): retrieval quality — hybrid search end to end.
 Story: ``.agent/user_stories/retrieval-quality.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_retrieval_quality.py -v --no-cov
 Seeding reuses the real importer against ``tests/fixtures/docs/`` with the
 deterministic mock embeddings (same pattern as the earlier story suites).
 The four tests map the story's acceptance criteria:
 1. multi-format fixture import — hidden doc excluded, ``/api/docs`` counts
 2. "How did I install gitlab?" — grounded (not deflected), gitlab chip,
   ``query_log`` row with the gitlab doc in ``sources``
 3. keyword-only question ("kafkabridge") beats the vector ranking — the
   FTS-OR gate grounds it end to end despite weak cosine
 4. "sourdough" — deflected bubble + ≥2 "Maybe try" chips
 """
 from __future__ import annotations
 import asyncio
 from pathlib import Path
 from threading import Thread
 from typing import Any
 import httpx
 from playwright.sync_api import Page, expect
 from sqlalchemy import select, text
 from app.config import Settings, get_settings
 from app.db import SessionLocal
 from app.models import QueryLog
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 GITLAB_QUESTION = "How did I install gitlab?"
 KEYWORD_QUESTION = "How does kafkabridge work?"
 OFF_TOPIC = "sourdough starter"
 MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread (Playwright owns the test loop)."""
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 def _ask(page: Page, message: str) -> None:
    page.fill("#message-input", message)
    page.click("#send-btn")
 def test_multi_format_import_hidden_doc_excluded(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    """A9 (revised): all seven formats import; hidden (dot) paths never do."""
    summary = _reset_db(mock_llm, seed=True)
    assert summary is not None
    # Eight A9-format fixture files; .hidden/junk.md must never be walked.
    assert summary.added == 8
    assert summary.formats == {"md": 4, "yaml": 1, "json": 1, "py": 1, "txt": 1}
    # Phase 16: the catalog is admin-only — perform the real form login,
    # then call the API with the signed cookie the browser now holds.
    login(page, app_url, next="/sources.html")
    cookies = {
        c["name"]: c["value"]
        for c in page.context.cookies()
        if "name" in c and "value" in c
    }
    r = httpx.get(f"{app_url}/api/docs", timeout=10, cookies=cookies)
    assert r.status_code == 200
    docs = r.json()["documents"]
    assert len(docs) == 8
    assert all(".hidden" not in d["path"] for d in docs)
    assert {d["path"] for d in docs} >= {
        "homelab/container_gitlab/gitlab.md",
        "homelab/container_gitlab/gitlab-compose.yaml",
        "homelab/networking/static-dns.json",
        "homelab/scripts/uptime_probe.py",
        "homelab/ssh/ssh_aliases.txt",
    }
    # The Sources page (we're already on it, signed in) reflects the set.
    expect(page.locator("#stat-docs")).to_have_text("8")
    expect(page.locator("#docs-tbody tr", has_text=".hidden")).to_have_count(0)
 def test_gitlab_question_is_grounded_with_gitlab_chip(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    """The ranking problem that motivated this phase: a tool-name question
    must land on the tool's own document — not a generic template."""
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, GITLAB_QUESTION)
    expect(page.locator(".msg.user .bubble")).to_contain_text(GITLAB_QUESTION)
    bubble = page.locator(".msg.brain .bubble").first
    bubble.wait_for(state="visible", timeout=30_000)
    expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
    # Grounded: no deflected bubble at all.
    expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
    # The gitlab document is cited (a chip carrying its path).
    chip = page.locator(".msg.brain .source-chip", has_text="container_gitlab/gitlab.md")
    expect(chip).to_have_count(1, timeout=30_000)
    # Durable record: not deflected, and the gitlab doc is in sources.
    with SessionLocal() as db:
        row = db.scalars(select(QueryLog)).one()
    assert row.question == GITLAB_QUESTION
    assert row.deflected is False
    assert "container_gitlab/gitlab.md" in row.sources
 def test_keyword_only_question_beats_vector_ranking(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    """The FTS-OR gate end to end: "kafkabridge" appears in exactly one
    fixture doc (static-dns.json) and the question's cosine overlap is
    weak — the lexical branch is what grounds the answer."""
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, KEYWORD_QUESTION)
    bubble = page.locator(".msg.brain .bubble").first
    bubble.wait_for(state="visible", timeout=30_000)
    expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
    expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
    # The FTS-matched doc is the TOP source chip (it beats the vector rank).
    first_chip = page.locator(".msg.brain .source-chip").first
    first_chip.wait_for(state="visible", timeout=30_000)
    expect(first_chip).to_contain_text("static-dns.json")
    with SessionLocal() as db:
        row = db.scalars(select(QueryLog)).one()
    # Weak vector score…
    assert row.top_score < get_settings().relevance_threshold
    # …but a lexical hit grounded it (the FTS-OR branch).
    assert (row.fts_hits or 0) >= 1
    assert row.deflected is False
    assert "homelab/networking/static-dns.json" in row.sources
 def test_off_topic_still_deflects_with_chips(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    """A8 (revised): deflection requires weak cosine AND zero FTS hits.
    "sourdough" matches nothing in the KB lexically → honest deflection."""
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    _ask(page, OFF_TOPIC)
    bubble = page.locator(".msg.brain.is-deflected .bubble").first
    bubble.wait_for(state="visible", timeout=30_000)
    expect(page.locator(".msg.brain.is-deflected")).to_have_count(1)
    chips = page.locator(".msg.brain.is-deflected .maybe-try .suggestion-chip")
    expect(chips.first).to_be_visible(timeout=30_000)
    assert chips.count() >= 2, "deflection must offer 2-3 alternative chips"
    assert all(c.strip() for c in chips.all_inner_texts())
    with SessionLocal() as db:
        row = db.scalars(select(QueryLog)).one()
    assert row.question == OFF_TOPIC
    assert row.deflected is True
    assert 0.0 < row.top_score < get_settings().relevance_threshold
    assert row.fts_hits == 0  # deflection is only reached with zero hits
@@ -0,0 +1,301 @@
 """Phase 15 E2E (Playwright): tune how Brain answers (steering notes).
 Phase 16 adaptation: tuning is admin-only — every test performs the real
 form login (``e2e.auth_helpers.login``) before touching the tuning UI.
 Story: ``.agent/user_stories/steering-notes.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_steering.py -v --no-cov
 The steering loop: "Tune" under a completed answer → short instruction →
 stored in Postgres (``steering_notes``) → injected into the system prompt
 of every subsequent turn as the ``<tuning>`` section. The mock LLM
 echoes the first tuning note into its answer
 (`` (tuning: <first note line>)``), so prompt injection is observable in
 the UI deterministically. Notes are listed newest-first in the header
 "Tuning" panel, where each can be deleted.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_tune_under_answer_persists_and_steers``
 2. ``test_delete_note_stops_steering``
 3. ``test_note_rendered_as_text_xss_safe``
 4. ``test_tuning_panel_a11y``
 """
 from __future__ import annotations
 import asyncio
 from pathlib import Path
 from threading import Thread
 from typing import Any
 from playwright.sync_api import Page, expect
 from sqlalchemy import select, text
 from app.config import Settings
 from app.db import SessionLocal
 from app.models import SteeringNote
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 from e2e.auth_helpers import login
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
 NOTE = "STEEER-MARKER be concise"
 XSS_NOTE = "<script>window.__xss = true; alert('xss')</script>"
 #: index.html ships exactly two classic/module script tags.
 BASE_SCRIPT_COUNT = 2
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log + steering notes), re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
 def _ask(page: Page, question: str) -> None:
    """Send one turn and wait until the grounded answer has fully landed."""
    page.fill("#message-input", question)
    page.click("#send-btn")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(
        MOCK_ANSWER_MARKER, timeout=30_000
    )
    expect(page.locator("#send-btn")).to_be_enabled()
    expect(page.locator("#send-label")).to_have_text("Send")
 def _tune_and_save(page: Page, note: str) -> None:
    """Tune the last completed brain bubble and save *note*."""
    tune = page.locator(".msg.brain .tune-btn").last
    expect(tune).to_be_visible()
    tune.click()
    form = page.locator(".msg.brain .tune-form").last
    expect(form).to_be_visible()
    form.locator("textarea").fill(note)
    form.locator(".tune-save").click()
    saved = page.locator(".msg.brain .tune-saved").last
    expect(saved).to_contain_text("Saved — future answers will follow this.", timeout=15_000)
 def _open_panel(page: Page) -> None:
    page.click("#steering-toggle")
    expect(page.locator("#steering-panel")).to_be_visible()
 # ---------------------------------------------------------------------------
 # 1. Tune under an answer → persisted → next answer carries the note
 # ---------------------------------------------------------------------------
 def test_tune_under_answer_persists_and_steers(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    summary = _reset_db(mock_llm, seed=True)
    assert summary is not None and summary.added == 8  # A9 formats
    page.set_default_timeout(30_000)
    page.goto(app_url)
    login(page, app_url, next="/")  # phase 16: tuning is admin-only
    _ask(page, QUESTION)
    # The Tune control: ghost button in the answer's meta row, ≥44px.
    tune = page.locator(".msg.brain .tune-btn").last
    expect(tune).to_have_count(1)
    expect(tune).to_have_attribute("type", "button")
    box = tune.bounding_box()
    assert box is not None and box["height"] >= 44
    _tune_and_save(page, NOTE)
    # Persisted in Postgres.
    with SessionLocal() as db:
        rows = db.scalars(select(SteeringNote)).all()
        assert [r.note for r in rows] == [NOTE]
    # The header panel shows the note with an updated count badge.
    _open_panel(page)
    expect(page.locator("#steering-count")).to_have_text("1")
    expect(page.locator("#steering-list .steering-note")).to_have_count(1)
    expect(page.locator("#steering-list .steering-note-text")).to_have_text(NOTE)
    page.click("#steering-toggle")  # close again
    # The NEXT answer carries the note — it reached the system prompt.
    _ask(page, QUESTION)
    bubble = page.locator(".msg.brain .bubble").last
    expect(bubble).to_contain_text(f"(tuning: {NOTE})")
 # ---------------------------------------------------------------------------
 # 2. Delete from the panel → count 0 → steering stops
 # ---------------------------------------------------------------------------
 def test_delete_note_stops_steering(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    login(page, app_url, next="/")  # phase 16: tuning is admin-only
    _ask(page, QUESTION)
    _tune_and_save(page, NOTE)
    # Steering is live: one more answer carries the marker.
    _ask(page, QUESTION)
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(f"(tuning: {NOTE})")
    # Delete the note from the panel.
    _open_panel(page)
    expect(page.locator("#steering-count")).to_have_text("1")
    page.locator("#steering-list .steering-delete").click()
    expect(page.locator("#steering-list .steering-note")).to_have_count(0)
    expect(page.locator("#steering-count")).to_have_text("0")
    expect(page.locator("#steering-empty")).to_be_visible()
    expect(page.locator("#steering-announcer")).to_contain_text("deleted")
    with SessionLocal() as db:
        assert db.scalars(select(SteeringNote)).all() == []
    # The next answer no longer carries the marker.
    _ask(page, QUESTION)
    bubble = page.locator(".msg.brain .bubble").last
    expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
    expect(bubble).not_to_contain_text("STEEER-MARKER")
 # ---------------------------------------------------------------------------
 # 3. Notes render as text (XSS-safe)
 # ---------------------------------------------------------------------------
 def test_note_rendered_as_text_xss_safe(
    page: Page, app_url: str, mock_llm: int, db_ready: None
 ) -> None:
    _reset_db(mock_llm, seed=True)
    page.set_default_timeout(30_000)
    page.goto(app_url)
    login(page, app_url, next="/")  # phase 16: tuning is admin-only
    dialogs: list[str] = []
    def _handle_dialog(d) -> None:
        dialogs.append(d.message)
        d.dismiss()
    page.on("dialog", _handle_dialog)
    _ask(page, QUESTION)
    _tune_and_save(page, XSS_NOTE)
    # Panel: the payload is visible as LITERAL text…
    _open_panel(page)
    expect(page.locator("#steering-list .steering-note-text")).to_have_text(XSS_NOTE)
    # …never as an executed element: no script tag anywhere, no dialog.
    assert page.locator("#steering-panel script").count() == 0
    expect(page.locator("script")).to_have_count(BASE_SCRIPT_COUNT)
    assert dialogs == [], f"the note must never execute as script: {dialogs}"
    assert page.evaluate("() => window.__xss === undefined") is True
 # ---------------------------------------------------------------------------
 # 4. Tuning panel accessibility
 # ---------------------------------------------------------------------------
 def test_tuning_panel_a11y(page: Page, app_url: str, db_ready: None) -> None:
    _reset_db(mock_port=0, seed=False)  # no KB seeding needed for the panel a11y
    page.set_default_timeout(30_000)
    page.goto(app_url)
    login(page, app_url, next="/")  # phase 16: the panel is admin-only
    toggle = page.locator("#steering-toggle")
    panel = page.locator("#steering-panel")
    announcer = page.locator("#steering-announcer")
    # Initial: closed, correctly wired, polite live region present.
    expect(toggle).to_have_attribute("aria-expanded", "false")
    expect(toggle).to_have_attribute("aria-controls", "steering-panel")
    expect(panel).to_have_attribute("role", "region")
    assert "Tuning notes" in (panel.get_attribute("aria-label") or "")
    expect(panel).to_be_hidden()
    assert announcer.get_attribute("role") == "status"
    assert announcer.get_attribute("aria-live") == "polite"
    # Accessible name comes from its visible text (icon is aria-hidden).
    assert "Tuning" in toggle.inner_text()
    # Open: expanded + the designed empty state.
    toggle.click()
    expect(toggle).to_have_attribute("aria-expanded", "true")
    expect(panel).to_be_visible()
    expect(page.locator("#steering-empty")).to_be_visible()
    expect(page.locator("#steering-count")).to_have_text("0")
    # Add a note (API), then re-open the panel to refresh it.
    page.evaluate(
        """async () => {
            const r = await fetch('/api/steering', {
                method: 'POST',
                headers: {'Content-Type': 'application/json'},
                body: JSON.stringify({note: 'a11y note one'}),
            });
            if (!r.ok) throw new Error('steering POST failed: ' + r.status);
        }"""
    )
    toggle.click()  # close
    toggle.click()  # re-open (refreshes the list)
    note_item = page.locator("#steering-list .steering-note")
    expect(note_item).to_have_count(1)
    expect(note_item.locator(".steering-note-text")).to_have_text("a11y note one")
    # The per-note delete is a real, labeled button (≥44px target).
    delete = page.locator("#steering-list .steering-delete")
    expect(delete).to_have_attribute("type", "button")
    assert (delete.get_attribute("aria-label") or "").startswith("Delete tuning note:")
    box = delete.bounding_box()
    assert box is not None and box["height"] >= 44
    # Delete: list empties, count updates, the live region announces it.
    delete.click()
    expect(page.locator("#steering-list .steering-note")).to_have_count(0)
    expect(page.locator("#steering-count")).to_have_text("0")
    expect(announcer).to_contain_text("deleted")
    # And the toggle closes cleanly again.
    toggle.click()
    expect(toggle).to_have_attribute("aria-expanded", "false")
    expect(panel).to_be_hidden()
@@ -63,7 +63,7 @@ def _seed_kb(mock_port: int) -> ImportSummary:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    summary = _run_in_thread(_import_fixtures(mock_port))
-    assert summary is not None and summary.added == 3
+    assert summary is not None and summary.added == 8  # A9 formats
    return summary
@@ -0,0 +1,281 @@
 """Phase 17 E2E (Playwright, mock-only): the model's "thinking" display.
 Story: ``.agent/user_stories/thinking-display.md``
 Run in isolation (DB must be up: ``podman compose up -d db``):
    uv run pytest tests/e2e/test_thinking_display.py -v --no-cov
 MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported here. The real
 ``turbo`` thinks on *every* turn, which would break the no-thinking
 regression test (scenario 4) — the deterministic mock's ``think out loud``
 trigger (mock_llm.py) keeps all five scenarios reproducible.
 Test → story mapping (Playwright Mapping Rule):
 1. ``test_thinking_block_streams_open_then_collapses``
 2. ``test_thinking_toggle_after_done``
 3. ``test_thinking_restored_after_reload``
 4. ``test_no_thinking_block_without_trigger``
 5. ``test_thinking_with_deflection``
 Determinism note: the mock paces every SSE frame at 0.02s and the thinking
 text is ~700–900 chars (≈ 60–75 frames ≈ 1.2–1.5s) before the first
 content frame, so "attach → assert open" runs well inside the open window
 on headless Chromium; all other assertions are made after the send button
 re-enables (fully settled state).
 """
 from __future__ import annotations
 import asyncio
 import json
 import re
 from collections.abc import Iterator
 from pathlib import Path
 from threading import Thread
 from typing import Any
 import pytest
 from playwright.sync_api import Page, expect
 from sqlalchemy import text
 from app.config import Settings
 from app.db import SessionLocal
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
 REPO = Path(__file__).resolve().parents[2]
 FIXTURES = REPO / "tests" / "fixtures" / "docs"
 THINK_QUESTION = "think out loud — how is my kubernetes cluster set up?"
 PLAIN_QUESTION = "How is my Kubernetes cluster set up?"
 THINK_DEFLECT_QUESTION = "think out loud — tell me about quantum wormhole cooling"
 MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
 DEFLECT_PHRASE = r"haven't done anything like that"
 #: Line fragment the mock's deterministic scratchpad must carry — the
 #: suite keys off it (mock_llm.compose_thinking).
 THINKING_FRAGMENT = "Step 2: Check my notes"
 STORAGE_KEY = "bor.chat.v1"
 #: Phase-10 viewer URL + phase-13 back=/ (byte-identical to the chip the
 #: persistence suite pins — grounded-turn sources are unchanged by 17).
 CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
 async def _import_fixtures(mock_port: int) -> ImportSummary:
    kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
    settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
    return await import_sources([FIXTURES], LLMClient(settings))
 def _run_in_thread(coro: Any) -> Any:
    """Run a coroutine on a worker thread.
    Playwright's sync API keeps an asyncio loop running on the test thread,
    so ``asyncio.run`` cannot be called directly from a test body.
    """
    box: dict[str, Any] = {}
    def runner() -> None:
        try:
            box["value"] = asyncio.run(coro)
        except BaseException as e:  # noqa: BLE001 — re-raised on the test thread
            box["error"] = e
    t = Thread(target=runner)
    t.start()
    t.join()
    if "error" in box:
        raise box["error"]
    return box["value"]
 def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
    """Truncate the KB (and query log), then optionally re-import fixtures."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log"))
        db.commit()
    if not seed:
        return None
    return _run_in_thread(_import_fixtures(mock_port))
@pytest.fixture()
 def seeded_kb(mock_llm: int, db_ready: None) -> Iterator[None]:
    """A fresh KB seeded from ``tests/fixtures/docs`` (8 docs, A9 formats),
    truncated again on teardown. ``db_ready`` (conftest) skips with clear
    instructions when Postgres is down."""
    summary = _reset_db(mock_llm, seed=True)
    assert summary is not None and summary.added == 8
    yield
    _reset_db(mock_llm, seed=False)
 def send_and_wait(page: Page, question: str) -> None:
    """Type into #message-input, submit via #composer, then wait until the
    last brain message settles (send button re-enabled, label "Send")."""
    page.fill("#message-input", question)
    page.evaluate("() => document.querySelector('#composer').requestSubmit()")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
    # The mock streams at 0.02s/chunk, so thinking + answer land in a few
    # seconds — 30s is generous on headless Chromium.
    expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
    expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
    expect(page.locator("#send-label")).to_have_text("Send")
 # ---------------------------------------------------------------------------
 # 1. Streaming: the block attaches OPEN at the first thinking event, then
 #    auto-collapses when the first answer token lands
 # ---------------------------------------------------------------------------
 def test_thinking_block_streams_open_then_collapses(
    page: Page, app_url: str, seeded_kb: None
 ) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    page.fill("#message-input", THINK_QUESTION)
    page.evaluate("() => document.querySelector('#composer').requestSubmit()")
    expect(page.locator(".msg.user .bubble").last).to_contain_text(THINK_QUESTION)
    # The block attaches at the FIRST thinking event — before any answer
    # token — and is created OPEN.
    details = page.locator(".msg.brain").last.locator("details.thinking")
    details.wait_for(state="attached", timeout=10_000)
    # The ~800-char thinking stream (≈1.3s) keeps the block open right
    # after attach — assert while it is still streaming.
    expect(details).to_have_attribute("open", "")
    expect(details.locator(".thinking-text")).not_to_have_text("")
    # First answer token: the block auto-collapses and stays closed.
    bubble = page.locator(".msg.brain .bubble").last
    expect(bubble).not_to_have_text("", timeout=30_000)
    expect(details).not_to_have_attribute("open")
    # Settled: full scratchpad, grounded mock answer, source chip(s),
    # and the re-enabled send button.
    expect(details.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
    expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip.first).to_be_visible()
    expect(chip.first).to_have_attribute("href", CHIP_HREF)
    expect(page.locator("#send-btn")).to_be_enabled()
    expect(page.locator("#send-label")).to_have_text("Send")
 # ---------------------------------------------------------------------------
 # 2. Toggle: after settle the block is closed; the summary re-opens it
 #    (a real keyboard-focusable control) and closes it again
 # ---------------------------------------------------------------------------
 def test_thinking_toggle_after_done(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    send_and_wait(page, THINK_QUESTION)
    last = page.locator(".msg.brain").last
    details = last.locator("details.thinking")
    expect(details).to_have_count(1)
    expect(details).not_to_have_attribute("open")  # auto-collapsed at first token
    # The summary is a real, keyboard-focusable control.
    details.locator("summary").focus()
    assert page.evaluate("() => document.activeElement.tagName") == "SUMMARY"
    # Open: the full scratchpad is visible.
    details.locator("summary").click()
    expect(details).to_have_attribute("open", "")
    text_el = details.locator(".thinking-text")
    expect(text_el).to_be_visible()
    expect(text_el).to_contain_text(THINKING_FRAGMENT)
    expect(text_el).to_contain_text("nothing is invented")
    # Closed again — user control in both directions.
    details.locator("summary").click()
    expect(details).not_to_have_attribute("open")
    expect(text_el).not_to_be_visible()
 # ---------------------------------------------------------------------------
 # 3. Persistence: the thinking block (and its text) survives a reload,
 #    restored COLLAPSED — phase-14 restore path + phase-17 field
 # ---------------------------------------------------------------------------
 def test_thinking_restored_after_reload(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    send_and_wait(page, THINK_QUESTION)
    # The live block is collapsed; capture what it shows and what the
    # turn persisted (raw text, same as what was rendered).
    details = page.locator(".msg.brain").last.locator("details.thinking")
    expect(details).not_to_have_attribute("open")
    captured = details.locator(".thinking-text").text_content()
    assert captured
    # The persisted raw text is the same scratchpad (renderMarkdown turns
    # the line breaks into <br>, which textContent drops — compare without
    # whitespace).
    raw = json.loads(
        page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
    )["messages"][1]["thinking"]
    assert re.sub(r"\s+", "", raw) == re.sub(r"\s+", "", captured)
    page.reload()
    expect(page.locator("#empty-state")).to_be_hidden()
    restored = page.locator(".msg.brain").last.locator("details.thinking")
    expect(restored).to_have_count(1)
    expect(restored).not_to_have_attribute("open")  # restored COLLAPSED
    expect(restored.locator(".thinking-text")).to_have_text(captured)
    # Answer bubble + source chip are intact (phase-14 restore path).
    expect(page.locator(".msg.brain .bubble").last).to_contain_text(MOCK_ANSWER_MARKER)
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip.first).to_have_attribute("href", CHIP_HREF)
 # ---------------------------------------------------------------------------
 # 4. No thinking, no block: a model/turn that emits no reasoning renders
 #    exactly as before (no layout regression)
 # ---------------------------------------------------------------------------
 def test_no_thinking_block_without_trigger(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    send_and_wait(page, PLAIN_QUESTION)
    # No trigger → no thinking events → no block anywhere on the page.
    expect(page.locator("details.thinking")).to_have_count(0)
    # The turn itself is complete and grounded, exactly as before phase 17.
    chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
    expect(chip.first).to_be_visible()
    expect(chip.first).to_have_attribute("href", CHIP_HREF)
 # ---------------------------------------------------------------------------
 # 5. Coexistence: the honesty gate (deflection) and the thinking block
 #    on the same turn
 # ---------------------------------------------------------------------------
 def test_thinking_with_deflection(page: Page, app_url: str, seeded_kb: None) -> None:
    page.set_default_timeout(30_000)
    page.goto(app_url)
    send_and_wait(page, THINK_DEFLECT_QUESTION)
    last = page.locator(".msg.brain").last
    # The honesty gate fired: amber deflected bubble + "Maybe try" chips.
    expect(last).to_have_class(re.compile(r"is-deflected"))
    expect(last.locator(".bubble")).to_contain_text(
        re.compile(DEFLECT_PHRASE, re.IGNORECASE)
    )
    chips = last.locator(".maybe-try .suggestion-chip")
    expect(chips.first).to_be_visible()
    assert chips.count() >= 2
    # And the thinking block came along, closed, with its scratchpad.
    details = last.locator("details.thinking")
    expect(details).to_have_count(1)
    expect(details).not_to_have_attribute("open")
    expect(details.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
@@ -0,0 +1,5 @@
 # Vendored Junk
 This file lives under a dot-prefixed directory and must **never** be
 imported into the knowledge base (A9 hidden-dir skip). It exists so the
 retrieval-quality E2E can prove the filter works.
@@ -0,0 +1,24 @@
 # gitlab stack — single container + gitlab-data volume
 services:
  gitlab:
    image: gitlab/gitlab-ce:17.2.1-ce.0
    container_name: gitlab
    restart: unless-stopped
    hostname: "gitlab.reeseapps.com"
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.reeseapps.com'
        gitlab_rails['gitlab_shell_ssh_port'] = 2222
    ports:
      - "8929:80"
      - "2222:22"
    volumes:
      - gitlab-data:/var/opt/gitlab
    shm_size: "256m"
    deploy:
      resources:
        limits:
          memory: 4G
 volumes:
  gitlab-data:
@@ -0,0 +1,27 @@
 # Gitlab
 Gitlab CE runs as a single Docker container on the `gitlab` host
 (`10.0.1.14`), managed by Ansible (`deployments/gitlab/`).
 ## Install
 1. Install Docker and the compose plugin on the host.
 2. Create the `gitlab-data` volume: `docker volume create gitlab-data`.
 3. Run the stack from `gitlab-compose.yaml`:
   `docker compose -f gitlab-compose.yaml up -d`
 4. Wait ~2 minutes for the initial gitlab migration to finish.
 ## Access
 - Web UI: https://gitlab.reeseapps.com (Traefik routes it to port 8929).
 - Root password: `gitlab-root-password` file in the repo (rotated yearly).
 - Backup: nightly `gitlab-backup create` at 03:30, copy to BorgBase.
 ## Operations
 - Upgrade gitlab: bump the image tag in the compose file,
  `docker compose up -d gitlab`, watch the logs for the version banner.
 - Logs: `docker logs -f gitlab` or the gitlab admin area → Admin
  area → Logs.
 - If the container is OOM-killed, raise the memory limit in the compose
  file (it needs 4GB free).
@@ -0,0 +1,14 @@
 {
  "comment": "Static DNS overrides for the homelab Pi-hole (applied by the ddns updater).",
  "hosts": {
    "kafkabridge": "10.0.3.7",
    "k3s-control": "10.0.1.10",
    "gitea": "10.0.2.21",
    "ntfy": "10.0.2.30"
  },
  "domains": [
    "reeseapps.com",
    "homelab.lan"
  ],
  "expiry_days": 365
 }
@@ -0,0 +1,60 @@
 """Uptime probe — the homelab healthcheck runner.
 Polls every service listed in ``CHECKS`` every 5 minutes and posts a
 failure to the ntfy topic ``homelab-alerts``.
 """
 from __future__ import annotations
 import subprocess
 #: (name, health URL) for every long-running service.
 CHECKS: list[tuple[str, str]] = [
    ("k3s", "https://10.0.1.10:6443/healthz"),
    ("gitea", "https://gitea.reeseapps.com/api/healthz"),
    ("ntfy", "https://ntfy.reeseapps.com/health"),
    ("gitlab", "https://gitlab.reeseapps.com/-/health_check"),
 ]
 def probe(name: str, url: str) -> bool:
    """One HTTP check; returns True when the service answered 200."""
    result = subprocess.run(
        ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "10", url],
        capture_output=True,
        text=True,
    )
    return result.stdout.strip() == "200"
 def notify_failure(name: str) -> None:
    """Push an alert to ntfy (best effort — alerting must not crash the probe)."""
    subprocess.run(
        [
            "curl",
            "-s",
            "-X",
            "POST",
            "https://ntfy.reeseapps.com/homelab-alerts",
            "-H",
            "Title: homelab check failed",
            "-d",
            f"{name} is down",
        ],
        capture_output=True,
    )
 def run_round() -> int:
    """Probe everything once; returns the number of failing services."""
    failed = 0
    for name, url in CHECKS:
        if not probe(name, url):
            failed += 1
            notify_failure(name)
    return failed
 if __name__ == "__main__":
    import sys
    sys.exit(run_round())
@@ -0,0 +1,14 @@
 SSH notes for the homelab jump host.
 All admin hosts are reachable through the jump box at 10.0.1.2
 (`ssh reese@jump`). The `~/.ssh/config` aliases:
 k3s — the kubernetes control plane node (10.0.1.10, user talos)
 gitlab — the gitlab container host (10.0.1.14)
 nuc — the low-power media box (10.0.1.20)
 Keys: ed25519 per host, no passwords. The old RSA key was retired in
 2025 and its line removed from authorized_keys on every host.
 Forwarding X11 stays off everywhere; use `ssh -L` port forwards for the
 occasional GUI tool instead.
@@ -52,12 +52,17 @@ def test_suggestions_honors_bor_suggestions_env_override(monkeypatch) -> None:
@pytest.mark.parametrize(
    ("path", "marker"),
-    [("/", "Brain of Reese"), ("/sources.html", "Knowledge base")],
+    [
        ("/", "Brain of Reese"),
        ("/sources.html", "Knowledge base"),
        ("/document.html", "Brain of Reese"),  # phase 10: viewer page
        ("/login.html", "Sign in"),  # phase 16: admin sign-in page
    ],
 )
 def test_html_pages_served_locally_no_cdn(client, path: str, marker: str) -> None:
-    """No-CDN check (PLAN §7.3, re-verified on BOTH pages in phase 07):
+    """No-CDN check (PLAN §7.3, re-verified on BOTH pages in phase 07 and
-    each page is served by FastAPI and references only same-origin assets
+    on the viewer page in phase 10): each page is served by FastAPI and
-    (no https:// script/link tags)."""
+    references only same-origin assets (no https:// script/link tags)."""
    r = client.get(path)
    assert r.status_code == 200
    assert marker in r.text
@@ -68,6 +73,10 @@ def test_html_pages_served_locally_no_cdn(client, path: str, marker: str) -> Non
 def test_styles_and_js_served(client) -> None:
    assert client.get("/assets/styles.css").status_code == 200
    assert client.get("/assets/app.js").status_code == 200
    assert client.get("/assets/sources.js").status_code == 200
    assert client.get("/assets/markdown.js").status_code == 200  # phase 10: shared renderer
    assert client.get("/assets/document.js").status_code == 200  # phase 10: viewer page
    assert client.get("/assets/login.js").status_code == 200  # phase 16: login page
 # Emoji code points banned from UI chrome (phase 08): the pictograph
@@ -92,10 +101,22 @@ def _find_emoji(text: str) -> list[str]:
@pytest.mark.parametrize(
-    "path", ["/", "/sources.html", "/assets/app.js", "/assets/styles.css"]
+    "path",
    [
        "/",
        "/sources.html",
        "/document.html",
        "/login.html",  # phase 16
        "/assets/app.js",
        "/assets/sources.js",
        "/assets/markdown.js",
        "/assets/document.js",
        "/assets/login.js",  # phase 16
        "/assets/styles.css",
    ],
 )
 def test_ui_chrome_has_no_emoji(client, path: str) -> None:
-    """Permanent regression guard (phase 08): the UI chrome — both pages,
+    """Permanent regression guard (phase 08): the UI chrome — all pages,
    the JS that renders it, and the stylesheet — is emoji-free."""
    r = client.get(path)
    assert r.status_code == 200
@@ -0,0 +1,190 @@
 """Integration: the auth surface (phase 16) + the public-API regression
 guards.
 Covers the full login/logout lifecycle against the real app (TestClient
 keeps the cookie jar): wrong password → 401 + still-gated; correct →
 204 + cookie → admin everywhere gated; logout → 403 again. And the
 **anonymous** guarantees that phase 16 must not break: the document
 viewer stays public (soft rule) and ``POST /api/chat`` still streams.
 Requires:  podman compose up -d db
 """
 from __future__ import annotations
 import asyncio
 from collections.abc import Iterator
 from pathlib import Path
 import pytest
 from fastapi.testclient import TestClient
 from sqlalchemy import text
 from test_chat_api import FakeRagLLM, _stream_chat
 from app.api import chat as chat_api
 from app.main import app as fastapi_app
 from app.models import Document
 from app.rag.importer import import_sources
 from tests.conftest import ADMIN_PASSWORD
 FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
@pytest.fixture(autouse=True)
 def clean_tables(db) -> Iterator[None]:
    """Docs + steering + query log are global state: reset around tests."""
    db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
    db.commit()
    yield
    db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
    db.commit()
@pytest.fixture()
 def seeded_kb(db) -> Iterator[FakeRagLLM]:
    """Fresh Postgres with the fixture docs imported (real pipeline)."""
    db.execute(text("TRUNCATE chunks, documents, query_log"))
    db.commit()
    llm = FakeRagLLM()
    summary = asyncio.run(import_sources([FIXTURES], llm, session=db))
    assert summary.added == 8  # A9 formats; .hidden/ skipped
    yield llm
    db.execute(text("TRUNCATE chunks, documents, query_log"))
    db.commit()
 def _seed_one_doc(db) -> Document:
    """One minimal document (for the public document-content endpoint)."""
    doc = Document(
        source="docs",
        path="homelab/kubernetes.md",
        full_path="/tmp/kubernetes.md",
        title="Kubernetes Homelab Cluster",
        content="# Kubernetes Homelab Cluster\n\nTalos on 3 nodes.",
        content_hash="c" * 64,
    )
    db.add(doc)
    db.commit()
    db.refresh(doc)
    return doc
 def test_wrong_password_401_and_still_gated(client: TestClient) -> None:
    r = client.post("/api/login", json={"password": "not-the-password"})
    assert r.status_code == 401
    assert r.json() == {"detail": "invalid password"}
    # No session state was created by the failed attempt.
    assert "bor_session" not in client.cookies
    r = client.post("/api/login", json={"password": ""})  # empty → same 401
    assert r.status_code == 401
    assert r.json() == {"detail": "invalid password"}
    # The gated surface stays closed (anonymous).
    assert client.get("/api/docs").status_code == 403
    r = client.get("/api/steering")
    assert r.status_code == 403
    assert r.json() == {"detail": "admin only"}
    assert client.post("/api/steering", json={"note": "x"}).status_code == 403
    assert client.get("/api/whoami").json() == {
        "authenticated": False,
        "role": "anonymous",
    }
 def test_login_logout_lifecycle(client: TestClient) -> None:
    # Anonymous shape before anything.
    who = client.get("/api/whoami").json()
    assert who == {"authenticated": False, "role": "anonymous"}
    # Wrong first, right second — one generic 401, then success.
    assert client.post("/api/login", json={"password": "nope"}).status_code == 401
    r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
    assert r.status_code == 204
    assert "bor_session" in client.cookies  # the signed session cookie
    # Admin: whoami + the gated endpoints all open up.
    assert client.get("/api/whoami").json() == {
        "authenticated": True,
        "role": "admin",
    }
    assert client.get("/api/docs").status_code == 200
    created = client.post("/api/steering", json={"note": "  be terse  "})
    assert created.status_code == 201
    note = created.json()
    assert note["note"] == "be terse"
    listing = client.get("/api/steering")
    assert listing.status_code == 200
    assert [n["note"] for n in listing.json()["notes"]] == ["be terse"]
    assert client.delete(f"/api/steering/{note['id']}").status_code == 204
    assert client.get("/api/steering").json() == {"notes": []}
    # Logout: 204, cookie gone, gated again.
    assert client.post("/api/logout").status_code == 204
    assert "bor_session" not in client.cookies
    assert client.get("/api/whoami").json() == {
        "authenticated": False,
        "role": "anonymous",
    }
    assert client.get("/api/docs").status_code == 403
    r = client.get("/api/steering")
    assert r.status_code == 403
    assert r.json() == {"detail": "admin only"}
    # Logout is idempotent (anonymous logout is still a clean 204).
    assert client.post("/api/logout").status_code == 204
    assert client.get("/api/whoami").json()["authenticated"] is False
 def test_forged_cookie_is_rejected(client: TestClient) -> None:
    client.cookies.set("bor_session", "tampered-session-blob")
    assert client.get("/api/whoami").json() == {
        "authenticated": False,
        "role": "anonymous",
    }
    assert client.get("/api/docs").status_code == 403
 def test_anonymous_document_content_stays_public(client: TestClient, db) -> None:
    """Soft rule (phase 16): the catalog is gated, the viewer is not."""
    _seed_one_doc(db)
    r = client.get(
        "/api/documents/content", params={"source": "docs", "path": "homelab/kubernetes.md"}
    )
    assert r.status_code == 200
    body = r.json()
    assert body["title"] == "Kubernetes Homelab Cluster"
    assert "Talos" in body["content"]
    assert set(body) == {
        "source",
        "path",
        "title",
        "format",
        "content",
        "indexed_at",
        "chunks",
    }
    # Unknown docs still 404 anonymously (no enumeration of titles).
    r = client.get("/api/documents/content", params={"source": "docs", "path": "nope.md"})
    assert r.status_code == 404
 def test_anonymous_chat_still_streams(
    client: TestClient, seeded_kb: FakeRagLLM
 ) -> None:
    """Regression guard: sign-in must not have locked chat (A10 public)."""
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        status, content_type, frames = _stream_chat(client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    assert status == 200
    assert content_type.startswith("text/event-stream")
    deltas = [f for f in frames if f.get("type") == "delta"]
    assert len(deltas) >= 2  # genuinely streamed
    assert frames[-1]["type"] == "done"
    assert frames[-1]["deflected"] is False
    assert frames[-1]["sources"][0]["path"] == "homelab/kubernetes.md"
@@ -28,7 +28,7 @@ from app.config import Settings, get_settings
 from app.main import app as fastapi_app
 from app.models import Chunk, QueryLog
 from app.rag.importer import import_sources
-from app.rag.llm import EmbeddingError, LLMError
+from app.rag.llm import EmbeddingError, LLMError, StreamPiece
 FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
@@ -53,6 +53,7 @@ class FakeRagLLM:
    def __init__(
        self,
        answer: str = "Hey — you've got this! Talos, Cilium, three nodes. 🧠",
        thinking: str = "",
        embed_error: Exception | None = None,
        stream_error: Exception | None = None,
        fail_mid_stream: bool = False,
@@ -60,6 +61,7 @@ class FakeRagLLM:
        self.settings = Settings(_env_file=None)  # pyright: ignore[reportCallIssue]
        self.embed_batches = 0
        self.answer = answer
        self.thinking = thinking
        self.embed_error = embed_error
        self.stream_error = stream_error
        self.fail_mid_stream = fail_mid_stream
@@ -77,14 +79,20 @@ class FakeRagLLM:
        return _token_vec(text)
    async def chat_stream(self, messages: list[dict[str, str]]):
        """Typed stream (phase 17): ``thinking`` slices (same 12-char
        cadence as content) **before** the content pieces. With the
        default ``thinking=""`` this yields content-only pieces — today's
        behavior, new yield type."""
        self.seen_messages.append(messages)
        if self.stream_error is not None:
            raise self.stream_error
        if self.fail_mid_stream:
-            yield "partial "
+            yield StreamPiece("content", "partial ")
            raise LLMError("mid-stream dropout")
        for i in range(0, len(self.thinking), 12):
            yield StreamPiece("thinking", self.thinking[i : i + 12])
        for i in range(0, len(self.answer), 12):
-            yield self.answer[i : i + 12]
+            yield StreamPiece("content", self.answer[i : i + 12])
@pytest.fixture()
@@ -94,7 +102,7 @@ def seeded_kb(db) -> Iterator[FakeRagLLM]:
    db.commit()
    llm = FakeRagLLM()
    summary = asyncio.run(import_sources([FIXTURES], llm, session=db))
-    assert summary.added == 3
+    assert summary.added == 8  # A9 formats; .hidden/ skipped
    yield llm
    db.execute(text("TRUNCATE chunks, documents, query_log"))
    db.commit()
@@ -150,6 +158,75 @@ def test_chat_streams_deltas_then_done_with_sources(client, db, seeded_kb: FakeR
    assert "HONESTY GATE" in system["content"]
 def test_chat_streams_thinking_before_deltas(client, db, seeded_kb: FakeRagLLM) -> None:
    """Phase 17: ``thinking`` frames precede every ``delta`` frame and
    reassemble to the model's reasoning; the ``done`` contract is
    unchanged."""
    thinker = FakeRagLLM(
        thinking=(
            "Step 1: parse the question. Step 2: check the kubernetes doc. "
            "Step 3: name Talos, Cilium, three nodes. Step 4: answer."
        )
    )
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: thinker
    try:
        _, _, frames = _stream_chat(client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    thinking = [f for f in frames if f.get("type") == "thinking"]
    deltas = [f for f in frames if f.get("type") == "delta"]
    assert len(thinking) >= 1  # genuinely streamed
    assert len(deltas) >= 2
    # Every thinking frame precedes every delta frame.
    ordered = [f["type"] for f in frames if f["type"] in ("thinking", "delta")]
    assert ordered == ["thinking"] * len(thinking) + ["delta"] * len(deltas)
    assert all(set(f.keys()) == {"type", "text"} for f in thinking)
    assert "".join(f["text"] for f in thinking) == thinker.thinking
    assert "".join(d["text"] for d in deltas) == thinker.answer
    # Done still last; sources unchanged by the thinking extension.
    done = frames[-1]
    assert done["type"] == "done"
    assert done["deflected"] is False
    assert done["suggestions"] == []
    assert done["sources"][0]["path"] == "homelab/kubernetes.md"
    assert done["sources"][0]["source"] == "docs"
    assert not any(f.get("type") == "error" for f in frames)
 def test_chat_thinking_suppressed_when_disabled(
    client, db, monkeypatch: pytest.MonkeyPatch
 ) -> None:
    """Phase 17 kill-switch: ``BOR_STREAM_THINKING=0`` drops every
    ``thinking`` frame; the delta stream is byte-identical to the
    thinking-free case."""
    thinker = FakeRagLLM(thinking="hidden reasoning that must never reach the wire")
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: thinker
    # Same honesty gate the conftest/module already use (mock-calibrated
    # 0.30 from the environment) — only the kill-switch changes.
    live = get_settings()
    monkeypatch.setattr(
        chat_api,
        "get_settings",
        lambda: Settings(
            _env_file=None,  # pyright: ignore[reportCallIssue]
            relevance_threshold=live.relevance_threshold,
            stream_thinking=False,
        ),
    )
    try:
        _, _, frames = _stream_chat(client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    assert not any(f.get("type") == "thinking" for f in frames)
    deltas = [f for f in frames if f.get("type") == "delta"]
    assert "".join(d["text"] for d in deltas) == thinker.answer
    assert frames[-1]["type"] == "done"
    assert not any(f.get("type") == "error" for f in frames)
 def test_chat_writes_query_log_row(client, db, seeded_kb: FakeRagLLM) -> None:
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
@@ -163,12 +240,19 @@ def test_chat_writes_query_log_row(client, db, seeded_kb: FakeRagLLM) -> None:
    assert row.question == QUESTION
    assert row.deflected is False
    total_chunks = db.scalar(select(func.count()).select_from(Chunk))
-    assert row.chunk_hits == min(get_settings().top_k_chunks, total_chunks)
+    # chunk_hits is the fused candidate set (cosine top-N ∪ FTS top-N).
    assert 1 <= row.chunk_hits <= total_chunks
    assert row.top_score > 0.0  # genuine token-overlap cosine, best hit
    assert row.top_score >= get_settings().relevance_threshold  # why the gate answered
    assert row.top_score <= 1.0
    assert "docs/homelab/kubernetes.md" in row.sources
    assert row.latency_ms >= 0
    # Why the gate answered (A8 revised): cosine over the threshold OR a
    # lexical hit. The mock-calibrated threshold (0.30, see tests/conftest.py)
    # makes the cosine branch true here; the FTS branch is covered too —
    # "kubernetes" / "cluster" match the doc's tsvector.
    thr = get_settings().relevance_threshold
    assert row.top_score >= thr or (row.fts_hits or 0) > 0
    assert (row.fts_hits or 0) >= 1  # the lexical branch really fired
 def test_off_topic_question_deflects_honestly(client, db, seeded_kb: FakeRagLLM) -> None:
@@ -202,14 +286,46 @@ def test_off_topic_question_deflects_honestly(client, db, seeded_kb: FakeRagLLM)
    assert "Talos Linux" not in system["content"]  # full doc content never sent
    assert "<documents>" not in system["content"]
-    # Durable record: deflected=true + the weak top_score.
+    # Durable record: deflected=true + the weak top_score. Deflection is
    # only reached when the cosine is under the threshold AND no chunk
    # FTS-matches the question — so fts_hits must be zero here.
    row = db.scalars(select(QueryLog)).one()
    assert row.question == OFF_TOPIC
    assert row.deflected is True
    assert 0.0 < row.top_score < get_settings().relevance_threshold
    assert row.fts_hits == 0
    assert row.chunk_hits >= 1
 def test_keyword_question_grounded_by_lexical_hit_despite_weak_cosine(
    client, db, seeded_kb: FakeRagLLM
 ) -> None:
    """Phase 09: a name-your-tool question the vector model barely ranks
    ("kafkabridge" only appears in static-dns.json) must still be grounded
    via the FTS branch — LOW only fires at weak cosine AND zero hits."""
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _, _, frames = _stream_chat(client, "How does kafkabridge work?")
    finally:
        fastapi_app.dependency_overrides.clear()
    done = frames[-1]
    assert done["type"] == "done"
    assert done["deflected"] is False  # weak cosine, but a lexical hit
    assert done["suggestions"] == []
    sources = done["sources"]
    assert sources and sources[0]["path"] == "homelab/networking/static-dns.json"
    (system, _user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
    assert "<relevance>HIGH</relevance>" in system["content"]  # grounded prompt
    row = db.scalars(select(QueryLog)).one()
    assert row.deflected is False
    assert row.top_score < get_settings().relevance_threshold  # weak vector score
    assert (row.fts_hits or 0) >= 1  # …and it is the FTS hit that grounds it
    assert "docs/homelab/networking/static-dns.json" in row.sources
 def test_chat_empty_kb_streams_empty_sources(client, db) -> None:
    db.execute(text("TRUNCATE chunks, documents, query_log"))
    db.commit()
@@ -12,15 +12,15 @@ from sqlalchemy import text
 from app.models import Chunk, Document
-def test_docs_empty_shape(client, db) -> None:
+def test_docs_empty_shape(admin_client, db) -> None:
    db.execute(text("TRUNCATE chunks, documents"))
    db.commit()
-    r = client.get("/api/docs")
+    r = admin_client.get("/api/docs")
    assert r.status_code == 200
    assert r.json() == {"documents": []}
-def test_docs_populated_shape_sorted_with_chunk_counts(client, db) -> None:
+def test_docs_populated_shape_sorted_with_chunk_counts(admin_client, db) -> None:
    db.execute(text("TRUNCATE chunks, documents"))
    db.commit()
    now = datetime.now(UTC)
@@ -50,7 +50,7 @@ def test_docs_populated_shape_sorted_with_chunk_counts(client, db) -> None:
    )
    db.commit()
-    r = client.get("/api/docs")
+    r = admin_client.get("/api/docs")
    assert r.status_code == 200
    body = r.json()
    # Ordered by (source, path): Deployments < Homelab.
@@ -69,8 +69,8 @@ def test_docs_populated_shape_sorted_with_chunk_counts(client, db) -> None:
    db.commit()
-def test_docs_response_matches_schema_shape(client, db) -> None:
+def test_docs_response_matches_schema_shape(admin_client, db) -> None:
-    r = client.get("/api/docs")
+    r = admin_client.get("/api/docs")
    assert r.status_code == 200
    body = r.json()
    assert set(body) == {"documents"}
@@ -0,0 +1,137 @@
 """Integration tests: GET /api/documents/content — the viewer's data source.
 Uses the real compose Postgres (``db`` fixture) and FastAPI's TestClient:
 * 200 with the full field set for a seeded document (all formats);
 * 404 for an unknown (source, path) pair;
 * 404 for traversal-style ``path`` values (no filesystem access → no leak).
 """
 from __future__ import annotations
 from datetime import UTC, datetime
 from sqlalchemy import text
 from app.models import Chunk, Document
 def _seed_doc(
    db,
    source: str = "Homelab",
    path: str = "kubernetes.md",
    title: str = "Kubernetes Homelab Cluster",
    content: str = "# Kubernetes\n\nTalos on 3 nodes.",
    chunks: int = 2,
 ) -> None:
    """Truncate the KB and insert one document with ``chunks`` chunk rows."""
    db.execute(text("TRUNCATE chunks, documents"))
    db.commit()
    doc = Document(
        source=source,
        path=path,
        full_path=f"/tmp/{path}",
        title=title,
        content=content,
        content_hash="a" * 64,
        indexed_at=datetime.now(UTC),
    )
    db.add(doc)
    db.flush()
    if chunks:
        db.add_all(
            Chunk(document_id=doc.id, position=i, content=f"chunk {i}", embedding=[0.01] * 768)
            for i in range(chunks)
        )
    db.commit()
 def test_content_200_all_fields(client, db) -> None:
    _seed_doc(db)
    try:
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "kubernetes.md"},
        )
        assert r.status_code == 200
        body = r.json()
        assert set(body) == {"source", "path", "title", "format", "content", "indexed_at", "chunks"}
        assert body["source"] == "Homelab"
        assert body["path"] == "kubernetes.md"
        assert body["title"] == "Kubernetes Homelab Cluster"
        assert body["format"] == "md"
        assert body["content"] == "# Kubernetes\n\nTalos on 3 nodes."
        assert body["chunks"] == 2
        datetime.fromisoformat(body["indexed_at"])  # raises if not ISO-8601
    finally:
        db.execute(text("TRUNCATE chunks, documents"))
        db.commit()
 def test_content_404_unknown_path(client, db) -> None:
    _seed_doc(db)
    try:
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "nope/missing.md"},
        )
        assert r.status_code == 404
        assert r.json() == {"detail": "document not found"}
        # A pair that exists under a DIFFERENT source is also 404 — both
        # values must match the row.
        r = client.get(
            "/api/documents/content",
            params={"source": "Deployments", "path": "kubernetes.md"},
        )
        assert r.status_code == 404
    finally:
        db.execute(text("TRUNCATE chunks, documents"))
        db.commit()
 def test_content_404_traversal_style_path_no_leak(client, db) -> None:
    """DB-only lookup: traversal strings are just non-existent rows — 404,
    and the response must not carry anything from the filesystem."""
    _seed_doc(db)
    try:
        for path in ("../../etc/passwd", "../kubernetes.md", "..%2F..%2Fetc%2Fpasswd"):
            r = client.get(
                "/api/documents/content",
                params={"source": "Homelab", "path": path},
            )
            assert r.status_code == 404, path
            assert r.json() == {"detail": "document not found"}, path
            assert "root:" not in r.text, path
    finally:
        db.execute(text("TRUNCATE chunks, documents"))
        db.commit()
 def test_content_format_from_suffix(client, db) -> None:
    """format = lowercased path suffix: yaml documents (phase 09 corpus) and
    the no-suffix fallback both flow through the same endpoint."""
    try:
        _seed_doc(
            db,
            path="container_gitlab/gitlab-compose.yaml",
            title="gitlab-compose",
            content="services:\n  gitlab:\n    image: gitlab/gitlab-ce",
            chunks=0,
        )
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "container_gitlab/gitlab-compose.yaml"},
        )
        assert r.status_code == 200
        body = r.json()
        assert body["format"] == "yaml"
        assert body["chunks"] == 0  # outerjoin → zero, not missing
        _seed_doc(db, path="README", title="README", content="plain text, no suffix", chunks=0)
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "README"},
        )
        assert r.status_code == 200
        assert r.json()["format"] == "text"  # no-suffix fallback
    finally:
        db.execute(text("TRUNCATE chunks, documents"))
        db.commit()
@@ -22,23 +22,41 @@ EXPECTED_DOCS = {
    ("docs", "homelab/kubernetes.md"),
    ("docs", "homelab/backups.md"),
    ("docs", "deployments/new-service.md"),
    ("docs", "homelab/container_gitlab/gitlab.md"),
    ("docs", "homelab/container_gitlab/gitlab-compose.yaml"),
    ("docs", "homelab/networking/static-dns.json"),
    ("docs", "homelab/scripts/uptime_probe.py"),
    ("docs", "homelab/ssh/ssh_aliases.txt"),
 }
-def test_import_fixtures_end_to_end(client, db) -> None:
+def test_import_fixtures_end_to_end(admin_client, db) -> None:
    db.execute(text("TRUNCATE chunks, documents, query_log"))
    db.commit()
    llm = FakeEmbedder()
    summary = asyncio.run(import_sources([FIXTURES], llm, session=db))
-    assert (summary.files, summary.added, summary.unchanged) == (3, 3, 0)
+    # Eight A9-format files; .hidden/junk.md is out of scope (A9 revised).
-    assert summary.chunks >= 3
+    assert (summary.files, summary.added, summary.unchanged) == (8, 8, 0)
    assert summary.chunks >= 8
    assert summary.formats == {"md": 4, "yaml": 1, "json": 1, "py": 1, "txt": 1}
    # PLAN §9 per-format summary line: highest count first, then alpha.
    assert summary.format_counts() == "md:4,json:1,py:1,txt:1,yaml:1"
    docs = db.scalars(select(Document)).all()
    assert {(d.source, d.path) for d in docs} == EXPECTED_DOCS
    titles = {d.path: d.title for d in docs}
    assert titles["homelab/kubernetes.md"] == "Kubernetes Homelab Cluster"
    assert titles["deployments/new-service.md"] == "Deploying a New Service"
    assert titles["homelab/container_gitlab/gitlab.md"] == "Gitlab"
    # Non-markdown titles come from the file stem (a leading ``#`` or docstring
    # line is a comment there, not a heading).
    assert titles["homelab/container_gitlab/gitlab-compose.yaml"] == "gitlab-compose"
    assert titles["homelab/scripts/uptime_probe.py"] == "uptime_probe"
    assert titles["homelab/networking/static-dns.json"] == "static-dns"
    assert titles["homelab/ssh/ssh_aliases.txt"] == "ssh_aliases"
    # Hidden junk was never imported.
    assert not any(".hidden" in d.path for d in docs)
    # Full content is stored — that is what the RAG context will be.
    k8s = next(d for d in docs if d.path == "homelab/kubernetes.md")
    assert "Talos Linux" in k8s.content and k8s.content_hash
@@ -49,16 +67,16 @@ def test_import_fixtures_end_to_end(client, db) -> None:
        assert c.embedding is not None and len(c.embedding) == 768
    # The Sources page consumes exactly this shape.
-    r = client.get("/api/docs")
+    r = admin_client.get("/api/docs")  # phase 16: the catalog is admin-only
    assert r.status_code == 200
    body = r.json()
-    assert len(body["documents"]) == 3
+    assert len(body["documents"]) == 8
    assert all(d["chunks"] >= 1 for d in body["documents"])
    # Idempotent re-run: nothing re-embedded.
    calls_before = len(llm.calls)
    s2 = asyncio.run(import_sources([FIXTURES], llm, session=db))
-    assert s2.unchanged == 3 and s2.added == 0
+    assert s2.unchanged == 8 and s2.added == 0
    assert len(llm.calls) == calls_before  # unchanged → no embedding requests
    db.execute(text("TRUNCATE chunks, documents, query_log"))
@@ -0,0 +1,72 @@
 """Integration: migration 0002 (hybrid retrieval) schema contract.
 Asserts the state the migration must leave on the live schema:
 ``chunks.tsv`` as a stored generated tsvector, its GIN index, and the
 nullable ``query_log.fts_hits`` column (pre-0002 rows stay NULL, so it
 must accept NULL and an int). Requires ``podman compose up -d db``.
 """
 from __future__ import annotations
 from sqlalchemy import text
 def test_migration_0002_schema_contract(db) -> None:
    tsv_col = db.execute(
        text(
            "SELECT count(*) FROM information_schema.columns"
            " WHERE table_name = 'chunks' AND column_name = 'tsv'"
        )
    ).scalar()
    assert tsv_col == 1, "chunks.tsv (stored tsvector) is missing"
    gin = db.execute(
        text(
            "SELECT count(*) FROM pg_indexes"
            " WHERE tablename = 'chunks' AND indexdef ILIKE '%USING gin%'"
            " AND indexdef ILIKE '%tsv%'"
        )
    ).scalar()
    assert gin == 1, "GIN index on chunks.tsv is missing"
    fts = db.execute(
        text(
            "SELECT is_nullable = 'YES' FROM information_schema.columns"
            " WHERE table_name = 'query_log' AND column_name = 'fts_hits'"
        )
    ).scalar()
    assert fts is True, "query_log.fts_hits must exist and be nullable (pre-0002 rows)"
 def test_tsv_is_generated_and_lexically_queryable(db) -> None:
    """The tsvector is generated from ``content`` (not maintained by app
    code) and answers a tsquery — the retrieval path's lexical branch."""
    doc_id = db.execute(text("SELECT gen_random_uuid()")).scalar()
    try:
        db.execute(
            text(
                "INSERT INTO documents (id, source, path, full_path, title, content,"
                " content_hash, indexed_at) VALUES"
                " (:id, 'mig_test', 't.md', '/t.md', 'T', 'kafkabridge routes here',"
                " repeat('0', 64), now())"
            ),
            {"id": doc_id},
        )
        db.execute(
            text(
                "INSERT INTO chunks (id, document_id, position, content) VALUES"
                " (gen_random_uuid(), :id, 0, 'kafkabridge routes here')"
            ),
            {"id": doc_id},
        )
        db.commit()
        hit = db.execute(
            text(
                "SELECT count(*) FROM chunks"
                " WHERE tsv @@ to_tsquery('english', 'kafkabridge')"
            )
        ).scalar()
        assert hit == 1
    finally:
        db.execute(text("DELETE FROM chunks WHERE document_id = :id"), {"id": doc_id})
        db.execute(text("DELETE FROM documents WHERE id = :id"), {"id": doc_id})
        db.commit()
@@ -0,0 +1,253 @@
 """Integration: steering notes (phase 15) — CRUD + system-prompt injection.
 Real Postgres (``podman compose up -d db``); the chat path reuses the
 deterministic fake LLM from ``test_chat_api`` (token-overlap embeddings),
 so the stored note's journey — API → Postgres → ``<tuning>`` section of
 the captured system prompt — is verified end-to-end without a network.
 Requires:  podman compose up -d db
 """
 from __future__ import annotations
 import asyncio
 import logging
 import uuid
 from collections.abc import Iterator
 from datetime import UTC, datetime, timedelta
 from pathlib import Path
 import pytest
 from fastapi.testclient import TestClient
 from sqlalchemy import select, text
 from test_chat_api import FakeRagLLM, _stream_chat
 from app.api import chat as chat_api
 from app.main import app as fastapi_app
 from app.models import SteeringNote
 from app.rag.importer import import_sources
 FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
 QUESTION = "How is my Kubernetes cluster set up?"
 OFF_TOPIC = "How do I bake sourdough bread?"
 NOTE = "STEEER-MARKER be concise"
@pytest.fixture(autouse=True)
 def clean_steering(db) -> Iterator[None]:
    """Steering notes + query log are global state: reset around every test."""
    db.execute(text("TRUNCATE steering_notes, query_log"))
    db.commit()
    yield
    db.execute(text("TRUNCATE steering_notes, query_log"))
    db.commit()
@pytest.fixture()
 def seeded_kb(db) -> Iterator[FakeRagLLM]:
    """Fresh Postgres with the fixture docs imported (real pipeline)."""
    db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
    db.commit()
    llm = FakeRagLLM()
    summary = asyncio.run(import_sources([FIXTURES], llm, session=db))
    assert summary.added == 8  # A9 formats; .hidden/ skipped
    yield llm
    db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
    db.commit()
 def _turn_log_lines(caplog: pytest.LogCaptureFixture) -> list[str]:
    """The per-turn ``question=…`` log lines (PLAN §9) from this test."""
    return [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
 # ---------- CRUD ----------
 def test_create_note_returns_201_and_stores_trimmed(admin_client: TestClient, db) -> None:
    r = admin_client.post("/api/steering", json={"note": f"  {NOTE}  "})
    assert r.status_code == 201
    body = r.json()
    assert body["note"] == NOTE  # trimmed before storage
    uuid.UUID(body["id"])  # valid UUID
    assert body["created_at"]
    rows = db.scalars(select(SteeringNote)).all()
    assert [row.note for row in rows] == [NOTE]
 def test_list_notes_empty(admin_client: TestClient) -> None:
    r = admin_client.get("/api/steering")
    assert r.status_code == 200
    assert r.json() == {"notes": []}
 def test_list_notes_newest_first(admin_client: TestClient, db) -> None:
    base = datetime.now(UTC)
    db.add_all(
        [
            SteeringNote(note="oldest", created_at=base),
            SteeringNote(note="newest", created_at=base + timedelta(hours=2)),
            SteeringNote(note="middle", created_at=base + timedelta(hours=1)),
        ]
    )
    db.commit()
    r = admin_client.get("/api/steering")
    assert r.status_code == 200
    body = r.json()
    assert [n["note"] for n in body["notes"]] == ["newest", "middle", "oldest"]
    for n in body["notes"]:
        assert set(n) == {"id", "note", "created_at"}
        uuid.UUID(n["id"])
 def test_delete_note_returns_204_and_removes(admin_client: TestClient, db) -> None:
    created = admin_client.post("/api/steering", json={"note": NOTE}).json()
    assert admin_client.delete(f"/api/steering/{created['id']}").status_code == 204
    assert admin_client.get("/api/steering").json() == {"notes": []}
    assert db.scalars(select(SteeringNote)).all() == []
 def test_delete_unknown_note_returns_404(admin_client: TestClient) -> None:
    r = admin_client.delete(f"/api/steering/{uuid.uuid4()}")
    assert r.status_code == 404
    assert "not found" in r.json()["detail"]
 def test_delete_invalid_id_returns_422(admin_client: TestClient) -> None:
    assert admin_client.delete("/api/steering/not-a-uuid").status_code == 422
 def test_create_rejects_empty_and_blank_notes(admin_client: TestClient) -> None:
    assert admin_client.post("/api/steering", json={"note": ""}).status_code == 422
    assert admin_client.post("/api/steering", json={"note": " \t\n "}).status_code == 422
    assert admin_client.get("/api/steering").json() == {"notes": []}
 def test_create_enforces_2000_char_limit(admin_client: TestClient) -> None:
    assert admin_client.post("/api/steering", json={"note": "x" * 2001}).status_code == 422
    r = admin_client.post("/api/steering", json={"note": "x" * 2000})
    assert r.status_code == 201
    assert len(r.json()["note"]) == 2000
 # ---------- chat turn: note reaches the system prompt ----------
 def test_chat_turn_high_mode_receives_note_in_system_prompt(
    admin_client: TestClient, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
 ) -> None:
    admin_client.post("/api/steering", json={"note": NOTE})
    caplog.set_level(logging.INFO, logger="app.chat")
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _, _, frames = _stream_chat(admin_client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    assert frames[-1]["type"] == "done"
    assert frames[-1]["deflected"] is False
    # The LLM received the HIGH prompt with the <tuning> section.
    (system, user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
    assert user["content"] == QUESTION
    assert "<relevance>HIGH</relevance>" in system["content"]
    assert "<tuning>" in system["content"]
    assert f"1. {NOTE}" in system["content"]
    assert "<documents>" in system["content"]
    # The section sits between the relevance marker and the documents.
    assert (
        system["content"].index("<relevance>HIGH</relevance>")
        < system["content"].index("<tuning>")
        < system["content"].index("</tuning>")
        < system["content"].index("<documents>")
    )
    # The per-turn log line records tuning=N (PLAN §9).
    lines = _turn_log_lines(caplog)
    assert lines and "tuning=1" in lines[-1]
 def test_chat_turn_low_mode_receives_note_in_system_prompt(
    admin_client: TestClient, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
 ) -> None:
    admin_client.post("/api/steering", json={"note": NOTE})
    caplog.set_level(logging.INFO, logger="app.chat")
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _, _, frames = _stream_chat(admin_client, OFF_TOPIC)
    finally:
        fastapi_app.dependency_overrides.clear()
    assert frames[-1]["deflected"] is True
    (system, _user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
    assert "<relevance>LOW</relevance>" in system["content"]
    assert "DEFLECT_MODE" in system["content"]
    assert "<tuning>" in system["content"]
    assert f"1. {NOTE}" in system["content"]
    lines = _turn_log_lines(caplog)
    assert lines and "tuning=1" in lines[-1]
 def test_chat_turn_without_notes_has_no_tuning_section(
    admin_client: TestClient, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
 ) -> None:
    caplog.set_level(logging.INFO, logger="app.chat")
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _stream_chat(admin_client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    (system, _user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
    assert "<tuning>" not in system["content"]
    lines = _turn_log_lines(caplog)
    assert lines and "tuning=0" in lines[-1]
 def test_chat_turn_numbers_notes_oldest_first(admin_client: TestClient, db, seeded_kb) -> None:
    base = datetime.now(UTC)
    db.add_all(
        [
            SteeringNote(note="older note", created_at=base),
            SteeringNote(note="newer note", created_at=base + timedelta(hours=1)),
        ]
    )
    db.commit()
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _stream_chat(admin_client, QUESTION)
    finally:
        fastapi_app.dependency_overrides.clear()
    (system, _user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
    assert "<tuning>" in system["content"]
    assert "1. older note" in system["content"]
    assert "2. newer note" in system["content"]
    assert system["content"].index("1. older note") < system["content"].index("2. newer note")
 def test_multiple_turns_keep_reading_notes(admin_client: TestClient, seeded_kb) -> None:
    """The note steers EVERY subsequent turn, not just the next one."""
    admin_client.post("/api/steering", json={"note": NOTE})
    fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
    try:
        _stream_chat(admin_client, QUESTION)
        _stream_chat(admin_client, QUESTION)
        assert len(seeded_kb.seen_messages) == 2
        for messages in seeded_kb.seen_messages:
            assert f"1. {NOTE}" in messages[0]["content"]
        # Delete → the following turn is clean again.
        note_id = admin_client.get("/api/steering").json()["notes"][0]["id"]
        assert admin_client.delete(f"/api/steering/{note_id}").status_code == 204
        _stream_chat(admin_client, QUESTION)
        assert len(seeded_kb.seen_messages) == 3
        assert "<tuning>" not in seeded_kb.seen_messages[-1][0]["content"]
    finally:
        fastapi_app.dependency_overrides.clear()
@@ -0,0 +1,153 @@
 """Unit tests: single-admin auth (phase 16; A10 revised).
 Covers the config gate (fail-loud, including via ``create_app``), the
 constant-time password check, the ``require_admin`` dependency, the
 whoami payload shape, and the sign_in/sign_out session semantics.
 """
 from __future__ import annotations
 import pytest
 from fastapi import HTTPException
 from starlette.middleware.sessions import Session
 from starlette.requests import Request
 import app.main as main_mod
 from app.api.auth import whoami
 from app.config import Settings
 from app.core.auth import (
    check_password,
    ensure_admin_configured,
    require_admin,
    sign_in,
    sign_out,
 )
 from app.schemas import WhoamiResponse
 def _settings(**kwargs: object) -> Settings:
    return Settings(_env_file=None, **kwargs)  # pyright: ignore[reportCallIssue]
 # ---------- ensure_admin_configured (fail-loud) ----------
 def test_configured_passes() -> None:
    ensure_admin_configured(_settings(admin_password="pw", session_secret="s"))
@pytest.mark.parametrize(
    ("password", "secret", "named"),
    [
        ("", "s", "BOR_ADMIN_PASSWORD"),
        ("pw", "", "BOR_SESSION_SECRET"),
        ("", "", "BOR_ADMIN_PASSWORD"),
        ("", "", "BOR_SESSION_SECRET"),
    ],
 )
 def test_missing_vars_raise_naming_them(password: str, secret: str, named: str) -> None:
    with pytest.raises(RuntimeError) as exc:
        ensure_admin_configured(_settings(admin_password=password, session_secret=secret))
    assert named in str(exc.value)
    # Both vars are named when both are missing.
    if not password and not secret:
        assert "BOR_ADMIN_PASSWORD" in str(exc.value)
        assert "BOR_SESSION_SECRET" in str(exc.value)
 def test_whitespace_only_counts_as_missing() -> None:
    with pytest.raises(RuntimeError, match="BOR_ADMIN_PASSWORD"):
        ensure_admin_configured(_settings(admin_password="   ", session_secret="s"))
 def test_create_app_raises_when_admin_password_missing(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr(main_mod.settings, "admin_password", "")
    with pytest.raises(RuntimeError, match="BOR_ADMIN_PASSWORD"):
        main_mod.create_app()
 def test_create_app_raises_when_session_secret_missing(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr(main_mod.settings, "session_secret", "")
    with pytest.raises(RuntimeError, match="BOR_SESSION_SECRET"):
        main_mod.create_app()
 def test_create_app_boots_when_configured(monkeypatch: pytest.MonkeyPatch) -> None:
    # conftest set both env vars before app.main imported — the factory
    # with valid config returns an app (and no static-dir warning fires
    # here: the frontend dir exists in the repo).
    monkeypatch.setattr(main_mod.settings, "admin_password", "pw")
    monkeypatch.setattr(main_mod.settings, "session_secret", "s")
    app2 = main_mod.create_app()
    assert app2 is not None
 # ---------- check_password (constant-time, one generic result) ----------
 def test_check_password_match() -> None:
    assert check_password("hunter2", "hunter2") is True
 def test_check_password_mismatch() -> None:
    assert check_password("hunter2", "hunter3") is False
 def test_check_password_empty_candidate() -> None:
    assert check_password("", "hunter2") is False
 def test_check_password_unicode() -> None:
    assert check_password("pässwörd", "pässwörd") is True
    assert check_password("pässwörd", "pässwörX") is False
 # ---------- require_admin (dependency: 403 for anonymous) ----------
 def _request_with_session(**session: object) -> Request:
    # request.session is a scope-backed property (SessionMiddleware puts
    # the Session into the scope) — build the scope the same way.
    return Request({"type": "http", "session": Session(dict(session))})
 def test_require_admin_passes_for_admin_session() -> None:
    require_admin(_request_with_session(admin=True))  # no exception
@pytest.mark.parametrize("session", [{}, {"admin": False}, {"admin": None}])
 def test_require_admin_403s_anonymous(session: dict) -> None:
    with pytest.raises(HTTPException) as exc:
        require_admin(_request_with_session(**session))
    assert exc.value.status_code == 403
    assert exc.value.detail == "admin only"
 # ---------- whoami payload shape ----------
 def test_whoami_anonymous_payload() -> None:
    body = whoami(_request_with_session())
    assert body == WhoamiResponse(authenticated=False, role="anonymous")
    assert set(body.model_dump()) == {"authenticated", "role"}
 def test_whoami_admin_payload() -> None:
    body = whoami(_request_with_session(admin=True))
    assert body == WhoamiResponse(authenticated=True, role="admin")
 # ---------- sign_in / sign_out session semantics ----------
 def test_sign_in_marks_session_admin_and_modified() -> None:
    session = Session({})
    sign_in(session)
    assert session["admin"] is True
    assert session.modified is True  # the middleware will persist the cookie
 def test_sign_out_clears_session() -> None:
    session = Session({"admin": True, "stray": "x"})
    sign_out(session)
    assert dict(session) == {}
    assert "admin" not in session
@@ -20,6 +20,7 @@ from app.api import chat as chat_api
 from app.config import Settings
 from app.main import app as fastapi_app
 from app.models import Document, QueryLog
 from app.rag.llm import StreamPiece
 from app.rag.retriever import RetrievedChunk, weak_hit_titles
 from app.rag.suggestions import MAX_SUGGESTIONS, derive_suggestions
@@ -45,13 +46,19 @@ def _doc(title: str, content: str) -> Document:
    )
-def _chunk(doc: Document, score: float) -> RetrievedChunk:
+def _chunk(
    doc: Document, score: float, cosine: float | None = None, fts_hit: bool = False
 ) -> RetrievedChunk:
    """Fake candidate: *score* is the fused rank score; *cosine* (defaults to
    *score*) is the vector-similarity gate input."""
    return RetrievedChunk(
        chunk_id=uuid.uuid4(),
        position=0,
        content=doc.content[:32],
        score=score,
        document=doc,
        cosine=score if cosine is None else cosine,
        fts_hit=fts_hit,
    )
@@ -89,6 +96,68 @@ def test_gate_is_env_tunable_via_settings() -> None:
    assert chat_api.plan_turn(hits, _settings(threshold=0.25)).deflected is False
 # ---------- hybrid gate matrix (A8, revised: cosine AND fts) ----------
 def test_gate_weak_cosine_with_fts_hit_still_answers() -> None:
    """cosine < threshold but a lexical hit ⇒ HIGH — the FTS-OR branch.
    This is the name-your-tool case: "kafkabridge" grounds despite weak
    vector overlap."""
    doc = _doc("Static DNS", "DNS_DOC_CONTENT")
    plan = chat_api.plan_turn(
        [_chunk(doc, 0.02, cosine=0.10, fts_hit=True)], _settings(threshold=0.30)
    )
    assert plan.deflected is False
    assert plan.top_score == pytest.approx(0.10)  # gate input is the cosine
    assert plan.fts_hits == 1
    assert "DNS_DOC_CONTENT" in plan.system_prompt
    assert plan.suggestions == []
 def test_gate_weak_cosine_zero_fts_deflects() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_CONTENT")
    plan = chat_api.plan_turn([_chunk(doc, 0.02, cosine=0.10)], _settings(threshold=0.30))
    assert plan.deflected is True
    assert plan.top_score == pytest.approx(0.10)
    assert plan.fts_hits == 0
 def test_gate_strong_cosine_without_fts_answers() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_CONTENT")
    plan = chat_api.plan_turn([_chunk(doc, 0.90, cosine=0.90)], _settings(threshold=0.30))
    assert plan.deflected is False
    assert plan.fts_hits == 0
 def test_gate_fts_hits_counts_all_lexical_candidates() -> None:
    a = _doc("Alpha", "ALPHA_CONTENT")
    b = _doc("Beta", "BETA_CONTENT")
    chunks = [
        _chunk(a, 0.03, cosine=0.05, fts_hit=True),
        _chunk(a, 0.02, cosine=0.04, fts_hit=True),  # same doc, second chunk
        _chunk(b, 0.01, cosine=0.03),
    ]
    plan = chat_api.plan_turn(chunks, _settings(threshold=0.30))
    assert plan.deflected is False
    assert plan.fts_hits == 2  # per chunk, not per doc
 def test_gate_lexical_only_chunk_does_not_inflate_cosine() -> None:
    """top_score stays the best *vector* cosine even when a lexical-only
    chunk (cosine 0.0 by construction) carries the highest fused score."""
    a = _doc("Alpha", "ALPHA_CONTENT")
    b = _doc("Beta", "BETA_CONTENT")
    chunks = [
        _chunk(a, 0.50, cosine=0.55),  # vector rank 1
        _chunk(b, 0.90, cosine=0.0, fts_hit=True),  # lexical rank 1 wins the ranking
    ]
    plan = chat_api.plan_turn(chunks, _settings(threshold=0.30))
    assert plan.top_score == pytest.approx(0.55)
    assert plan.deflected is False  # 0.55 >= 0.30 anyway
    # ranking follows the fused score: Beta's doc is the top source
    assert plan.docs[0].title == "Beta"
 def test_gate_zero_chunks_deflects_with_fallback_chips() -> None:
    plan = chat_api.plan_turn([], _settings())
    assert plan.deflected is True
@@ -208,11 +277,22 @@ class _CannedLLM:
    async def chat_stream(self, messages: list[dict[str, str]]):
        self.seen.append(messages)
        for i in range(0, len(self.answer), 12):
-            yield self.answer[i : i + 12]
+            yield StreamPiece("content", self.answer[i : i + 12])
 class _FakeSteeringResult:
    """Empty steering-note result (no stored notes in these unit tests)."""
    def all(self) -> list[Any]:
        return []
 class _FakeSession:
-    """Stands in for the DB session: records the QueryLog row it is given."""
+    """Stands in for the DB session: records the QueryLog row it is given.
    ``scalars`` always yields no steering notes (phase 15) so the chat
    turn's ``load_steering_notes`` call stays a no-op here.
    """
    def __init__(self) -> None:
        self.added: list[Any] = []
@@ -224,6 +304,9 @@ class _FakeSession:
    def commit(self) -> None:
        self.commits += 1
    def scalars(self, _stmt: Any) -> _FakeSteeringResult:
        return _FakeSteeringResult()
@pytest.fixture()
 def gate_env(monkeypatch: pytest.MonkeyPatch) -> Iterator[tuple[_FakeSession, _CannedLLM]]:
@@ -233,6 +316,13 @@ def gate_env(monkeypatch: pytest.MonkeyPatch) -> Iterator[tuple[_FakeSession, _C
    llm = _CannedLLM()
    monkeypatch.setitem(fastapi_app.dependency_overrides, chat_api.get_db, lambda: session)
    monkeypatch.setitem(fastapi_app.dependency_overrides, chat_api.get_llm, lambda: llm)
    # These tests assert against a specific gate threshold; keep it stable
    # regardless of the production default (0.62) or any .env.
    monkeypatch.setattr(
        chat_api,
        "get_settings",
        lambda: Settings(_env_file=None, relevance_threshold=0.30),  # pyright: ignore[reportCallIssue]
    )
    yield session, llm
    fastapi_app.dependency_overrides.clear()
@@ -254,7 +344,7 @@ def _ask(client: TestClient, message: str) -> list[dict[str, Any]]:
 def _fake_retriever(chunks: list[RetrievedChunk]) -> Any:
-    def retrieve(_db: Any, _vec: list[float]) -> list[RetrievedChunk]:
+    def retrieve(_db: Any, _question: str, _vec: list[float]) -> list[RetrievedChunk]:
        return chunks
    return retrieve
@@ -0,0 +1,252 @@
 """Unit: the chat-persistence contract in the static frontend (phase 14).
 The browser behavior itself is E2E-covered (tests/e2e/test_chat_persistence.py);
 here we pin the localStorage persistence markers in app.js/index.html/
 styles.css so a silent regression (key rename, dropped try/catch, missing
 restore, New chat control lost) is caught without a browser.
 Pinned design (PLAN §7.4 note / phase 14):
 * versioned key ``bor.chat.v1`` → ``{v: 1, messages: [...]}``, raw text only;
 * save points: user message on send, brain message on ``done``;
 * every ``localStorage`` access wrapped in try/catch (failure-safe);
 * size budget ~700k chars, oldest dropped first;
 * ``#new-chat-btn`` in the chat header (chat page only), ≥44px, ghost pill.
 """
 from __future__ import annotations
 import re
 from pathlib import Path
 FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
 APP_JS = FRONTEND / "assets" / "app.js"
 INDEX_HTML = FRONTEND / "index.html"
 SOURCES_HTML = FRONTEND / "sources.html"
 DOCUMENT_HTML = FRONTEND / "document.html"
 STYLES_CSS = FRONTEND / "assets" / "styles.css"
 def _js() -> str:
    return APP_JS.read_text(encoding="utf-8")
 def _css() -> str:
    return STYLES_CSS.read_text(encoding="utf-8")
 def _index() -> str:
    return INDEX_HTML.read_text(encoding="utf-8")
 def test_versioned_storage_key_and_v1_payload() -> None:
    """`bor.chat.v1` (versioned — a format bump is a clean start) with the
    {v, messages} payload shape (A11: raw localStorage JSON, no library)."""
    js = _js()
    assert 'const STORAGE_KEY = "bor.chat.v1"' in js
    assert "export const STORAGE_VERSION = 1" in js
    # The payload written to the key is always {v: STORAGE_VERSION, messages}
    # (two write paths: saveConversation and the trimToBudget size probe).
    assert js.count("v: STORAGE_VERSION, messages") >= 2
    # Restore validates the version before trusting anything.
    assert "data.v !== STORAGE_VERSION" in js
 def test_storage_size_budget_drops_oldest_first() -> None:
    """~700k-char serialized budget (far under the ~5MB quota); the loop
    drops messages from the FRONT (oldest) until the state fits."""
    js = _js()
    assert "export const STORAGE_BUDGET_CHARS = 700_000" in js
    assert "out.length <= 1" in js, "never drop the last remaining message"
    assert "out = out.slice(1)" in js, "oldest-first drop (slice(1), not pop)"
    assert "STORAGE_BUDGET_CHARS" in js
 def test_every_storage_access_is_failure_safe() -> None:
    """AC4: every localStorage access (getItem/setItem/removeItem) must be
    inside a try/ that is closer than the enclosing function boundary —
    private mode or quota exhaustion must never throw into the UI."""
    js = _js()
    accesses = list(re.finditer(r"localStorage\.(?:getItem|setItem|removeItem)", js))
    assert len(accesses) == 3, f"expected exactly 3 localStorage accesses, got {len(accesses)}"
    for m in accesses:
        try_idx = js.rfind("try {", 0, m.start())
        fn_idx = js.rfind("function ", 0, m.start())
        assert try_idx != -1, f"no try before {m.group(0)!r}"
        assert try_idx > fn_idx, (
            f"{m.group(0)!r} is not inside its function's try block "
            f"(function boundary at {fn_idx} is after try at {try_idx})"
        )
    # Each access has its own catch that degrades silently.
    assert js.count("} catch {") >= len(accesses)
 def test_raw_text_only_stored_and_re_rendered_on_restore() -> None:
    """The value is raw text (re-rendered through the escape-first markdown
    on restore) — no HTML is ever stored. Restore re-applies the full
    brain-message chrome: is-deflected styling, maybe-try chips, sources."""
    js = _js()
    # Phase 18: restore landings are forced ("auto" + force) one-shot
    # scrollReveal calls — the only forced scrolls in the app.
    assert 'addMessage("user", renderMarkdown(m.text), "auto", true)' in js
    assert 'addMessage("brain", renderMarkdown(m.text), "auto", true)' in js
    assert "wrap.classList.add(\"is-deflected\")" in js
    assert "appendMaybeTry(wrap, m.suggestions)" in js
    assert "appendSources(wrap, m.sources)" in js
    # Restore runs on load (module scope, after the handlers are wired).
    assert "restoreConversation();" in js
    # Corrupt/legacy payloads degrade to a clean start, never a crash.
    assert "Array.isArray(data.messages)" in js
 def test_save_points_user_on_send_and_brain_on_done() -> None:
    """Save points: the user message is stored the moment it is sent (BEFORE
    the fetch — a failed turn keeps the question); the brain message is
    stored on `done` with the done metadata (sources/deflected/suggestions)."""
    js = _js()
    user_push = js.find('conversation.push({ who: "user", text })')
    assert user_push != -1
    assert user_push < js.find('fetch("/api/chat"'), (
        "the user message must be saved before the turn starts"
    )
    # Brain save point is wired into the done handler with full metadata
    # (phase 17: the persisted text is finalText — the empty-answer
    # fallback substitution — and the optional thinking field rides along
    # in the same meta object).
    done_idx = js.find('ev.type === "done"')
    assert done_idx != -1
    done_block = js[done_idx : done_idx + 1300]
    assert "rememberBrainTurn(finalText || acc" in done_block
    assert "thinking: thinkingAcc || undefined" in done_block
    assert "deflected: !!ev.deflected" in done_block
    assert "sources: ev.sources" in done_block
    assert "suggestions: ev.suggestions" in done_block
    # rememberBrainTurn stores raw text and saves immediately.
    assert "text: rawText ||" in js
    body = js[js.find("function rememberBrainTurn") :]
    assert "saveConversation()" in body[: body.find("\n}\n") + 3]
 def test_new_chat_clears_key_and_ui() -> None:
    """New chat: clears the stored key, the rendered list, restores the
    empty state, and reuses the #send-status live region for the
    confirmation. A live turn is never hijacked."""
    js = _js()
    fn_start = js.find("function startNewChat")
    assert fn_start != -1
    body = js[fn_start : js.find("\n}\n", fn_start)]
    assert "clearStoredConversation()" in body
    assert 'querySelectorAll(".msg")' in body
    assert "emptyState.hidden = false" in body
    assert "setUiState(UI_STATE.idle)" in body
    assert "sendStatus.textContent" in body, "confirmation via the live region"
    assert "UI_STATE.thinking" in body and "UI_STATE.streaming" in body, (
        "new chat must be ignored while a turn is in flight"
    )
    assert "removeItem(STORAGE_KEY)" in js
 def test_new_chat_button_in_chat_header_only() -> None:
    """#new-chat-btn lives in the chat header (index.html) as a real
    type=button with an accessible name — and nowhere else (A10: chat-page
    only control)."""
    html = _index()
    btn = re.search(r'<button[^>]*id="new-chat-btn"[^>]*>', html)
    assert btn, "index.html must contain #new-chat-btn"
    tag = btn.group(0)
    assert 'type="button"' in tag
    assert 'aria-label="New chat"' in tag
    nav_idx = html.find('<nav class="app-nav"')
    assert nav_idx != -1 and btn.start() > nav_idx, (
        "the button belongs after the nav, inside .header-inner"
    )
    main_idx = html.find('main id="main"')
    assert main_idx != -1 and btn.start() < main_idx, "the button belongs in the header"
    assert 'id="new-chat-btn"' not in SOURCES_HTML.read_text(encoding="utf-8")
    assert 'id="new-chat-btn"' not in DOCUMENT_HTML.read_text(encoding="utf-8")
 def test_new_chat_button_style_contract() -> None:
    """Ghost pill like a nav link: Phase-08 tokens, ≥44px target, hover like
    .nav-link, focus-visible via the global rule; icon-only on phones with
    the label hidden (aria-label keeps the accessible name)."""
    css = _css()
    block = re.search(r"\.new-chat-btn \{([\s\S]*?)\n\}", css)
    assert block, "styles.css must style .new-chat-btn"
    body = block.group(1)
    assert "min-height: 44px" in body
    assert "border-radius: 999px" in body
    assert "var(--line)" in body, "ghost: 1px line border, transparent background"
    assert "background: transparent" in body
    assert "var(--ink-soft)" in body
    hover = re.search(r"\.new-chat-btn:hover \{([\s\S]*?)\n\}", css)
    assert hover and "--brand-soft" in hover.group(1) and "--brand-ink" in hover.group(1), (
        "hover must match the nav-link brand pair"
    )
    # Mobile (≤640px): label hidden, icon shown — the pill stays ≥44px via
    # min-height and never breaks the fixed-height header bar.
    mobile = re.search(r"@media \(max-width: 640px\) \{([\s\S]*?)\n\}", css)
    assert mobile, "mobile media query missing"
    assert ".new-chat-label { display: none; }" in mobile.group(1)
    assert ".new-chat-btn svg { display: block; }" in mobile.group(1)
 def test_brain_turn_persists_optional_thinking_field() -> None:
    """Phase 17: the done save point carries `thinking: thinkingAcc ||
    undefined` — `undefined` drops the key from the JSON, so turns without
    thinking persist byte-identical to before (no version bump). A
    thinking-without-answer turn (reasoning exhausts max_tokens) renders
    + persists the shared empty-answer fallback: what the user saw is what
    is stored."""
    js = _js()
    done_idx = js.find('ev.type === "done"')
    error_idx = js.find('ev.type === "error"')
    assert -1 < done_idx < error_idx, "done branch missing from the turn handler"
    branch = js[done_idx:error_idx]
    assert "thinking: thinkingAcc || undefined" in branch
    assert (
        'const finalText = acc || (sawThinking ? EMPTY_ANSWER_FALLBACK : "")'
        in branch
    )
    assert "renderMarkdown(finalText)" in branch, (
        "the substituted fallback must render into the bubble"
    )
 def test_restore_renders_collapsed_thinking_block() -> None:
    """Phase 17: a stored brain message carrying `thinking` re-renders the
    block COLLAPSED above its bubble (escape-first markdown, as everywhere
    else in the persistence contract); messages without the field render
    exactly as before — no block."""
    js = _js()
    fn_start = js.find("function renderStoredMessage")
    assert fn_start != -1
    body = js[fn_start : js.find("\n}\n", fn_start)]
    assert "if (m.thinking)" in body
    assert "ensureThinkingBlock(wrap)" in body
    assert "block.open = false" in body, "restored blocks must be collapsed"
    assert "renderMarkdown(m.thinking)" in body
 def test_thinking_block_css_uses_phase08_tokens() -> None:
    """Phase 17 styling (Phase-08 tokens, WCAG AA): the block frame, the
    ≥44px summary control (brand-ink ≈8.7:1 on surface) and the scrollable
    scratchpad (ink-soft ≈6.9:1 on surface, 320px cap)."""
    css = _css()
    block = re.search(r"details\.thinking \{([\s\S]*?)\n\}", css)
    assert block, "styles.css must style details.thinking"
    body = block.group(1)
    assert "var(--surface)" in body
    assert "var(--line)" in body
    assert "var(--brand-soft)" in body
    assert "var(--radius-sm)" in body
    summary = re.search(r"details\.thinking summary \{([\s\S]*?)\n\}", css)
    assert summary, "the summary must be a styled focusable control"
    sbody = summary.group(1)
    assert "min-height: 44px" in sbody
    assert "var(--brand-ink)" in sbody
    assert "cursor: pointer" in sbody
    text = re.search(r"details\.thinking \.thinking-text \{([\s\S]*?)\n\}", css)
    assert text, "the .thinking-text scroll area must be styled"
    tbody = text.group(1)
    assert "var(--ink-soft)" in tbody
    assert "max-height: 320px" in tbody
    assert "overflow-y: auto" in tbody
@@ -1,11 +1,25 @@
-"""Unit tests: markdown-aware chunker (PLAN §5 policy)."""
+"""Unit tests: format-aware chunker (PLAN §5 policy, A9 formats).
 The markdown policy tests are the original contract (md output stays
 unchanged); the per-format tests cover the phase-09 dispatcher
 (yaml/yml, json, py, txt) and the 1200-char hard cap for every format.
 """
 from __future__ import annotations
 from itertools import pairwise
 import pytest
-from app.rag.chunker import HARD_MAX_CHARS, chunk_markdown, extract_title
+from app.rag.chunker import (
    HARD_MAX_CHARS,
    chunk_document,
    chunk_json,
    chunk_markdown,
    chunk_python,
    chunk_text,
    chunk_yaml,
    extract_title,
 )
 ANCHOR = "## Big"
 ANCHOR_PREFIX = f"{ANCHOR}\n\n"
@@ -157,3 +171,213 @@ def test_extract_title_prefers_h1() -> None:
    assert extract_title("## not a title\n\nbody") == ""
    assert extract_title("## sub only", fallback="stem") == "stem"
    assert extract_title("", fallback="fallback") == "fallback"
 # ---------------------------------------------------------------------------
 # Format dispatcher (chunk_document) — A9 multi-format ingestion
 # ---------------------------------------------------------------------------
 def test_dispatch_by_lowercased_suffix() -> None:
    md = "# T\n\n## A\n\nbody\n"
    assert chunk_document(md, "notes/Doc.MD") == chunk_markdown(md)
    assert chunk_document(md, "notes/doc.MARKDOWN") == chunk_markdown(md)
    assert chunk_document("p1\n\np2\n", "x.TXT") == chunk_text("p1\n\np2\n")
    assert chunk_document("a: 1\n", "x.YAML") == chunk_yaml("a: 1\n")
    assert chunk_document("a: 1\n", "x.Yml") == chunk_yaml("a: 1\n")
    assert chunk_document('{"a": 1}', "x.Json") == chunk_json('{"a": 1}')
    assert chunk_document("def f(): pass\n", "x.PY") == chunk_python("def f(): pass\n")
 def test_dispatch_unknown_suffix_falls_back_to_paragraphs() -> None:
    assert chunk_document("hello\n\nworld", "data.csv") == ["hello\nworld"]
 def test_dispatch_ignores_directory_part_of_path() -> None:
    assert chunk_document("def f(): pass\n", "a/b/c/script.py") == chunk_python("def f(): pass\n")
 # ---------------------------------------------------------------------------
 # yaml / yml
 # ---------------------------------------------------------------------------
 def test_yaml_splits_on_top_level_keys_and_keeps_key_anchors() -> None:
    doc = (
        "# leading comment\n"
        "services:\n"
        "  gitlab:\n"
        "    image: gitlab/gitlab-ce\n"
        "  prometheus:\n"
        "    image: prom/prometheus\n"
        "volumes:\n"
        "  gitlab-data:\n"
    )
    chunks = chunk_yaml(doc)
    joined = "\n".join(chunks)
    for key in ("services:", "volumes:"):
        assert key in joined
    # Indented keys are NOT block starts — they stay inside their parent block.
    assert not any(c.startswith("  gitlab:") for c in chunks)
    # The leading comment stays with the first block (preamble).
    assert chunks[0].startswith("# leading comment")
    assert "gitlab/gitlab-ce" in joined and "prom/prometheus" in joined
 def test_yaml_document_separators_start_new_blocks() -> None:
    a = "site_a: " + "a" * 500 + "\n"
    b = "site_b: " + "b" * 500 + "\n"
    chunks = chunk_yaml(a + "---\n" + b, target_chars=600, overlap_chars=0)
    # Each site is long enough to force its own chunk; the separator must not
    # glue them into one over-budget chunk.
    assert len(chunks) >= 2
    assert all(len(c) <= 600 for c in chunks)
    assert not any("site_a" in c and "site_b" in c for c in chunks)
 def test_yaml_oversized_key_block_is_split_under_hard_cap() -> None:
    doc = "big_list:\n" + ("  - " + "x" * 60 + "\n") * 60  # one ~3800-char block
    chunks = chunk_yaml(doc)
    assert len(chunks) >= 2
    assert all(len(c) <= HARD_MAX_CHARS for c in chunks)
    # Overlap re-prints (≤50 chars per split), so only a little content is
    # re-stated — the bulk of the block must survive.
    assert sum(len(c) for c in chunks) >= len(doc) - 300
 def test_yaml_empty_content() -> None:
    assert chunk_yaml("") == []
    assert chunk_yaml("\n\n  \n") == []
 # ---------------------------------------------------------------------------
 # json
 # ---------------------------------------------------------------------------
 def test_json_splits_on_top_level_keys_pretty_printed() -> None:
    doc = '{"hosts": {"kafkabridge": "10.0.3.7"}, "count": 3}'
    chunks = chunk_json(doc, target_chars=45, overlap_chars=0)  # force 1 chunk/block
    assert len(chunks) == 2
    first, second = chunks
    assert '"hosts"' in first and "kafkabridge" in first
    assert '"count"' in second
    # Pretty-printed (indent=2), not the compact input form.
    assert '"kafkabridge": "10.0.3.7"' in first
    assert not any('{"hosts"' in c for c in chunks)
 def test_json_each_key_block_is_self_contained() -> None:
    doc = '{"a": "x", "b": "y"}'
    chunks = chunk_json(doc, target_chars=13, overlap_chars=0)  # force 1 chunk/block
    assert [c for c in chunks if '"a"' in c] and [c for c in chunks if '"b"' in c]
    assert not any('"a"' in c and '"b"' in c for c in chunks)
 def test_json_oversized_value_falls_under_hard_cap() -> None:
    doc = '{"blob": "' + "z" * 4000 + '"}'
    chunks = chunk_json(doc)
    assert len(chunks) >= 2
    assert all(len(c) <= HARD_MAX_CHARS for c in chunks)
    assert "".join(chunks).count("z") >= 4000
 def test_json_top_level_list_is_one_pretty_block() -> None:
    chunks = chunk_json("[1, 2, 3]")
    assert chunks == ["[\n  1,\n  2,\n  3\n]"]
 def test_json_unparseable_falls_back_to_paragraph_packing() -> None:
    doc = "{broken json\n\nsecond paragraph here\n"
    assert chunk_json(doc) == chunk_text(doc)
    assert chunk_json("not json at all") == chunk_text("not json at all")
 # ---------------------------------------------------------------------------
 # python
 # ---------------------------------------------------------------------------
 def test_python_splits_on_top_level_defs_and_classes() -> None:
    doc = (
        '"""Module doc."""\n'
        "import asyncio\n"
        "\n"
        "CONST = 1\n"
        "\n"
        "def alpha():\n"
        "    return 1\n"
        "\n"
        "class Beta:\n"
        "    def run(self):\n"
        "        return 2\n"
    )
    chunks = chunk_python(doc, target_chars=60, overlap_chars=0)  # force 1 chunk/block
    assert len(chunks) == 3
    assert chunks[0].startswith('"""Module doc."""')
    assert "CONST = 1" in chunks[0]  # preamble ends at the first def/class
    assert chunks[1].startswith("def alpha")
    assert chunks[2].startswith("class Beta")
    assert "def run" in chunks[2]  # nested def stays inside the class block
 def test_python_decorators_stay_with_their_definition() -> None:
    doc = "@app.get('/x')\ndef handler():\n    return 'x'\n"
    chunks = chunk_python(doc)
    assert chunks[0].startswith("@app.get")
 def test_python_oversized_function_falls_back_to_line_packing() -> None:
    doc = "def big():\n" + "\n".join(f"    val_{i:03d} = {i}  # padding" for i in range(80))
    chunks = chunk_python(doc)
    assert len(chunks) >= 2
    assert all(len(c) <= HARD_MAX_CHARS for c in chunks)
    assert "val_000" in chunks[0]
    assert "val_079" in chunks[-1]
    assert sum(len(c) for c in chunks) >= len(doc) - 100
 def test_python_unparseable_source_falls_back_to_paragraphs() -> None:
    src = "def broken(:\n\nstill text\n"
    assert chunk_python(src) == chunk_text(src)
 # ---------------------------------------------------------------------------
 # txt
 # ---------------------------------------------------------------------------
 def test_txt_paragraph_packing() -> None:
    doc = "alpha\n\nbeta\n\ngamma\n"
    chunks = chunk_text(doc)
    assert chunks == ["alpha\nbeta\ngamma"]  # all three fit the target
 def test_txt_long_doc_packs_with_overlap() -> None:
    doc = "\n\n".join(f"para {i} " + "l" * 300 for i in range(6))
    chunks = chunk_text(doc, target_chars=800, overlap_chars=100)
    assert len(chunks) >= 2
    assert all(len(c) <= 800 for c in chunks)
    assert all(f"para {i}" in "\n".join(chunks) for i in range(6))
 # ---------------------------------------------------------------------------
 # Hard cap across every format (aipi ~1024-token request cap)
 # ---------------------------------------------------------------------------
@pytest.mark.parametrize(
    ("content", "path"),
    [
        ("# T\n\n" + "word " * 1200, "big.md"),
        ("key: " + "v" * 5000 + "\n", "big.yaml"),
        ('{"blob": "' + "z" * 5000 + '"}', "big.json"),
        ("def f():\n" + "    x = 1\n" * 1000, "big.py"),
        ("line of text\n\n" * 800, "big.txt"),
    ],
 )
 def test_hard_cap_holds_for_every_format(content: str, path: str) -> None:
    chunks = chunk_document(content, path)
    assert chunks, "expected at least one chunk"
    for c in chunks:
        assert len(c) <= HARD_MAX_CHARS, f"{path}: {len(c)} chars"
@@ -5,6 +5,7 @@ import json
 from typing import Any
 import pytest
 from pydantic import ValidationError
 from pydantic_settings import SettingsError
 from app.config import Settings
@@ -16,16 +17,32 @@ def _settings(**kwargs: Any) -> Settings:
    return Settings(**kwargs)  # pyright: ignore[reportCallIssue]  (kwarg exists at runtime)
-def test_defaults_match_locked_decisions() -> None:
+def test_defaults_match_locked_decisions(monkeypatch: pytest.MonkeyPatch) -> None:
    # The test process sets BOR_RELEVANCE_THRESHOLD=0.30 for the mock-
    # calibrated in-process suites (see tests/conftest.py) — the *default*
    # under test is the production one.
    monkeypatch.delenv("BOR_RELEVANCE_THRESHOLD", raising=False)
    s = _settings()
    assert s.llm_chat_model == "turbo"
    assert s.llm_embed_model == "embed"
    assert s.embedding_dim == 768
    assert s.llm_base_url.endswith("/v1")
-    assert 0 < s.relevance_threshold < 1
+    # A8 (revised): the honesty gate input is the best cosine, default 0.62.
-    assert s.top_k_chunks >= 1
+    assert s.relevance_threshold == 0.62
    # A7 (revised): hybrid retrieval — cosine top-N ∪ FTS top-N, RRF-fused.
    assert s.hybrid_vector_candidates >= 1
    assert s.hybrid_lexical_candidates >= 1
    assert s.rrf_k >= 1
    assert s.top_n_docs >= 1
    # Owner instruction 2026-08-22: answers may run up to 32 768 tokens.
    assert s.max_output_tokens == 32_768
    # Phase 17: the model's thinking streams by default (kill-switch off).
    assert s.stream_thinking is True
    assert len(s.suggestions) >= 3
    # A9 (revised): the import scope covers the seven A9 formats.
    assert s.import_extension_set == {
        ".md", ".markdown", ".txt", ".yaml", ".yml", ".json", ".py"
    }
 def test_env_override(monkeypatch) -> None:
@@ -36,6 +53,45 @@ def test_env_override(monkeypatch) -> None:
    assert s.llm_chat_model == "juggernaut"
 def test_max_output_tokens_env_override(monkeypatch) -> None:
    monkeypatch.setenv("BOR_MAX_OUTPUT_TOKENS", "1234")
    s = _settings()
    assert s.max_output_tokens == 1234
 def test_stream_thinking_default_true_and_env_parse(monkeypatch: pytest.MonkeyPatch) -> None:
    """Phase 17 kill-switch (``BOR_STREAM_THINKING``): on by default,
    ``0``/``false`` turn the ``thinking`` SSE frames off."""
    assert _settings().stream_thinking is True
    assert _settings(stream_thinking=False).stream_thinking is False
    monkeypatch.setenv("BOR_STREAM_THINKING", "0")
    assert _settings().stream_thinking is False
    monkeypatch.setenv("BOR_STREAM_THINKING", "false")
    assert _settings().stream_thinking is False
    monkeypatch.setenv("BOR_STREAM_THINKING", "1")
    assert _settings().stream_thinking is True
 def test_import_extensions_env_override_is_a_csv_list(monkeypatch) -> None:
    monkeypatch.setenv("BOR_IMPORT_EXTENSIONS", "md,yml")
    s = _settings()
    assert s.import_extension_set == {".md", ".yml"}
 def test_import_extensions_rejects_unknown_format(monkeypatch) -> None:
    """A typo in the CSV fails at startup (loudly), not by silently
    walking zero files."""
    monkeypatch.setenv("BOR_IMPORT_EXTENSIONS", "md,docx")
    with pytest.raises(ValidationError, match="docx"):
        _settings()
 def test_import_extensions_rejects_empty(monkeypatch) -> None:
    monkeypatch.setenv("BOR_IMPORT_EXTENSIONS", "   ")
    with pytest.raises(ValidationError):
        _settings()
 def test_suggestions_default_is_three_plus_real_questions() -> None:
    s = _settings()
    assert len(s.suggestions) >= 3
@@ -0,0 +1,290 @@
 """Unit: document viewer (phase 10).
 Python side:
 * ``doc_format`` — format from the path suffix (incl. ``.markdown`` and the
  no-suffix fallback);
 * the content endpoint's 200/404 mapping — tested WITHOUT a database by
  stubbing the session via FastAPI's dependency override (unknown pairs and
  traversal-style paths map to 404 ``{detail: "document not found"}``;
  known pairs map to the full ``DocContent`` shape).
 Frontend side:
 * the viewer URL builder — its real query-encoding behavior (paths with
  spaces/slashes) executed under node when available, plus source pins that
  run everywhere;
 * the shared-renderer extraction — ``markdown.js`` holds the renderer,
  loaded by BOTH pages via a relative ``<script src>`` before the module
  scripts.
 """
 from __future__ import annotations
 import re
 import shutil
 import subprocess
 from datetime import UTC, datetime
 from pathlib import Path
 import pytest
 from fastapi.testclient import TestClient
 from app.api.docs import doc_format
 from app.db import get_db
 from app.main import create_app
 from app.models import Document
 FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
 APP_JS = FRONTEND / "assets" / "app.js"
 SOURCES_JS = FRONTEND / "assets" / "sources.js"
 DOCUMENT_JS = FRONTEND / "assets" / "document.js"
 MARKDOWN_JS = FRONTEND / "assets" / "markdown.js"
 HAVE_NODE = shutil.which("node") is not None
 def _read(path: Path) -> str:
    return path.read_text(encoding="utf-8")
 # ---------------------------------------------------------------------------
 # doc_format — format-from-suffix (phase 10, PLAN §4)
 # ---------------------------------------------------------------------------
@pytest.mark.parametrize(
    ("path", "expected"),
    [
        ("kubernetes.md", "md"),
        ("notes/sub/deep.markdown", "markdown"),
        ("NOTES/ARCHIVE.MD", "md"),
        ("homelab/container_gitlab/gitlab-compose.yaml", "yaml"),
        ("homelab/networking/static-dns.json", "json"),
        ("homelab/scripts/uptime_probe.py", "py"),
        ("homelab/ssh/ssh_aliases.txt", "txt"),
        ("noext", "text"),  # no suffix → fallback
        ("a/b", "text"),  # no suffix → fallback
    ],
 )
 def test_doc_format_from_suffix(path: str, expected: str) -> None:
    assert doc_format(path) == expected
 # ---------------------------------------------------------------------------
 # Content endpoint mapping — stubbed session, no database required
 # ---------------------------------------------------------------------------
 class _FakeResult:
    def __init__(self, row: object) -> None:
        self._row = row
    def first(self) -> object:
        return self._row
 class _FakeSession:
    def __init__(self, row: object) -> None:
        self._row = row
    def execute(self, _stmt: object) -> _FakeResult:
        return _FakeResult(self._row)
 def _client_with_row(row: object) -> TestClient:
    """Fresh app whose ``get_db`` dependency is a stub returning ``row``
    (``None`` → no matching document row)."""
    app = create_app()
    app.dependency_overrides[get_db] = lambda: _FakeSession(row)
    return TestClient(app)
 def test_content_unknown_pair_maps_to_404() -> None:
    with _client_with_row(None) as client:
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "nope.md"},
        )
    assert r.status_code == 404
    assert r.json() == {"detail": "document not found"}
 def test_content_traversal_style_path_maps_to_404() -> None:
    """``../``-style values are just non-existent rows → 404, never a
    file read (DB-only endpoint, no filesystem access)."""
    with _client_with_row(None) as client:
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "../../etc/passwd"},
        )
    assert r.status_code == 404
    assert r.json() == {"detail": "document not found"}
    assert "root:" not in r.text  # nothing leaked
 def test_content_known_pair_maps_to_doc_content() -> None:
    doc = Document(
        source="Homelab",
        path="notes/deep mark.md",
        full_path="/tmp/deep mark.md",
        title="Deep Mark",
        content="# Deep Mark\n\nbody",
        content_hash="f" * 64,
    )
    doc.indexed_at = datetime(2026, 8, 22, 1, 2, 3, tzinfo=UTC)
    with _client_with_row((doc, 3)) as client:
        r = client.get(
            "/api/documents/content",
            params={"source": "Homelab", "path": "notes/deep mark.md"},
        )
    assert r.status_code == 200
    body = r.json()
    assert set(body) == {"source", "path", "title", "format", "content", "indexed_at", "chunks"}
    assert body["source"] == "Homelab"
    assert body["path"] == "notes/deep mark.md"
    assert body["title"] == "Deep Mark"
    assert body["format"] == "md"
    assert body["content"] == "# Deep Mark\n\nbody"
    assert body["indexed_at"] == "2026-08-22T01:02:03+00:00"
    assert body["chunks"] == 3
 def test_content_requires_both_params() -> None:
    with _client_with_row(None) as client:
        assert client.get("/api/documents/content").status_code == 422
        assert client.get("/api/documents/content", params={"source": "Homelab"}).status_code == 422
 # ---------------------------------------------------------------------------
 # Viewer URL builder — encoded query (spaces/slashes in real paths)
 # ---------------------------------------------------------------------------
 def test_viewer_url_builder_present_in_chat_and_sources() -> None:
    """Both entry points (chat chips, Sources rows) build the same
    encoded viewer URL and open it in a new tab with rel=noopener.
    Phase 13: the chat builder additionally carries ``back=/`` (encoded
    %2F) so the viewer's back button returns to the chat; Sources links
    intentionally omit the param (the viewer's /sources.html default)."""
    for name, js in (("app.js", _read(APP_JS)), ("sources.js", _read(SOURCES_JS))):
        assert '"/document.html?source=" + encodeURIComponent(' in js, name
        assert '"&path=" + encodeURIComponent(' in js, name
    app_js = _read(APP_JS)
    # Chat: 3-arg builder with back defaulting to the chat page.
    assert 'function documentUrl(source, path, back = "/")' in app_js
    assert '"&back=" + encodeURIComponent(back)' in app_js
    assert 'chip.href = documentUrl(s.source, s.path, "/")' in app_js
    assert 'chip.target = "_blank"' in app_js
    assert 'chip.rel = "noopener"' in app_js
    sources_js = _read(SOURCES_JS)
    # Sources: unchanged 2-arg builder — no back param in the URL.
    assert "function documentUrl(source, path)" in sources_js
    assert 'link.className = "doc-link"' in sources_js
    assert "link.href = documentUrl(d.source, d.path)" in sources_js
    assert 'link.target = "_blank"' in sources_js
    assert 'link.rel = "noopener"' in sources_js
    # The full path stays the hover name on the ellipsized cell AND the link.
    assert "pathTd.title = d.path" in sources_js
    assert "link.title = d.path" in sources_js
 def _run_node(script: str) -> str:
    proc = subprocess.run(["node", "-e", script], capture_output=True, text=True, timeout=60)
    assert proc.returncode == 0, f"node failed: {proc.stderr}"
    return proc.stdout
 def _extract_function(js: str, name: str) -> str:
    match = re.search(
        rf"(?:export )?function {name}\(source, path(?:, back = \"/\")?\) \{{.*?\n\}}", js, re.S
    )
    assert match, f"{name}(source, path) not found"
    return match.group(0).replace("export ", "", 1)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
 def test_viewer_url_builder_encodes_spaces_and_slashes() -> None:
    """Behavioral check of the real builder (app.js) under node: slashes
    and spaces in source/path values must come out percent-encoded, and
    the back target is appended + encoded (phase 13)."""
    fn = _extract_function(_read(APP_JS), "documentUrl")
    out = _run_node(
        f"{fn}\n"
        "console.log(documentUrl('Homelab', 'kubernetes.md'));\n"
        "console.log(documentUrl('Homelab', 'notes/my file.yaml'));\n"
        "console.log(documentUrl('H omelab', 'a/b.md'));\n"
        "console.log(documentUrl('Homelab', 'kubernetes.md', '/sources.html'));"
    )
    assert out.splitlines() == [
        "/document.html?source=Homelab&path=kubernetes.md&back=%2F",
        "/document.html?source=Homelab&path=notes%2Fmy%20file.yaml&back=%2F",
        "/document.html?source=H%20omelab&path=a%2Fb.md&back=%2F",
        "/document.html?source=Homelab&path=kubernetes.md&back=%2Fsources.html",
    ]
 # ---------------------------------------------------------------------------
 # Shared renderer extraction (phase 10 step 2)
 # ---------------------------------------------------------------------------
 def test_renderer_extracted_to_shared_markdown_js() -> None:
    """The renderer moved to assets/markdown.js (not duplicated in app.js)
    and BOTH pages load it via a relative <script src> before their module
    scripts — so the globals exist when app.js/document.js run."""
    md = _read(MARKDOWN_JS)
    assert "function renderMarkdown(md)" in md
    assert "function escapeHtml(s)" in md
    app_js = _read(APP_JS)
    assert "function renderMarkdown" not in app_js, "renderer must live in markdown.js"
    assert "function escapeHtml" not in app_js
    for page in ("index.html", "document.html"):
        html = _read(FRONTEND / page)
        assert re.search(r'<script src="assets/markdown\.js"></script>', html), (
            f"{page} must load markdown.js via a relative <script src>"
        )
        assert html.index('src="assets/markdown.js"') < html.index('type="module"'), (
            f"{page}: markdown.js must load before the module script"
        )
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
 def test_markdown_renderer_stays_xss_safe_and_unchanged() -> None:
    """Behavioral check (node) that the extracted renderer still escapes
    first: hostile content never becomes live HTML; basic transforms work."""
    js = _read(MARKDOWN_JS)
    out = _run_node(
        js
        + "\nconsole.log(renderMarkdown('# Title\\n\\n<script>alert(1)</script>"
        + "\\n\\n**bold** and `code`'));"
    )
    html = out.strip()
    assert "<script>" not in html  # never live HTML
    assert "&lt;script&gt;alert(1)&lt;/script&gt;" in html
    assert "<strong>bold</strong>" in html
    assert "<code>code</code>" in html
    assert "<h3>Title</h3>" in html
 def test_viewer_js_rendering_contracts() -> None:
    """document.js: raw formats go in via textContent (never parsed as
    HTML), markdown via the shared renderer, 404 → designed not-found
    state, and (phase 13) the back link resolves the ``back`` param —
    same-origin relative URLs only, /sources.html default, no browser
    history heuristics (both entry points are fresh tabs)."""
    js = _read(DOCUMENT_JS)
    assert "pre.textContent = doc.content" in js  # raw formats: text node
    assert "renderMarkdown(doc.content)" in js  # md/markdown: shared renderer
    assert "showNotFound" in js
    # Phase 13: deterministic back-target resolution, no history heuristics.
    assert "history.length" not in js
    assert "window.history.back" not in js
    assert 'backParam.startsWith("/")' in js  # same-origin relative only…
    assert 'backParam.startsWith("//")' in js  # …and not protocol-relative
    assert 'backLink.href = backTarget' in js  # deterministic anchor navigation
    assert '"/sources.html"' in js  # default target + no-JS fallback value
    assert '"Chat"' in js and '"Sources"' in js  # labels for the two entry points
    assert "encodeURIComponent" in js  # content fetch uses the same encoding
@@ -91,3 +91,101 @@ def test_busy_button_style_tokens() -> None:
    assert re.search(r"\.spinner \{[^}]*width: 16px", css)
    assert "Thinking…" in js
    assert 'sendLabel.textContent' in js
 # ---------- thinking display (phase 17) ----------
 def test_thinking_event_is_a_first_class_turn_branch() -> None:
    """Phase 17: `thinking` SSE frames stream live into the collapsible
    Thinking block — the typing dots make way, the 120s pre-token guard
    clears (the stream is alive), and the text renders through the
    escape-first markdown renderer (XSS-safe). While open, the stream is
    pinned to the bottom of the block."""
    js = _js()
    thinking_idx = js.find('ev.type === "thinking"')
    delta_idx = js.find('ev.type === "delta"')
    assert -1 < thinking_idx < delta_idx, "the turn handler must branch on thinking frames"
    branch = js[thinking_idx:delta_idx]
    assert "thinkingAcc += ev.text" in branch
    assert "sawThinking = true" in branch
    assert "clearTurnTimeout()" in branch, "first thinking frame clears the 120s guard"
    assert "removeTyping()" in branch, "the live block replaces the typing dots"
    assert "ensureThinkingBlock(wrap)" in branch
    assert "renderMarkdown(thinkingAcc)" in branch, "escape-first renderer (XSS-safe)"
    assert "textEl.scrollTop = textEl.scrollHeight" in branch, "bottom-pinned while open"
 def test_thinking_block_helpers_are_idempotent() -> None:
    """ensureThinkingBlock returns the existing `.thinking` details or
    creates it OPEN above the .bubble; closeThinkingBlock is a no-op
    without a block and never reopens one once the answer started."""
    js = _js()
    fn = js.find("function ensureThinkingBlock")
    assert fn != -1, "ensureThinkingBlock must exist (near addTyping/removeTyping)"
    body = js[fn : js.find("\n}\n", fn)]
    assert "block.open = true" in body, "created open — the stream is the show"
    assert "insertBefore" in body
    assert 'querySelector(".bubble")' in body, "the block sits ABOVE the bubble"
    fn2 = js.find("function closeThinkingBlock")
    assert fn2 != -1, "closeThinkingBlock must exist"
    body2 = js[fn2 : js.find("\n}\n", fn2)]
    assert "block.open = false" in body2
 def test_delta_branch_collapses_block_and_transitions_to_streaming() -> None:
    """The first answer delta transitions thinking → streaming (even when
    thinking created the wrap first) and auto-collapses the block —
    idempotent, and it never reopens once the answer started."""
    js = _js()
    delta_idx = js.find('ev.type === "delta"')
    done_idx = js.find('ev.type === "done"')
    assert -1 < delta_idx < done_idx
    branch = js[delta_idx:done_idx]
    assert "uiState === UI_STATE.thinking" in branch
    assert "setUiState(UI_STATE.streaming)" in branch
    assert "closeThinkingBlock(wrap)" in branch
 def test_done_branch_sets_sawdone_and_closes_block() -> None:
    """On `done` the turn marks itself complete (sawDone — the stream-drop
    guard keys off it) and settles the thinking block closed."""
    js = _js()
    done_idx = js.find('ev.type === "done"')
    error_idx = js.find('ev.type === "error"')
    assert -1 < done_idx < error_idx
    branch = js[done_idx:error_idx]
    assert "sawDone = true" in branch
    assert "closeThinkingBlock(wrap)" in branch
 def test_stream_drop_guard_reports_severed_stream() -> None:
    """A stream that delivered frames but no `done` event ends in the error
    state (never a silent idle with a half bubble); the zero-frame case
    falls through to the existing empty-answer fallback. The guard runs
    after readSSE, before that fallback."""
    js = _js()
    assert "let sawDone = false" in js
    assert re.search(r"if \(!sawDone && !aborted && \(acc \|\| thinkingAcc\)\)", js), (
        "sawDone stream-drop guard missing after readSSE"
    )
    assert "The stream ended before my answer finished" in js
    sse_idx = js.find("await readSSE(res,")
    guard_idx = js.find("!sawDone && !aborted")
    fallback_idx = js.find("!aborted && !wrap")
    assert -1 < sse_idx < guard_idx < fallback_idx, (
        "guard must sit between readSSE and the zero-frame fallback"
    )
 def test_thinking_chevron_stills_under_reduced_motion() -> None:
    """Phase 17: the only motion in the thinking block (the summary
    chevron rotation) is disabled under prefers-reduced-motion."""
    css = _css()
    blocks = re.findall(
        r"@media \(prefers-reduced-motion: reduce\) \{([\s\S]*?)\n\}", css
    )
    assert any(
        "details.thinking summary::before" in b and "transition: none" in b
        for b in blocks
    ), "chevron transition must still under reduced motion"
@@ -0,0 +1,139 @@
 """Unit: the follow-the-bottom scroll contract in the static frontend
 (phase 18, owner choice 2026-08-23).
 The JS behavior itself is E2E-covered (tests/e2e/test_follow_bottom_scroll.py);
 here we pin the exported band constant and the single-gate markers that the
 story depends on — scrollIntoView appears exactly once in app.js, inside
 scrollReveal — so a silent regression back to unconditional per-delta /
 per-chunk scrolls is caught without a browser.
 """
 from __future__ import annotations
 import re
 from pathlib import Path
 FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
 APP_JS = FRONTEND / "assets" / "app.js"
 def _js() -> str:
    return APP_JS.read_text(encoding="utf-8")
 def _fn_body(js: str, name: str) -> str:
    """Source of the function starting at `function <name>` (to its closing
    brace at column 0) — same slicing style as test_frontend_feedback.py."""
    fn = js.find(f"function {name}")
    assert fn != -1, f"{name} must exist in app.js"
    return js[fn : js.find("\n}\n", fn)]
 def test_near_bottom_constant_exported_at_200px() -> None:
    """The "pinned to the bottom" band (the composer zone) must be an
    *exported* constant — unit-pinned, same pattern as TURN_TIMEOUT_MS."""
    js = _js()
    assert re.search(r"export\s+const\s+NEAR_BOTTOM_PX\s*=\s*200\s*;", js), (
        "app.js must export `const NEAR_BOTTOM_PX = 200`"
    )
 def test_is_near_bottom_uses_document_scroller() -> None:
    """isNearBottom measures the DOCUMENT scroller (there is no inner
    scroll container — the page scrolls on the window): distance from the
    bottom of the document <= NEAR_BOTTOM_PX."""
    js = _js()
    body = _fn_body(js, "isNearBottom")
    for ref in (
        "documentElement.scrollHeight",
        "window.scrollY",
        "window.innerHeight",
        "NEAR_BOTTOM_PX",
    ):
        assert ref in body, f"isNearBottom must reference {ref!r}"
    assert "<=" in body, "the pinned band is an upper bound, not exact equality"
 def test_single_scroll_gate() -> None:
    """scrollReveal is the ONE scroll call site in app.js: it fires only
    when forced or when the user is pinned to the bottom, keeps
    `block: "end"`, and both addMessage (behavior + force passthrough) and
    addTyping (defaults) delegate to it."""
    js = _js()
    body = _fn_body(js, "scrollReveal")
    assert "force || isNearBottom()" in body, "gate: force OR pinned to the bottom"
    assert "scrollIntoView" in body
    assert 'block: "end"' in body
    # The regression pin: exactly one scrollIntoView in the whole file, and
    # it lives inside scrollReveal.
    assert js.count("scrollIntoView") == 1, (
        "app.js must call scrollIntoView exactly once (inside scrollReveal)"
    )
    assert js.find("scrollIntoView") > js.find("function scrollReveal")
    # addMessage passes its behavior/force through; addTyping uses defaults.
    add_body = _fn_body(js, "addMessage")
    assert "scrollReveal(wrap, scrollBehavior, force)" in add_body
    assert "force = false" in add_body
    typing_body = _fn_body(js, "addTyping")
    assert "scrollReveal(wrap)" in typing_body
 def test_submit_reveal_is_gated() -> None:
    """Submit keeps the plain default call — no force: the gate decides,
    and it does in real use because submitting from the composer means the
    user is pinned (inside the 200px band); a submit with the viewport away
    from the bottom does not yank it."""
    js = _js()
    send = js.find("async function handleSend")
    assert send != -1, "handleSend must exist"
    call = 'addMessage("user", renderMarkdown(text));'
    idx = js.find(call, send)
    assert idx != -1, "handleSend must reveal the user message via the plain default"
    assert 'addMessage("user", renderMarkdown(text),' not in js, (
        "the submit call must not pass a third/fourth argument (no force)"
    )
 def test_restore_force_landing() -> None:
    """Both restore call sites are the only `force`d scrolls: one-shot,
    non-smooth ("auto") landing on the last restored message (phase-14
    behavior preserved)."""
    js = _js()
    body = _fn_body(js, "renderStoredMessage")
    assert 'addMessage("user", renderMarkdown(m.text), "auto", true)' in body
    assert 'addMessage("brain", renderMarkdown(m.text), "auto", true)' in body
    # Forced restores are restore-only: exactly two ("auto", true) sites.
    assert js.count('"auto", true') == 2, "only the two restore calls may force"
 def test_streaming_scrolls_only_through_gate() -> None:
    """The per-chunk scrolls that used to yank the viewport (the phase-17
    thinking branch and the streaming delta branch) now go through
    scrollReveal with no raw scrollIntoView at either call site; the
    block's internal bottom-pinning (its own overflow, not the page) stays."""
    js = _js()
    thinking_idx = js.find('ev.type === "thinking"')
    delta_idx = js.find('ev.type === "delta"')
    done_idx = js.find('ev.type === "done"')
    assert -1 < thinking_idx < delta_idx < done_idx
    thinking_branch = js[thinking_idx:delta_idx]
    delta_branch = js[delta_idx:done_idx]
    assert "scrollReveal(wrap)" in thinking_branch
    assert "scrollReveal(wrap)" in delta_branch
    assert "scrollIntoView" not in thinking_branch
    assert "scrollIntoView" not in delta_branch
    assert "textEl.scrollTop = textEl.scrollHeight" in thinking_branch
 def test_turn_end_focus_does_not_scroll() -> None:
    """The turn-end focus-back (phase 06's "always focus back") must not
    move the viewport: focusing the composer while the user is scrolled up
    would yank them to the bottom at the moment the turn ends — the exact
    defect phase 18 removes. preventScroll keeps the keyboard flow.
    startNewChat keeps plain focus (the list is cleared, nothing to yank
    past)."""
    js = _js()
    finally_idx = js.find("// done | error → idle: always settle, always focus back")
    assert finally_idx != -1, "the turn's finally block must exist"
    block = js[finally_idx : js.find("\n}", finally_idx)]
    assert 'input.focus({ preventScroll: true })' in block
    assert "input.focus()" not in block
@@ -16,11 +16,15 @@ from app.models import Chunk, Document
 from app.rag.importer import (
    EXCLUDED_DIRS,
    import_sources,
-    iter_markdown_files,
+    iter_importable_files,
 )
 from app.rag.llm import EmbeddingError
 from tests.fakes import FakeEmbedder
 #: A9 default extension set as dotted suffixes (what the importer passes to
 #: the walker when no override is configured).
 DEFAULT_EXTS = frozenset({".md", ".markdown", ".txt", ".yaml", ".yml", ".json", ".py"})
 class _PoisonEmbedder(FakeEmbedder):
    """Fails (like a real endpoint) on any text containing 'poison'."""
@@ -50,7 +54,9 @@ def _cleanup_source(db, source: str) -> None:
    db.commit()
-def test_iter_markdown_files_excludes_noncontent_dirs(tmp_path: Path) -> None:
+def test_iter_importable_files_excludes_noncontent_dirs_and_hidden(tmp_path: Path) -> None:
    """Well-known non-content dirs, hidden (dot-) dirs/files, and non-A9
    extensions are all skipped; the A9 formats pass."""
    root = tmp_path / "proj"
    for d in (
        "notes/sub",
@@ -61,11 +67,20 @@ def test_iter_markdown_files_excludes_noncontent_dirs(tmp_path: Path) -> None:
        ".pytest_cache",
        "dist",
        "build",
        ".esphome/.espressif",  # vendored hidden cache — the real A9 case
    ):
        (root / d).mkdir(parents=True)
    files = {
        # content that must be found:
        "README.md": "readme",
        "notes/sub/deep.md": "deep",
        "compose.yaml": "services: {}",
        "legacy.YML": "a: b",  # case-insensitive suffix
        "notes/sub/agent.py": "x = 1",
        "config.json": "{}",
        "README.txt": "plain",
        "notes/sub/deep.markdown": "md2",
        # must be skipped:
        ".venv/lib/junk.md": "junk",
        "node_modules/x/j.md": "j",
        ".git/c.md": "g",
@@ -73,17 +88,40 @@ def test_iter_markdown_files_excludes_noncontent_dirs(tmp_path: Path) -> None:
        ".pytest_cache/c.md": "pc",
        "dist/d.md": "d",
        "build/b.md": "b",
        ".esphome/.espressif/secret.md": "vendor",
        ".secret.md": "hidden file",  # dot-prefixed FILE, not just dir
        "notes/sub/notes.csv": "a,b",  # not an A9 format
        "notes/sub/file.md.bak": "x",
    }
    for rel, text in files.items():
        (root / rel).write_text(text)
    (root / "notes" / "not-md.txt").write_text("skip me")
-    found = {p.relative_to(root).as_posix() for p in iter_markdown_files(root)}
+    found = {p.relative_to(root).as_posix() for p in iter_importable_files(root, DEFAULT_EXTS)}
-    assert found == {"README.md", "notes/sub/deep.md"}
+    assert found == {
        "README.md",
        "notes/sub/deep.md",
        "compose.yaml",
        "legacy.YML",
        "notes/sub/agent.py",
        "config.json",
        "README.txt",
        "notes/sub/deep.markdown",
    }
-def test_iter_markdown_files_missing_dir_yields_nothing(tmp_path: Path) -> None:
+def test_iter_importable_files_missing_dir_yields_nothing(tmp_path: Path) -> None:
-    assert iter_markdown_files(tmp_path / "definitely-missing") == []
+    assert iter_importable_files(tmp_path / "definitely-missing", DEFAULT_EXTS) == []
 def test_iter_importable_files_respects_custom_extension_filter(tmp_path: Path) -> None:
    """A narrower filter (e.g. md only) excludes the other A9 formats."""
    root = tmp_path / "filtered"
    root.mkdir()
    (root / "a.md").write_text("a")
    (root / "b.yaml").write_text("a: b")
    (root / "c.py").write_text("x = 1")
    found = {p.name for p in iter_importable_files(root, frozenset([".md"]))}
    assert found == {"a.md"}
 def test_excluded_dirs_match_plan_anchor_a9() -> None:
@@ -277,3 +315,65 @@ def test_chunk_positions_and_titles(db, tmp_path: Path) -> None:
        assert positions == list(range(len(doc.chunks))) and len(doc.chunks) >= 2
    finally:
        _cleanup_source(db, root.name)
 def test_multi_format_import_counts_per_format_and_titles_stem(db, tmp_path: Path) -> None:
    """A9 formats all import; the summary records per-format counts, and
    non-markdown titles come from the file stem (a ``#`` line is a comment
    there, not a heading)."""
    root = tmp_path / "multi"
    (root / "svc").mkdir(parents=True)
    (root / "guide.md").write_text("# Real Heading\n\nbody\n")
    (root / "svc" / "compose.yaml").write_text("# a comment\nservices:\n  gitlab: {}\n")
    (root / "svc" / "agent.py").write_text("# docstring-like comment\ndef ping():\n    return 1\n")
    (root / "inventory.json").write_text('{"hosts": []}\n')
    (root / "notes.txt").write_text("plain text notes\n")
    llm = FakeEmbedder()
    try:
        summary = asyncio.run(import_sources([root], llm, session=db))
        assert summary.files == 5
        assert summary.added == 5
        assert summary.formats == {"md": 1, "yaml": 1, "py": 1, "json": 1, "txt": 1}
        # PLAN §9 summary line: counts, highest first, ext:name pairs.
        assert summary.format_counts() == "json:1,md:1,py:1,txt:1,yaml:1"
        titles = {
            d.path: d.title
            for d in db.scalars(select(Document).where(Document.source == root.name)).all()
        }
        assert titles["guide.md"] == "Real Heading"  # markdown keeps the H1
        assert titles["svc/compose.yaml"] == "compose"  # …comment is not a heading
        assert titles["svc/agent.py"] == "agent"
        assert titles["inventory.json"] == "inventory"
        assert titles["notes.txt"] == "notes"
    finally:
        _cleanup_source(db, root.name)
 def test_prune_removes_files_now_excluded_by_format_filter(db, tmp_path: Path) -> None:
    """Previously-imported junk leaves the index: a file that no longer
    matches the A9 extension filter is pruned on the next ``prune=True`` run.
    This is how dot-dir READMEs imported before the scope fix get cleaned up."""
    root = tmp_path / "cleanup"
    root.mkdir()
    (root / "keep.md").write_text("# Keep\n\nkept\n")
    (root / "junk.md.bak").write_text("old junk that was once imported\n")
    llm = FakeEmbedder()
    try:
        # Seed: import both files as if they were valid at the time.
        (root / "junk.md").write_text("old junk that was once imported\n")
        (root / "junk.md.bak").unlink()
        asyncio.run(import_sources([root], llm, session=db))
        # Rename the junk out of the A9 formats, then prune.
        (root / "junk.md").rename(root / "junk.md.bak")
        summary = asyncio.run(import_sources([root], llm, session=db, prune=True))
        assert summary.pruned == 1
        assert summary.unchanged == 1  # keep.md survived
        assert db.scalar(
            select(Document).where(Document.source == root.name, Document.path == "junk.md")
        ) is None
        assert db.scalar(
            select(Document).where(Document.source == root.name, Document.path == "keep.md")
        ) is not None
    finally:
        _cleanup_source(db, root.name)
@@ -16,7 +16,13 @@ from typing import Any
 import pytest
 from app.config import Settings
-from app.rag.llm import EmbeddingDimensionError, EmbeddingError, LLMClient, LLMError
+from app.rag.llm import (
    EmbeddingDimensionError,
    EmbeddingError,
    LLMClient,
    LLMError,
    StreamPiece,
 )
 def _settings(**kwargs: Any) -> Settings:
@@ -236,11 +242,21 @@ def test_single_oversized_text_fails_actionably() -> None:
 # ---------- chat streaming (phase 03) ----------
-def _chunk(content: str | None = "text", empty: bool = False):
+def _chunk(
-    """One fake ChatCompletionChunk (``choices[].delta.content`` shape)."""
+    content: str | None = "text", empty: bool = False, reasoning: str | None = None
 ):
    """One fake ChatCompletionChunk (``choices[].delta`` shape).
    ``reasoning_content`` is present on the delta only when *reasoning*
    is not None — mirroring the real wire, where the field exists only
    when the model sends it.
    """
    if empty:
        return SimpleNamespace(choices=[])
-    return SimpleNamespace(choices=[SimpleNamespace(delta=SimpleNamespace(content=content))])
+    delta: SimpleNamespace = SimpleNamespace(content=content)
    if reasoning is not None:
        delta.reasoning_content = reasoning
    return SimpleNamespace(choices=[SimpleNamespace(delta=delta)])
 class _FakeChatStream:
@@ -273,16 +289,18 @@ class _FakeCompletions:
 def _make_stream_client(
-    chunks: list | None = None, fail: Exception | None = None
+    chunks: list | None = None,
    fail: Exception | None = None,
    **settings_kwargs: Any,
 ) -> tuple[LLMClient, _FakeCompletions]:
    completions = _FakeCompletions(chunks, fail)
    fake_openai = SimpleNamespace(chat=SimpleNamespace(completions=completions))
-    llm = LLMClient(_settings())
+    llm = LLMClient(_settings(**settings_kwargs))
    llm._client = fake_openai  # pyright: ignore[reportAttributeAccessIssue]
    return llm, completions
-async def _collect(llm: LLMClient, messages: list[dict[str, str]]) -> list[str]:
+async def _collect(llm: LLMClient, messages: list[dict[str, str]]) -> list[StreamPiece]:
    return [p async for p in llm.chat_stream(messages)]
@@ -291,7 +309,13 @@ def test_chat_stream_yields_deltas_in_order() -> None:
        [_chunk("Hey "), _chunk("you've "), _chunk("got this! 🧠")]
    )
    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
-    assert pieces == ["Hey ", "you've ", "got this! 🧠"]
+    # Content-only chunks yield content pieces in wire order.
    assert [(p.kind, p.text) for p in pieces] == [
        ("content", "Hey "),
        ("content", "you've "),
        ("content", "got this! 🧠"),
    ]
    assert all(isinstance(p, StreamPiece) for p in pieces)
 def test_chat_stream_uses_locked_generation_params() -> None:
@@ -302,13 +326,91 @@ def test_chat_stream_uses_locked_generation_params() -> None:
    assert completions.kwargs["model"] == "turbo"
    assert completions.kwargs["stream"] is True
    assert completions.kwargs["temperature"] == 0.4
-    assert completions.kwargs["max_tokens"] == 700
+    # Phase 11: the old hard 700-token cap is gone — answers may run up to
    # BOR_MAX_OUTPUT_TOKENS (default 32 768) so they are not cut off.
    assert completions.kwargs["max_tokens"] == 32_768
    assert completions.kwargs["messages"] == messages
 def test_chat_stream_max_tokens_comes_from_settings() -> None:
    """The output cap is operator-configurable, not a client constant."""
    llm, completions = _make_stream_client(
        [_chunk("x")], max_output_tokens=1234  # pyright: ignore[reportArgumentType]
    )
    asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert completions.kwargs is not None
    assert completions.kwargs["max_tokens"] == 1234
 def test_chat_stream_skips_empty_deltas_and_choiceless_chunks() -> None:
    llm, _ = _make_stream_client([_chunk("a"), _chunk(empty=True), _chunk(None), _chunk("b")])
-    assert asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) == ["a", "b"]
+    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert [(p.kind, p.text) for p in pieces] == [("content", "a"), ("content", "b")]
 def test_chat_stream_maps_reasoning_content_to_thinking_pieces() -> None:
    """The verified aipi wire field (``delta.reasoning_content``) maps to
    ``thinking`` pieces; content chunks are untouched by the presence of
    reasoning elsewhere in the stream."""
    llm, _ = _make_stream_client(
        [
            _chunk("", reasoning="Step 1: parse the question."),
            _chunk("", reasoning="Step 2: cite the doc."),
            _chunk("Talos."),
        ]
    )
    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert [(p.kind, p.text) for p in pieces] == [
        ("thinking", "Step 1: parse the question."),
        ("thinking", "Step 2: cite the doc."),
        ("content", "Talos."),
    ]
 def test_chat_stream_falls_back_to_reasoning_field() -> None:
    """Future-proofing: a bare ``delta.reasoning`` field (no
    ``reasoning_content``) is picked up by the fallback getattr."""
    chunk = SimpleNamespace(
        choices=[
            SimpleNamespace(delta=SimpleNamespace(content="ans", reasoning="why not"))
        ]
    )
    llm, _ = _make_stream_client([chunk])
    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert [(p.kind, p.text) for p in pieces] == [
        ("thinking", "why not"),
        ("content", "ans"),
    ]
 def test_chat_stream_thinking_yields_before_content_in_chunk() -> None:
    """One chunk carrying both fields yields the thinking piece first."""
    llm, _ = _make_stream_client([_chunk("answer", reasoning="hmm")])
    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert [(p.kind, p.text) for p in pieces] == [
        ("thinking", "hmm"),
        ("content", "answer"),
    ]
 def test_chat_stream_interleaved_thinking_and_content_order_preserved() -> None:
    """The piece sequence must match the chunk sequence exactly — a late
    or interleaved thinking chunk is emitted at its wire position."""
    llm, _ = _make_stream_client(
        [
            _chunk("", reasoning="t1"),
            _chunk("c1"),
            _chunk("", reasoning="t2"),
            _chunk("c2"),
        ]
    )
    pieces = asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
    assert [(p.kind, p.text) for p in pieces] == [
        ("thinking", "t1"),
        ("content", "c1"),
        ("thinking", "t2"),
        ("content", "c2"),
    ]
 def test_chat_stream_wraps_failures_as_llm_error() -> None:
@@ -22,21 +22,30 @@ def _doc(path: str, content: str, title: str) -> Document:
 def test_persona_rules_present_verbatim() -> None:
    # Aligned to the owner's working-tree persona edits (PLAN §6 revision,
    # 2026-08-22): no "you've got this" tagline, no mandated deflection
    # opening. The honesty gate itself (rule 3) is unchanged.
    for fragment in (
        'You are "Brain of Reese" — the digital brain of Reese, a self-hoster and',
-        'optimistic about the user\'s ability to do things ("you\'ve got this")',
+        "optimistic about the user's ability to do things",
        "Answer ONLY from the provided document context. Cite which document(s)",
        "you used, by path.",
        "Be concrete: names, versions, ports, hosts, schedules",
        'HONESTY GATE: if <relevance> is "LOW", you must NOT pretend to know',
-        'Start your answer with a variant of: "I haven\'t done anything like that."',
+        "Offer 2-3 alternative questions about things you DO have notes on.",
        "Then offer 2-3 alternative questions about things you DO have notes on.",
        "Never invent facts, hosts, or steps that are not in the context.",
        "Keep answers tight: short paragraphs, bullets where helpful.",
    ):
        assert fragment in PERSONA
 def test_persona_owner_edits_are_preserved() -> None:
    """PLAN §6 revision (2026-08-22): the removed elements must stay out."""
    assert 'you\'ve got this' not in PERSONA  # tagline removed by the owner
    assert "Start your answer with a variant of" not in PERSONA  # no mandated opening
    assert "HONESTY GATE" in PERSONA  # the gate itself is intact
 def test_high_prompt_carries_relevance_marker_and_full_documents() -> None:
    doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes Homelab Cluster")
    prompt = build_high_prompt([doc])
@@ -83,6 +92,27 @@ def test_low_prompt_with_no_titles() -> None:
    assert "nothing close at all" in build_deflect_prompt([])
 def test_zero_note_prompt_is_byte_identical_to_pre_steering() -> None:
    """Phase 15 contract: with no steering notes the prompt is exactly what
    it was before the <tuning> section existed."""
    doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes Homelab Cluster")
    block = (
        '<document source="Homelab" path="kubernetes.md" title="Kubernetes Homelab Cluster">\n'
        "Talos Linux on three nodes.\n"
        "</document>"
    )
    assert build_high_prompt([doc]) == _base("HIGH") + "\n<documents>\n" + block + "\n</documents>"
    assert build_deflect_prompt(["T1", "T2"]) == (
        _base("LOW")
        + "\nDEFLECT_MODE: retrieval was weak — the titles below are the closest "
        "your notes come to the question. They are titles only; do not pretend "
        "they answer it. Use them to propose 2-3 alternative questions.\n"
        + "- T1\n- T2"
    )
    assert "<tuning>" not in build_high_prompt([doc])
    assert "<tuning>" not in build_deflect_prompt([])
 def test_relevance_placeholder_rejected_for_garbage() -> None:
    with pytest.raises(ValueError, match="HIGH or LOW"):
        _base("MEDIUM")
@@ -8,6 +8,8 @@ from __future__ import annotations
 import uuid
 import pytest
 from app.models import Document
 from app.rag.retriever import TRUNCATION_MARKER, RetrievedChunk, select_documents
@@ -94,3 +96,102 @@ def test_under_budget_no_truncation() -> None:
 def test_empty_hits_yield_no_documents() -> None:
    assert select_documents([], n=2, max_chars=24_000) == []
 # ---------------------------------------------------------------------------
 # Hybrid retrieval (A7): RRF fusion + lexical tsquery
 # ---------------------------------------------------------------------------
 from app.rag.retriever import fuse, lexical_tsquery  # noqa: E402
 def _rc(
    doc_path: str, cosine: float = 0.0, fts_hit: bool = False, position: int = 0
 ) -> RetrievedChunk:
    return RetrievedChunk(
        chunk_id=uuid.uuid4(),
        position=position,
        content="x" * 20,
        score=0.0,
        document=_doc(doc_path, "x" * 20),
        cosine=cosine,
        fts_hit=fts_hit,
    )
 def test_lexical_tsquery_tokens_lowercased_deduped_in_order() -> None:
    assert lexical_tsquery("How did I Install GITLAB gitlab?") == "how | did | i | install | gitlab"
 def test_lexical_tsquery_punctuation_and_umlauts_ignored() -> None:
    assert lexical_tsquery("c3-r00t? -- what's up!") == "c3 | r00t | what | s | up"
 def test_lexical_tsquery_pure_symbols_return_none() -> None:
    assert lexical_tsquery("??? ???") is None
    assert lexical_tsquery("") is None
 def test_lexical_tsquery_stopwords_left_to_postgres() -> None:
    # lexical_tsquery passes raw tokens through; Postgres's to_tsquery
    # lexing drops the stopwords (verified against real PG in
    # test_retrieve_empty_kb / integration tests).
    assert lexical_tsquery("how do i") == "how | do | i"
 def test_fuse_combines_both_lists_for_double_hits() -> None:
    v1 = _rc("a.md", cosine=0.9)
    v2 = _rc("b.md", cosine=0.5)
    l1 = _rc("a.md", cosine=0.1)  # same chunk id -> matched in place
    a_id = v1.chunk_id
    l1.chunk_id = a_id
    out = fuse([v1, v2], [l1], k=60)
    by_id = {rc.chunk_id: rc for rc in out}
    # a: 1/61 (vector rank 1) + 1/61 (lexical rank 1); b: 1/62 only.
    assert by_id[a_id].score == pytest.approx(2 / 61)
    assert by_id[a_id].fts_hit is True
    assert by_id[v2.chunk_id].score == pytest.approx(1 / 62)
    assert by_id[v2.chunk_id].fts_hit is False
    assert [rc.chunk_id for rc in out] == [a_id, v2.chunk_id]
 def test_fuse_lexical_only_chunks_enter_with_zero_cosine() -> None:
    vector = [_rc("a.md", cosine=0.8)]
    lexical = [_rc("b.md", cosine=0.0, fts_hit=True)]
    out = fuse(vector, lexical, k=60)
    assert len(out) == 2
    b = next(rc for rc in out if rc.document.path == "b.md")
    assert b.cosine == 0.0
    assert b.fts_hit is True
    # Still ranked by its (only) RRF term.
    assert b.score == pytest.approx(1 / 61)
 def test_fuse_orders_by_score_then_cosine_then_path() -> None:
    # Two chunks share an RRF score (both rank 1 in different lists):
    # the higher-cosine one must sort first.
    hi = _rc("z.md", cosine=0.9)
    lo = _rc("a.md", cosine=0.2)
    out = fuse([hi], [lo], k=60)
    assert [rc.document.path for rc in out] == ["z.md", "a.md"]
    # Equal score AND cosine -> path order.
    p1 = _rc("b.md", cosine=0.5)
    p2 = _rc("a.md", cosine=0.5)
    out = fuse([p1], [p2], k=60)
    assert [rc.document.path for rc in out] == ["a.md", "b.md"]
    # Equal score, cosine, path -> position order.
    s1 = _rc("a.md", cosine=0.5, position=1)
    s2 = _rc("a.md", cosine=0.5, position=0)
    out = fuse([s1], [s2], k=60)
    assert [rc.position for rc in out] == [0, 1]
 def test_fuse_rejects_nonpositive_k() -> None:
    with pytest.raises(ValueError):
        fuse([], [], k=0)
    with pytest.raises(ValueError):
        fuse([], [], k=-1)
 def test_fuse_empty_lists() -> None:
    assert fuse([], [], k=60) == []
@@ -4,7 +4,7 @@ from __future__ import annotations
 import json
 from app.api.chat import sse_event
-from app.schemas import ChatErrorEvent
+from app.schemas import ChatErrorEvent, ChatThinkingEvent
 def _payload(frame: str) -> dict:
@@ -61,3 +61,18 @@ def test_error_event_shape_is_type_and_detail_only() -> None:
    dumped = ChatErrorEvent(detail="The chat model dropped the connection").model_dump()
    assert set(dumped.keys()) == {"type", "detail"}
    assert dumped["type"] == "error"  # default — call sites never spell it out
 def test_thinking_frame_serializes_exactly() -> None:
    """Phase 17 (PLAN §4 extension): the ``thinking`` frame is exactly
    ``{type: "thinking", text: str}`` — the sibling shape of ``delta``
    the client's readSSE handler will branch on."""
    frame = sse_event(ChatThinkingEvent(text="Step 1: check the docs…").model_dump())
    assert frame == 'data: {"type": "thinking", "text": "Step 1: check the docs…"}\n\n'
    assert _payload(frame) == {"type": "thinking", "text": "Step 1: check the docs…"}
 def test_thinking_event_shape_is_type_and_text_only() -> None:
    dumped = ChatThinkingEvent(text="hmm").model_dump()
    assert set(dumped.keys()) == {"type", "text"}
    assert dumped["type"] == "thinking"  # default — call sites never spell it out
@@ -0,0 +1,193 @@
 """Unit: steering notes (phase 15) — the <tuning> prompt section.
 Pure logic, no Postgres and no network: :func:`build_steering_section`
 (empty/one/many/budget-truncation), its placement in the HIGH and LOW
 prompts, and the ``plan_turn`` wiring (notes → prompt + ``tuning_count``).
 """
 from __future__ import annotations
 import uuid
 import pytest
 from app.api import chat as chat_api
 from app.config import Settings
 from app.models import Document
 from app.rag.prompts import build_deflect_prompt, build_high_prompt, build_steering_section
 from app.rag.retriever import TRUNCATION_MARKER, RetrievedChunk
 def _settings() -> Settings:
    return Settings(_env_file=None, relevance_threshold=0.30)  # pyright: ignore[reportCallIssue]
 def _doc(title: str, content: str) -> Document:
    return Document(
        id=uuid.uuid4(),
        source="Homelab",
        path=f"{title.lower().replace(' ', '-')}.md",
        full_path="/tmp/doc.md",
        title=title,
        content=content,
        content_hash="0" * 64,
    )
 def _chunk(doc: Document, score: float) -> RetrievedChunk:
    return RetrievedChunk(
        chunk_id=uuid.uuid4(),
        position=0,
        content=doc.content[:32],
        score=score,
        document=doc,
        cosine=score,
    )
 # ---------- build_steering_section ----------
 def test_steering_section_empty_when_no_notes() -> None:
    assert build_steering_section([]) == ""
 def test_steering_section_empty_when_notes_are_blank() -> None:
    assert build_steering_section(["", "   ", "\n\t"]) == ""
 def test_steering_section_single_note_numbered() -> None:
    section = build_steering_section(["be more concise"])
    assert section.startswith("<tuning>\n")
    assert section.endswith("\n</tuning>")
    assert "1. be more concise" in section
    assert TRUNCATION_MARKER not in section
 def test_steering_section_trims_note_edges() -> None:
    section = build_steering_section(["  be more concise  "])
    assert "1. be more concise" in section
    assert "1.   be more concise" not in section
 def test_steering_section_many_notes_numbered_in_order() -> None:
    section = build_steering_section(["alpha", "beta", "gamma"])
    assert "1. alpha" in section
    assert "2. beta" in section
    assert "3. gamma" in section
    assert section.index("1. alpha") < section.index("2. beta") < section.index("3. gamma")
    assert TRUNCATION_MARKER not in section
 def test_steering_section_budget_truncation_keeps_oldest_prefix_and_marker() -> None:
    # Each note is 300 chars; with a 600-char budget only note 1 fits, so
    # the oldest-fitting prefix is kept and the overflow is marked.
    notes = [f"note-{i} " + "x" * (300 - len(f"note-{i} ")) for i in range(3)]
    section = build_steering_section(notes, max_chars=600)
    assert TRUNCATION_MARKER in section
    assert len(section) <= 600
    assert "1. note-0" in section
    assert "note-1" not in section
    assert "note-2" not in section
    # The marker comes last, after the kept notes.
    assert section.index("1. note-0") < section.index(TRUNCATION_MARKER)
 def test_steering_section_fits_budget_exactly_when_all_notes_fit() -> None:
    section = build_steering_section(["a", "b", "c"], max_chars=10_000)
    assert TRUNCATION_MARKER not in section
    assert len(section) <= 10_000
 def test_steering_section_default_budget_from_settings(monkeypatch: pytest.MonkeyPatch) -> None:
    from app.rag import prompts as prompts_mod
    monkeypatch.setattr(
        prompts_mod, "get_settings", lambda: Settings(_env_file=None)  # pyright: ignore[reportCallIssue]
    )
    # 5 notes of 2000 chars (the API max) = 10k+ chars > the 8000 default.
    notes = [f"note-{i} " + "y" * (2000 - len(f"note-{i} ")) for i in range(5)]
    section = build_steering_section(notes)
    assert TRUNCATION_MARKER in section
    assert len(section) <= 8_000
 def test_steering_section_nonpositive_budget_is_empty() -> None:
    assert build_steering_section(["be concise"], max_chars=0) == ""
    assert build_steering_section(["be concise"], max_chars=-10) == ""
 def test_steering_section_tiny_budget_never_exceeds_cap() -> None:
    # Pathological budget: the section must never exceed the cap — bare
    # marker when it fits, no section at all when even that doesn't.
    assert len(build_steering_section(["a" * 500], max_chars=10)) <= 10
    fits_marker = build_steering_section(["a" * 500], max_chars=len(TRUNCATION_MARKER))
    assert fits_marker == TRUNCATION_MARKER
 # ---------- prompt placement (both modes) ----------
 def test_high_prompt_steering_sits_between_relevance_and_documents() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_CONTENT")
    prompt = build_high_prompt([doc], notes=["be concise"])
    i_rel = prompt.index("<relevance>HIGH</relevance>")
    i_open = prompt.index("<tuning>")
    i_close = prompt.index("</tuning>")
    i_docs = prompt.index("<documents>")
    assert i_rel < i_open < i_close < i_docs
    assert "1. be concise" in prompt
    assert "TALOS_DOC_CONTENT" in prompt  # documents still full
 def test_deflect_prompt_steering_sits_between_relevance_and_deflect_mode() -> None:
    prompt = build_deflect_prompt(["Title A", "Title B"], notes=["be concise", "cite paths"])
    i_rel = prompt.index("<relevance>LOW</relevance>")
    i_open = prompt.index("<tuning>")
    i_close = prompt.index("</tuning>")
    i_mode = prompt.index("DEFLECT_MODE")
    assert i_rel < i_open < i_close < i_mode
    assert "1. be concise" in prompt
    assert "2. cite paths" in prompt
    assert "- Title A" in prompt  # weak-hit titles still carried
    assert "DEFLECT_MODE" in prompt
 # ---------- plan_turn wiring (gate + steering, fake retriever rows) ----------
 def test_plan_turn_high_mode_injects_notes() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
    plan = chat_api.plan_turn(
        [_chunk(doc, 0.90)], _settings(), notes=["be concise", "assume NixOS"]
    )
    assert plan.deflected is False
    assert plan.tuning_count == 2
    assert "<tuning>" in plan.system_prompt
    assert "1. be concise" in plan.system_prompt
    assert "2. assume NixOS" in plan.system_prompt
    assert "<relevance>HIGH</relevance>" in plan.system_prompt
    assert "TALOS_DOC_SENT" in plan.system_prompt
 def test_plan_turn_low_mode_injects_notes() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_NEVER_SENT")
    plan = chat_api.plan_turn(
        [_chunk(doc, 0.10)], _settings(), notes=["be concise"]
    )
    assert plan.deflected is True
    assert plan.tuning_count == 1
    assert "DEFLECT_MODE" in plan.system_prompt
    assert "<tuning>" in plan.system_prompt
    assert "1. be concise" in plan.system_prompt
    assert "TALOS_DOC_NEVER_SENT" not in plan.system_prompt  # titles only, still
 def test_plan_turn_without_notes_has_no_tuning_section() -> None:
    doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
    for chunks in (
        [_chunk(doc, 0.90)],  # HIGH
        [_chunk(doc, 0.10)],  # LOW
    ):
        plan = chat_api.plan_turn(chunks, _settings())
        assert plan.tuning_count == 0
        assert "<tuning>" not in plan.system_prompt
@@ -55,6 +55,7 @@ dependencies = [
    { name = "alembic" },
    { name = "fastapi" },
    { name = "httpx" },
    { name = "itsdangerous" },
    { name = "openai" },
    { name = "pgvector" },
    { name = "psycopg", extra = ["binary"] },
@@ -80,6 +81,7 @@ requires-dist = [
    { name = "alembic", specifier = ">=1.13,<2.0" },
    { name = "fastapi", specifier = ">=0.115,<1.0" },
    { name = "httpx", specifier = ">=0.27,<1.0" },
    { name = "itsdangerous", specifier = ">=2.2,<3.0" },
    { name = "openai", specifier = ">=1.40,<3.0" },
    { name = "pgvector", specifier = ">=0.3,<1.0" },
    { name = "psycopg", extras = ["binary"], specifier = ">=3.1,<4.0" },
@@ -433,6 +435,15 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
 ]
 [[package]]
 name = "itsdangerous"
 version = "2.2.0"
 source = { registry = "https://pypi.org/simple" }
 sdist = { url = "https://files.pythonhosted.org/packages/9c/cb/8ac0172223afbccb63986cc25049b154ecfb5e85932587206f42317be31d/itsdangerous-2.2.0.tar.gz", hash = "sha256:e0050c0b7da1eea53ffaf149c0cfbb5c6e2e2b69c4bef22c81fa6eb73e5f6173", size = 54410, upload-time = "2024-04-16T21:28:15.614Z" }
 wheels = [
    { url = "https://files.pythonhosted.org/packages/04/96/92447566d16df59b2a776c0fb82dbc4d9e07cd95062562af01e408583fc4/itsdangerous-2.2.0-py3-none-any.whl", hash = "sha256:c6242fc49e35958c8b15141343aa660db5fc54d4f13a1db01a3f5891b98700ef", size = 16234, upload-time = "2024-04-16T21:28:14.499Z" },
 ]
 [[package]]
 name = "jiter"
 version = "0.16.0"
Author	SHA1	Message	Date
ducoterra	bc0158f858	feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport	2026-08-24 10:53:04 -04:00
ducoterra	b16deb2b1d	feat(chat): stream model thinking over SSE and show it in a collapsible block	2026-08-24 09:52:27 -04:00
ducoterra	cbc263a4b2	feat(auth): single-admin password login (signed cookie) — gate tuning + Sources catalog, keep chat and document viewer public	2026-08-23 19:58:39 -04:00
ducoterra	fc0d9a2d5c	feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt	2026-08-22 16:44:42 -04:00
ducoterra	19df7df99d	feat(ui): persist the chat conversation in localStorage — survives refresh and navigation, with a New chat reset	2026-08-22 15:52:53 -04:00
ducoterra	2485b50af0	fix(ui): document viewer back button returns to the page you came from (chat or sources)	2026-08-22 15:26:43 -04:00
ducoterra	8ca564cd83	fix(ui): uniform header bar height on chat, sources, and the document viewer	2026-08-22 15:14:31 -04:00
ducoterra	0da5275eeb	fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off	2026-08-22 11:30:19 -04:00
ducoterra	6ec6181c7b	feat(ui): clickable document viewer — open any cited document in the browser from chat chips and the sources table	2026-08-22 02:08:49 -04:00
ducoterra	7e8d14702e	feat(rag): hybrid FTS+vector retrieval and multi-format ingestion — name-your-tool questions find the right document	2026-08-22 01:27:02 -04:00