chore(agent): track .agent/ planning tree in git
Remove the blanket .agent/ gitignore so the phase roadmap, user stories, reports, and PLAN.md are versioned with the code. Only runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log) remain ignored. Update AGENTS.md git protocol rule to match.
This commit is contained in:
Binary file not shown.
@@ -0,0 +1,76 @@
|
||||
# Phase 02 — Story: Import Documents
|
||||
|
||||
**Story:** `.agent/user_stories/import-documents.md`
|
||||
**Context:** `.agent/PLAN.md` §5 (data model), §9 (logging), §11 (import workflow)
|
||||
|
||||
## Goal
|
||||
The importer (`scripts/import_docs.py`) + `GET /api/docs` + the Sources page
|
||||
rendering the indexed documents — the knowledge base becomes refreshable.
|
||||
|
||||
## Implementation steps
|
||||
1. `app/rag/__init__.py`, `app/rag/chunker.py` — markdown-aware chunker
|
||||
(PLAN §5 policy: heading splits, 2000-char target, 200 overlap, keep
|
||||
nearest heading). Pure functions, fully unit-testable.
|
||||
2. `app/rag/llm.py` — `LLMClient` (openai async) with `embed(texts) ->
|
||||
list[list[float]]` (batched, `BOR_EMBED_BATCH_SIZE`) and a
|
||||
`embed_one`; dimension check vs `settings.embedding_dim` with a loud,
|
||||
actionable error. (Chat streaming is added in Phase 03 on this client.)
|
||||
3. `app/rag/importer.py` — the core: directory walk (exclusion list, PLAN
|
||||
A9; `*.md` only), sha256 delta vs `documents.content_hash`,
|
||||
upsert-or-skip, two-phase chunk replace (insert doc → replace chunks →
|
||||
embed → commit), `--prune` support, per-file + summary logging.
|
||||
4. `scripts/import_docs.py` — CLI wrapper (argparse): repeatable
|
||||
`--source` (default `~/Homelab` `~/Deployments`, `expanduser`),
|
||||
`--prune`, `--limit`.
|
||||
5. `app/api/docs.py` — `GET /api/docs` → `{"documents": [DocSummary]}`
|
||||
(include `chunks` count via `func.count`); mount in `app/main.py`
|
||||
**before** the static mount.
|
||||
6. `frontend/assets/sources.js` + `sources.html` polish — wire the real
|
||||
endpoint (already scaffolded to expect this shape); keep the empty state.
|
||||
7. Update `README.md` §Knowledge Base Import with the final commands +
|
||||
exclusion list + "update your docs → re-run the script" workflow.
|
||||
|
||||
## UI Verification
|
||||
Compare `/sources.html` against the story's "UI Visualization & Structure":
|
||||
stat cards `auto-fit minmax(170px,1fr)`; full-width table (≥85% container);
|
||||
mono path column with `title` ellipsis; empty state with the exact command;
|
||||
`<caption class="visually-hidden">`, `scope="col"`, scroll wrapper
|
||||
`role="region" tabindex="0"`. No CDN refs. Take a 1280px and 375px
|
||||
screenshot pass before finishing.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: chunker (heading splits, overlap, short-doc single chunk, code
|
||||
fences kept intact), exclusion walk (temp tree with `.venv` junk),
|
||||
delta logic (unchanged/changed/pruned via tmp Postgres or in-memory fakes
|
||||
— real DB preferred since compose runs locally).
|
||||
- Integration: `GET /api/docs` empty shape + populated shape; importer
|
||||
end-to-end against `tests/fixtures/docs/` into a test schema.
|
||||
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**
|
||||
on `app/` (importer + chunker + client are the bulk; test them hard).
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite (DB must be up: `podman compose up -d db`):
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_import_documents.py -v --no-cov
|
||||
```
|
||||
|
||||
The test file implements the story's Playwright Mapping Rule (seed via the
|
||||
import function against `tests/fixtures/docs/` with the mock LLM; assert
|
||||
Sources page rows, layout width, and the empty state).
|
||||
|
||||
## Success criteria
|
||||
- [ ] `uv run python -m scripts.import_docs` (fixtures) imports all 3 docs,
|
||||
re-run reports `unchanged`
|
||||
- [ ] `GET /api/docs` + Sources page show the docs (real run: `~/Homelab`
|
||||
+ `~/Deployments` counts logged)
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] UI verification passed (screenshots attached to the phase record)
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] README import section updated
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(kb): markdown importer with sha256 deltas, chunking, batched embeddings, and Sources page"
|
||||
```
|
||||
@@ -0,0 +1,74 @@
|
||||
# Phase 03 — Story: Chat RAG Answer (happy path)
|
||||
|
||||
**Story:** `.agent/user_stories/chat-rag-answer.md`
|
||||
**Context:** `.agent/PLAN.md` §3 (data flow), §4 (SSE contract), §6 (persona), §9 (logging)
|
||||
|
||||
## Goal
|
||||
The core product loop: question → embed → cosine top-4 → full top-2
|
||||
documents → `turbo` (streamed) → chippy grounded answer with source chips.
|
||||
|
||||
## Implementation steps
|
||||
1. `app/rag/retriever.py` — `retrieve(db, question_embedding) ->
|
||||
list[RetrievedChunk]` (score = 1 − distance, `ORDER BY embedding <=> $1
|
||||
LIMIT BOR_TOP_K_CHUNKS`) + `select_documents(chunks, n) -> list[Document]`
|
||||
(distinct by `document_id`, ranked by best chunk score, cap content at
|
||||
`BOR_MAX_CONTEXT_CHARS` with `[…truncated…]`).
|
||||
2. `app/rag/prompts.py` — locked persona + HONESTY GATE prompt builder
|
||||
(PLAN §6 verbatim, `<relevance>HIGH|LOW</relevance>`, `<documents>`
|
||||
block; LOW mode includes the `DEFLECT_MODE` marker + weak-hit titles).
|
||||
3. `app/rag/llm.py` — add `chat_stream(messages) -> AsyncIterator[str]`
|
||||
(openai async, `stream=True`, `model=turbo`, temperature 0.4,
|
||||
max_tokens ~700).
|
||||
4. `app/api/chat.py` — `POST /api/chat` (ChatRequest) → `StreamingResponse`
|
||||
(SSE): emit `delta` events from the stream, then the `done` event
|
||||
(deflected, sources, suggestions); insert `query_log` row (deflected=
|
||||
false this phase); per-turn log line (PLAN §9); structured error events
|
||||
(`{"type":"error","detail":…}`) on LLM/DB failure.
|
||||
5. `frontend/assets/app.js` — replace the placeholder handler: `fetch` +
|
||||
`ReadableStream` SSE parser; render deltas live into a brain bubble
|
||||
(reuse the typing-indicator → streaming handoff); on `done`, append
|
||||
`.source-chip`s under the bubble; on error, show the banner (full
|
||||
state machine is Phase 06 — keep it simple-correct here).
|
||||
6. Tune `settings.suggestions` if the real Homelab import revealed better
|
||||
defaults (optional here; Phase 05 owns the chips).
|
||||
|
||||
## UI Verification
|
||||
Against the story's "UI Visualization & Structure": bubbles right/left
|
||||
(brand vs surface, ≥4.5:1 text), avatar 🧠, source chips mono/brand-soft
|
||||
with `source/path` and ellipsis, safe markdown (paste an answer containing
|
||||
`<script>alert(1)</script>` from the mock to prove it's escaped). Chat
|
||||
column 46rem centered. 1280px + 375px screenshot pass.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: retriever ordering/dedup/cap (fake rows), prompt builder (HIGH
|
||||
contains documents + `HIGH`, LOW contains `DEFLECT_MODE` + titles only,
|
||||
persona rules present verbatim), SSE event serialization.
|
||||
- Integration: `/api/chat` against the mock LLM with a seeded temp schema —
|
||||
assert SSE delta sequence, `done` payload (sources non-empty,
|
||||
deflected false), `query_log` row, error event when LLM unreachable.
|
||||
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_chat_rag.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: streamed grounded answer + `kubernetes.md`
|
||||
source chip + button recovery; DB `query_log` assertion; raw SSE shape
|
||||
check via `httpx`.
|
||||
|
||||
## Success criteria
|
||||
- [ ] end-to-end: question → streamed chippy answer citing `kubernetes.md`
|
||||
- [ ] `query_log` row per turn; per-turn log line in stdout
|
||||
- [ ] LLM-down path shows error banner, no stuck button
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] UI verification passed
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(rag): stream grounded chat answers via pgvector cosine retrieval with source citations"
|
||||
```
|
||||
@@ -0,0 +1,66 @@
|
||||
# Phase 04 — Story: Honest Deflection
|
||||
|
||||
**Story:** `.agent/user_stories/honest-deflection.md`
|
||||
**Context:** `.agent/PLAN.md` §4, §6 (honesty gate), §9
|
||||
|
||||
## Goal
|
||||
When retrieval finds nothing relevant, Brain says so — plainly, chippily —
|
||||
and offers real alternatives. No hallucinated confidence.
|
||||
|
||||
## Implementation steps
|
||||
1. `app/api/chat.py` — apply the gate: `best_score < settings.relevance_
|
||||
threshold` ⇒ build LOW prompt (`DEFLECT_MODE`, weak-hit titles only),
|
||||
else HIGH prompt. Set `deflected` on the `done` event + `query_log`.
|
||||
2. Deflection `suggestions[]`: ask `turbo` (same stream) to include 2–3
|
||||
alternative questions; simplest robust approach — have the LLM emit them
|
||||
inline in the answer AND have the server derive 2–3 chips from the
|
||||
weak-hit document titles (deterministic fallback if the model doesn't
|
||||
produce a parsable list). Ship the deterministic title-derived chips as
|
||||
the v1 behavior; model-generated list is a bonus if trivially parseable.
|
||||
3. `frontend/assets/app.js` — on `done.deflected`: add `.is-deflected`
|
||||
class to the bubble, render "Maybe try:" chips below it (same
|
||||
`.suggestion-chip` component; clicking fills the input — full submit
|
||||
behavior lands with Phase 05's chip component; wire what exists).
|
||||
4. `README.md` — document `BOR_RELEVANCE_THRESHOLD` tuning + the
|
||||
deflection behavior in Troubleshooting.
|
||||
|
||||
## UI Verification
|
||||
Against the story: amber bubble (`#fff7e8` bg / `#f59e0b` border) distinct
|
||||
from normal answers; "Maybe try:" chips ≥44px, brand-soft/brand-ink;
|
||||
contrast pairs verified (ink on accent-bg ≥ 9:1, accent-ink ≥ 8:1);
|
||||
chip group has an accessible name; mobile wraps cleanly.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: gate boundary with a fake retriever — score exactly 0.30 → HIGH;
|
||||
0.2999 → LOW; LOW prompt contains `DEFLECT_MODE` + titles, no full docs;
|
||||
HIGH unaffected. Suggestions derivation (2–3, non-empty, derived from
|
||||
titles).
|
||||
- Integration: mock LLM — off-topic question ("sourdough") ⇒ `done`
|
||||
`deflected: true`, `query_log.deflected=true`, weak `top_score` stored;
|
||||
on-topic question ⇒ `deflected: false`.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: off-topic question ⇒ `.is-deflected` bubble
|
||||
matching /haven't done anything like that/i + ≥2 "Maybe try:" chips; chip
|
||||
click behavior; (unit boundary test lives in pytest, not here).
|
||||
|
||||
## Success criteria
|
||||
- [ ] off-topic question never gets a confident fake answer
|
||||
- [ ] deflected bubble visually distinct + alternative chips render
|
||||
- [ ] `query_log.deflected` accurate; threshold env-tunable
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] UI verification passed
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(rag): honest deflection gate with amber UI state and alternative-question chips"
|
||||
```
|
||||
@@ -0,0 +1,59 @@
|
||||
# Phase 05 — Story: Suggestion Chips
|
||||
|
||||
**Story:** `.agent/user_stories/suggestion-chips.md`
|
||||
**Context:** `.agent/PLAN.md` §7 (UI/UX), story file for chip spec
|
||||
|
||||
## Goal
|
||||
Zero-friction onboarding: 3–4 real example questions on first load,
|
||||
clickable → filled → submitted, keyboard-first, mobile-scrollable.
|
||||
|
||||
## Implementation steps
|
||||
1. `app/config.py` — confirm `suggestions` is env-overridable
|
||||
(`BOR_SUGGESTIONS` as JSON list via pydantic-settings) and tune the
|
||||
defaults against the *actually imported* Homelab/Deployments topics
|
||||
(read a sample of `documents` titles; pick questions real answers
|
||||
exist for).
|
||||
2. `app.js` — extract a `renderChips(container, items, {onSelect})` helper;
|
||||
real `<button type="button" class="suggestion-chip" role="listitem">`
|
||||
inside `#suggestions[role="list"]`; onboarding `onSelect` = fill
|
||||
`#message-input` + focus + `composer.requestSubmit()`. Reuse the same
|
||||
helper for deflection chips (Phase 04) with the same submit behavior.
|
||||
3. Empty-state lifecycle: first user message hides `#empty-state` (already
|
||||
done in `addMessage`) — verify chips don't linger in the conversation.
|
||||
4. Mobile CSS check: chip row `nowrap + overflow-x auto` at ≤640px (tokens
|
||||
already exist — verify, don't duplicate).
|
||||
|
||||
## UI Verification
|
||||
Against the story: pills 999px radius, ≥44px, brand-soft/brand-ink (≥6:1),
|
||||
hover/active states; desktop centered wrap vs mobile single scroll row;
|
||||
Tab order reaches chips before the composer input is required; screen
|
||||
reader: group labeled "Suggested questions". Screenshot pass 1280px + 375px.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: `GET /api/suggestions` honors `BOR_SUGGESTIONS` env
|
||||
override (JSON list); default list has ≥3 non-empty strings.
|
||||
- Coverage: **>90%** on `app/` (JS is covered by E2E).
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_suggestion_chips.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: chips render (≥3, role=list); chip click
|
||||
submits (user bubble with exact chip text + mock reply); keyboard Tab+Enter
|
||||
activates; 375px chip row is a horizontal scroll row.
|
||||
|
||||
## Success criteria
|
||||
- [ ] onboarding chips render from the API; click = one-tap question
|
||||
- [ ] keyboard + SR usable; mobile scroll row
|
||||
- [ ] deflection chips share the component + submit behavior
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(ui): onboarding suggestion chips with one-tap submit, keyboard access, and mobile scroll row"
|
||||
```
|
||||
@@ -0,0 +1,64 @@
|
||||
# Phase 06 — Story: Loading Feedback & Progress
|
||||
|
||||
**Story:** `.agent/user_stories/loading-feedback.md`
|
||||
**Context:** `.agent/PLAN.md` §7.4 ("never stale" contract), §9
|
||||
|
||||
## Goal
|
||||
An unambiguous state machine — `idle → thinking → streaming → done |
|
||||
error → idle` — so the user always knows what's happening, and a stale
|
||||
Send button is impossible.
|
||||
|
||||
## Implementation steps
|
||||
1. `app.js` — formalize the state machine (single `setUiState(state)`
|
||||
function driving: typing indicator, send button disabled/spinner/label,
|
||||
`#send-status` live text). Replace ad-hoc busy handling from Phase 03.
|
||||
2. Pre-token: typing indicator (`role="status"`,
|
||||
`aria-label="Brain of Reese is thinking"`); after 10s pre-token, update
|
||||
the label with elapsed seconds (setInterval, cleared on state change).
|
||||
3. Streaming: first `delta` removes the typing indicator and starts
|
||||
appending to the answer bubble; button stays busy.
|
||||
4. Error paths: `{"type":"error"}` SSE event, non-2xx response, or
|
||||
**120s client-side timeout** (clear on first delta) → red banner
|
||||
`role="alert"` ("Try again — if this persists, check the LLM is
|
||||
reachable") + state → idle.
|
||||
5. `prefers-reduced-motion`: CSS already slows animations — verify; add a
|
||||
static fallback for the dots if needed.
|
||||
6. Server: confirm the per-turn log line includes `embed_ms` and
|
||||
`total_ms` (add if Phase 03 omitted it).
|
||||
|
||||
## UI Verification
|
||||
Walk the full state machine by hand (dev server + mock LLM slow path):
|
||||
submit → indicator + "Thinking…" disabled button → live tokens → done
|
||||
(enabled, focused input). Kill the mock mid-stream → banner + recovery.
|
||||
Contrast of disabled button + spinner OK; reduced-motion pass.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: SSE error event serialization; timeout constant
|
||||
exported/testable; (JS logic is E2E-covered).
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_loading_feedback.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping (mock LLM's 3s "pretend to think slowly"
|
||||
warm-up + a fixture that stops the mock): typing indicator visible during
|
||||
pre-token and gone by answer; button disabled→"Thinking…"→enabled "Send";
|
||||
streaming appends (two-timestamp length check); LLM-down ⇒ `role=alert`
|
||||
banner + button recovered.
|
||||
|
||||
## Success criteria
|
||||
- [ ] every in-flight state has a visible indicator; button never zombies
|
||||
- [ ] error + 120s timeout paths both recover cleanly
|
||||
- [ ] reduced-motion respected
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(ui): explicit chat state machine — typing indicator, streaming progress, timeout and error recovery"
|
||||
```
|
||||
@@ -0,0 +1,57 @@
|
||||
# Phase 07 — Story: Responsive, Polished, Accessible UI
|
||||
|
||||
**Story:** `.agent/user_stories/responsive-polish.md`
|
||||
**Context:** `.agent/PLAN.md` §7 (the whole UI/UX strategy)
|
||||
|
||||
## Goal
|
||||
The final visual + accessibility audit pass across chat and Sources. No
|
||||
new features — enforce PLAN §7 end-to-end and fix every deviation.
|
||||
|
||||
## Implementation steps
|
||||
1. Viewport sweep (360 / 375 / 768 / 1280 / 1600) on both pages: fix
|
||||
overflow, pinched columns, dead whitespace. Chat column stays ≤46rem
|
||||
centered; Sources table full-width with horizontal scroll <640px.
|
||||
2. A11y sweep (both pages): landmarks, skip link, labels on every input,
|
||||
`aria-label` on every icon-only control, `:focus-visible` outline on
|
||||
every focusable, `aria-live` regions intact, no contrast <4.5:1
|
||||
(compute, don't eyeball — use the E2E helper).
|
||||
3. Reduced-motion + long-content pass (60-char paths, long answers).
|
||||
4. No-CDN re-verification on **both** pages (extend the integration test
|
||||
to `/sources.html` if it only covers `/`).
|
||||
5. Final README polish pass: screenshots section (optional), quickstart
|
||||
sanity, "Update your documents" workflow prominent.
|
||||
|
||||
## UI Verification
|
||||
This phase IS the verification: the E2E below is the acceptance test.
|
||||
Additionally, manual screenshot pass at 1280px + 375px for both pages,
|
||||
reviewed against PLAN §7.1–7.4 before committing.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: no-CDN check extended to Sources page; health unchanged.
|
||||
- Coverage: **>90%** on `app/` (final state of the whole app).
|
||||
- Whole suite green: `uv run pytest` (unit+integration) — the entire
|
||||
repo must be green at this phase.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_responsive_polish.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: no horizontal overflow at 5 viewports (both
|
||||
pages); chat column capped + centered at 1600px; Sources table ≥80%
|
||||
container at 1280px; landmarks/labels/skip-link sweep; WCAG contrast
|
||||
pairs ≥4.5:1 (computed); reduced-motion honored.
|
||||
|
||||
## Success criteria
|
||||
- [ ] all six mapping tests pass at every viewport
|
||||
- [ ] zero known a11y deviations against PLAN §7.2
|
||||
- [ ] whole pytest suite green + coverage >90%
|
||||
- [ ] README polished
|
||||
- [ ] committed (this commit marks v1.0 feature-complete)
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(ui): responsive + WCAG AA polish pass across chat and sources — v1 feature complete"
|
||||
```
|
||||
@@ -0,0 +1,136 @@
|
||||
# Phase 08 — Story: Dark Tech Theme
|
||||
|
||||
**Story:** `.agent/user_stories/dark-tech-theme.md`
|
||||
**Context:** `.agent/PLAN.md` §7 (UI/UX strategy), §10 (testing)
|
||||
|
||||
## Goal
|
||||
Re-skin the whole UI to a dark, techy, emoji-free look with a subtly
|
||||
animated pure-CSS background — zero behavior or layout changes, WCAG 2.1
|
||||
AA re-proven on the new palette.
|
||||
|
||||
## Dependencies
|
||||
Phases 01–07 (all complete). Independent of 09/10; phase 10's viewer page
|
||||
inherits this theme, so 08 must land first.
|
||||
|
||||
## Locked decisions
|
||||
No anchors changed. Replaces the §7.2 light contrast pairs with the dark
|
||||
palette below (PLAN §7.2 already updated 2026-08-21 with owner
|
||||
permission). No new technology — pure CSS/HTML/inline SVG (A11).
|
||||
|
||||
## Implementation steps
|
||||
1. **Palette swap** — `frontend/assets/styles.css` `:root` tokens (all
|
||||
pairs computed, ≥4.5:1):
|
||||
|
||||
| token | dark value | computed pair |
|
||||
|---|---|---|
|
||||
| `--bg` | `#0a0e17` | ink on bg 16.2:1 |
|
||||
| `--surface` | `#121a2e` | ink on surface 14.5:1 |
|
||||
| `--ink` | `#e8ebf4` | — |
|
||||
| `--ink-soft` | `#9aa4bd` | ink-soft on surface 6.9:1 |
|
||||
| `--line` | `#26304a` | decorative |
|
||||
| `--brand` | `#6d78f2` | **dark ink `--bg` on brand 5.2:1** |
|
||||
| `--brand-soft` | `#232b52` | brand-ink on brand-soft 6.9:1 |
|
||||
| `--brand-ink` | `#a5b4fc` | brand-ink on surface 8.7:1 |
|
||||
| `--accent-bg` | `#2b2110` | accent-ink on accent-bg 9.5:1 |
|
||||
| `--accent-ink` | `#fbbf24` | — |
|
||||
| `--accent-line` | `#f59e0b` | unchanged (8.9:1 on bg) |
|
||||
| `--err-bg` / `--err-ink` | `#2d1318` / `#fca5a5` | 9.1:1 |
|
||||
| `--err-line` | `#ef4444` | 4.6:1 on err-bg (UI boundary) |
|
||||
| `--ok-bg` / `--ok-ink` | `#10241b` / `#6ee7a8` | 10.6:1 |
|
||||
|
||||
Button text is `--bg` (dark) on `--brand` — **never white on brand**
|
||||
(3.7:1, fails). Busy button: keep the `#a5b4fc` background (the
|
||||
`tests/unit/test_frontend_feedback.py` assertion greps this token)
|
||||
with a **dark** arc (`--bg`, 9.7:1). Update derived light-mode values:
|
||||
shadows (black-based, lower alpha), selection, typing dots, chip
|
||||
hover.
|
||||
2. **Emoji purge** — replace every emoji in chrome with inline SVG
|
||||
(`aria-hidden` kept, ~16–20px, `currentColor` where sensible):
|
||||
- `frontend/assets/app.js` (~L116, ~L132): avatars 🧠/🧑 → SVG
|
||||
circuit-node glyph (brain) / minimal silhouette (user) as JS string
|
||||
constants.
|
||||
- `frontend/index.html`: favicon 🧠 data-URI → SVG tech mark (hex +
|
||||
node, brand color on dark, <1 KB), still a `data:` URI;
|
||||
`.brand-mark` 🧠 → same mark; ⚠️ banner icon → SVG triangle; 👋
|
||||
empty state → SVG glyph.
|
||||
- `frontend/sources.html`: favicon, `.brand-mark`, 📂 empty state →
|
||||
SVG marks.
|
||||
3. **Tech details** — mono wordmark with letter-spacing; stat values
|
||||
mono; radii `10px`/`6px`; 1px `--line` borders on cards/bubbles/table;
|
||||
2px gradient hairline (brand→cyan, low alpha) under the sticky header.
|
||||
4. **Animated background (pure CSS, zero JS)** — working recipe:
|
||||
`html { background: var(--bg) }`, `body { background: transparent;
|
||||
position: relative }` (body must not create a stacking context):
|
||||
- `body::before` — fine grid: two `linear-gradient`s (1px lines,
|
||||
`--line` at ~35% alpha), `background-size: 44px 44px`, masked with a
|
||||
radial fade (visible center-top, fading to the edges), animated
|
||||
`background-position` `0 0 → 44px 44px`, 60s linear infinite
|
||||
(seamless loop — the delta equals one cell).
|
||||
- `body::after` — two large soft radial glows: indigo
|
||||
`rgba(109,120,242,0.14)` top-left, cyan `rgba(34,211,238,0.10)`
|
||||
bottom-right; 14s ease-in-out infinite alternate breathing
|
||||
(opacity/scale). No `filter: blur` (perf).
|
||||
- Both: `position: fixed; inset: 0; pointer-events: none; z-index:
|
||||
-1`. Keep glow alpha low — subtle, never competing with text.
|
||||
5. **Reduced motion** — `@media (prefers-reduced-motion: reduce)`:
|
||||
`body::before, body::after { animation: none }` (static grid + glows
|
||||
remain). Existing typing/spinner reduced-motion handling stays.
|
||||
6. **Test updates (behavior unchanged):**
|
||||
- `tests/e2e/test_honest_deflection.py` (~L104): deflection bubble
|
||||
`backgroundColor` assertion `rgb(255, 247, 232)` → `rgb(43, 33,
|
||||
16)`; the border assertion `rgb(245, 158, 11)` is unchanged.
|
||||
- New integration test
|
||||
`tests/integration/test_api.py::test_ui_chrome_has_no_emoji`: GET
|
||||
`/`, `/sources.html`, `/assets/app.js`, `/assets/styles.css` — assert
|
||||
no characters in the emoji code-point set (U+1F300–U+1FAFF,
|
||||
U+2600–U+27BF, U+2B00–U+2BFF, U+FE0F, U+200D, plus the specific
|
||||
glyphs previously used: 🧠 🧑 👋 📂 ⚠️).
|
||||
7. **PLAN.md §7.2** — dark contrast table already applied (2026-08-21,
|
||||
owner permission); no further plan edits in this phase.
|
||||
|
||||
## UI Verification
|
||||
Manual screenshot pass (1280px + 375px, both pages): grid is faint
|
||||
(barely-there), glows soft, no banding; brand button legible (dark
|
||||
text); deflection bubble distinct from normal answers; avatars crisp at
|
||||
16px; reduced-motion preview (DevTools emulation) shows the static
|
||||
background.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: no-CDN check still green on both pages (it covers
|
||||
`/sources.html` since Phase 07); new emoji-guard test above.
|
||||
- **Existing E2E regression check:** after the reskin, run the existing
|
||||
story suites in isolation and confirm they stay green — at minimum
|
||||
`test_chat_rag.py`, `test_honest_deflection.py`, `test_responsive_
|
||||
polish.py` (the contrast helper computes from live styles and must pass
|
||||
on the new palette).
|
||||
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` — **>90%**
|
||||
on `app/`.
|
||||
- `uv run ruff check . && uv run pyright` green.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_dark_tech_theme.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: dark bg + computed contrast pairs ≥4.5:1;
|
||||
no emoji in innerText/outerHTML on both pages; `body::before`/`::after`
|
||||
animate; reduced-motion context → `animation-name: none`; on-topic smoke
|
||||
(stream + chip + button recovery) unchanged; all assets local.
|
||||
|
||||
## Success criteria
|
||||
- [ ] both pages dark; every text pair ≥4.5:1 (computed in E2E)
|
||||
- [ ] zero emoji in chrome (E2E + new integration guard)
|
||||
- [ ] animated background subtle, pure CSS, reduced-motion honored
|
||||
- [ ] layout metrics + chat behavior unchanged (smoke E2E)
|
||||
- [ ] existing story E2E suites still green in isolation
|
||||
- [ ] unit + integration green, coverage >90%, ruff + pyright green
|
||||
- [ ] committed (force-add `.agent/PLAN.md` + this phase record — rule 8)
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A
|
||||
git add -f .agent/PLAN.md .agent/phases/todo/08_story_dark_tech_theme.md
|
||||
git commit --no-gpg-sign -m "feat(ui): dark tech theme — emoji-free chrome, subtle animated CSS background, WCAG AA dark palette"
|
||||
```
|
||||
@@ -0,0 +1,164 @@
|
||||
# Phase 09 — Story: Retrieval Quality — Multi-Format Ingestion + Hybrid Search
|
||||
|
||||
**Story:** `.agent/user_stories/retrieval-quality.md`
|
||||
**Context:** `.agent/PLAN.md` §3 (data flow), §5 (data model), §6 (retrieval), §11 (import)
|
||||
|
||||
## Goal
|
||||
Fix "RAG retrieval is terrible": ingest the full text-format set (not
|
||||
just `.md`), purge vendored-cache junk from the index, and replace
|
||||
pure-cosine top-4 with hybrid (vector + Postgres FTS, RRF-fused)
|
||||
retrieval so name-your-tool questions find the right document.
|
||||
|
||||
## Owner permission (recorded per phase protocol)
|
||||
> "I'm giving you explicit permission to update the locked decisions and
|
||||
> proceed with writing all 3 of these phases" — Reese, 2026-08-21.
|
||||
|
||||
This phase revises anchors **A9** (content scope: `*.md` only →
|
||||
`md, markdown, txt, yaml, yml, json, py` + hidden-dir skip), **A7**
|
||||
(pure-cosine top-4 → hybrid RRF retrieval; the whole-document context
|
||||
contract is preserved), **A8** (gate: LOW only when best cosine <
|
||||
threshold **and** zero FTS hits; threshold re-tuned 0.30 → 0.62 default).
|
||||
`PLAN.md` anchors were updated 2026-08-21 under this permission.
|
||||
|
||||
## Evidence (measured 2026-08-21 against the live KB + `embed` model)
|
||||
- "How did I install gitlab?": the best `gitlab.md` chunk ranks **7th**
|
||||
(cosine 0.804) — outside the top-4 window. Ranks 1–6: a vendored-cache
|
||||
README (`.esphome/.espressif/…/esp-tflite-micro/README.md`, 0.838) and
|
||||
generic templates (`project_readme_template.md`, `templates/…/foobar.
|
||||
md`, 0.81–0.82). The LLM therefore answered from junk docs and honestly
|
||||
reported "no notes on gitlab".
|
||||
- Corpus cosine range: **0.41–0.84** — the old 0.30 gate never
|
||||
discriminated.
|
||||
- FTS: `plainto_tsquery('english','gitlab')` matches **exactly**
|
||||
gitlab.md's 4 chunks and nothing else.
|
||||
- ~470 of 672 indexed docs live under dot-prefixed path components
|
||||
(vendored caches) that A9's exclusion list doesn't cover.
|
||||
|
||||
## Dependencies
|
||||
Phases 01–07 (02 importer, 03 retriever, 04 gate especially).
|
||||
Independent of 08 (backend + E2E only). Phase 10 builds on the new
|
||||
multi-format corpus.
|
||||
|
||||
## Implementation steps
|
||||
1. **Config** (`app/config.py`): `import_extensions` (csv, default
|
||||
`md,markdown,txt,yaml,yml,json,py`; env `BOR_IMPORT_EXTENSIONS`),
|
||||
`hybrid_vector_candidates` (30, `BOR_HYBRID_VECTOR_CANDIDATES`),
|
||||
`hybrid_lexical_candidates` (30, `BOR_HYBRID_LEXICAL_CANDIDATES`),
|
||||
`rrf_k` (60, `BOR_RRF_K`), `relevance_threshold` default **0.62**
|
||||
(re-tuned; env override stays). In `tests/e2e/conftest.py`'s
|
||||
`app_server` fixture set `BOR_RELEVANCE_THRESHOLD=0.30` — the mock's
|
||||
token-overlap embeddings need their own calibration; this keeps
|
||||
stories 02–07's E2E suites green.
|
||||
2. **Chunker** (`app/rag/chunker.py`): add a `chunk_document(content,
|
||||
path)` dispatcher by lowercased suffix + per-format functions —
|
||||
**stdlib only, no new dependencies**:
|
||||
- `yaml`/`yml`: split on `---` document separators and top-level keys
|
||||
(indent-0 `key:` lines); every chunk keeps its key line as anchor.
|
||||
- `json`: `json.dumps(obj, indent=2)` then split at top-level keys
|
||||
(track brace depth); unparseable JSON → paragraph packing.
|
||||
- `py`: stdlib `ast` top-level node line ranges → split at
|
||||
defs/classes; oversized functions fall back to line packing.
|
||||
- `txt`: paragraph packing (reuse `_paragraph_blocks`/`_pack_blocks`).
|
||||
- All formats honor `HARD_MAX_CHARS` (1200 — the aipi ~1024-token
|
||||
request cap) and the target/overlap settings; the `md` path stays
|
||||
byte-for-byte unchanged (existing chunker tests must stay green).
|
||||
3. **Importer** (`app/rag/importer.py`, `scripts/import_docs.py`):
|
||||
extension filter (case-insensitive, config-driven); **skip any path
|
||||
containing a dot-prefixed component** (hidden dirs); chunker dispatch
|
||||
by suffix; `--prune` now also drops docs whose files **no longer
|
||||
match the filter** (this is how the ~470 junk docs leave the index);
|
||||
summary log gains per-format counts
|
||||
(`formats=md:203,yaml:267,…`).
|
||||
4. **Migration `0002_hybrid_retrieval.py`** (alembic):
|
||||
- `chunks.tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('english',
|
||||
content)) STORED` + `CREATE INDEX … USING gin (chunks.tsv)`.
|
||||
- `query_log.fts_hits INT` (nullable; pre-existing rows stay NULL).
|
||||
5. **Retriever** (`app/rag/retriever.py`) — hybrid path:
|
||||
- `retrieve(db, question, question_embedding)`: vector top-N (cosine,
|
||||
as today) ∪ lexical top-N — `to_tsquery('english', <OR-joined
|
||||
stemmed tokens of the question>)` (skip pure-stopword/no-token
|
||||
questions → empty lexical list), ordered by `ts_rank` — fused with
|
||||
RRF: `score = Σ 1/(k + rank)` over the lists a chunk appears in
|
||||
(single-list chunks get one term; k from config).
|
||||
- `RetrievedChunk` gains `cosine` (for the gate) and `fts_hit: bool`
|
||||
alongside `score` (now the fused score, used for ranking);
|
||||
`select_documents` / `weak_hit_titles` keep working off `score`.
|
||||
- Deterministic tie-break: `(−fused, −cosine, document.path,
|
||||
chunk.position)`.
|
||||
6. **Chat flow + gate** (`app/api/chat.py`): pass the raw question into
|
||||
`retrieve`; **LOW only when `best_cosine < threshold and fts_hits ==
|
||||
0`** (`fts_hits` = count of lexical candidates matched); per-turn log
|
||||
line gains `fts_hits=…` (PLAN §9); `query_log` row stores `fts_hits`.
|
||||
7. **Eval script** `scripts/eval_retrieval.py`:
|
||||
`uv run python -m scripts.eval_retrieval "q1" "q2" …` (or
|
||||
`--from-file questions.txt`) — embeds via aipi, runs the hybrid
|
||||
search, prints top-5 docs per question with cosine/fts/fused scores +
|
||||
the gate verdict. Requires `AIPI_KEY` in the environment (same
|
||||
convention as `llm_probe.py`).
|
||||
8. **Re-import the live KB** (one-time; expect ~15–40 min of embedding
|
||||
batches — the importer logs per file):
|
||||
`uv run python -m scripts.import_docs --prune`. Expect ~470
|
||||
hidden-dir docs pruned and ~500 docs indexed (md + new formats). Then
|
||||
verify with the eval script:
|
||||
- "How did I install gitlab?" → top doc `active/container_gitlab/
|
||||
gitlab.md` (the compose yaml should land in the top-2).
|
||||
- "How is my Kubernetes cluster set up?" → kubernetes docs.
|
||||
- "sourdough starter" → LOW (deflect).
|
||||
If the gitlab case isn't #1, iterate the **fusion** (k, candidate
|
||||
counts, token handling) — not the threshold — until it is, and record
|
||||
the final numbers in the phase report.
|
||||
9. **E2E fixtures** (`tests/fixtures/docs/`): add
|
||||
`homelab/container_gitlab/gitlab.md` (H1 "Gitlab", docker install
|
||||
steps, "gitlab" repeated), `homelab/container_gitlab/
|
||||
gitlab-compose.yaml` (`services: gitlab: …`), a `.py` note, a `.json`
|
||||
note, a `.txt` note, and `.hidden/junk.md` (must never be imported).
|
||||
Follow the existing in-process seeding pattern from
|
||||
`tests/e2e/test_import_documents.py`.
|
||||
10. **README**: import workflow section — supported formats, hidden-dir
|
||||
skip, `scripts/eval_retrieval.py`, threshold tuning; note that the
|
||||
Sources count drops after the prune (intended cleanup).
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** chunker per format (yaml top-level + `---` split, json
|
||||
top-level keys + pretty-print + unparseable fallback, py ast split +
|
||||
oversized-func fallback, txt paragraphs, dispatch, 1200-cap) with md
|
||||
output unchanged; importer (hidden-dir skip, extension filter,
|
||||
prune-when-filtered-out, per-format summary); retriever (RRF math:
|
||||
both-lists / one-list / tie-break; OR tsquery construction incl.
|
||||
no-token and stopword-only questions; gate: `cosine ≥ T` → HIGH;
|
||||
`cosine < T` + `fts>0` → HIGH; `cosine < T` + `fts=0` → LOW; boundary
|
||||
exactly `T` → HIGH).
|
||||
- **Integration:** `/api/chat` hybrid against a seeded temp schema —
|
||||
keyword question grounded + `fts_hits` in `query_log`; off-topic
|
||||
deflected with `fts_hits=0`; migration up clean.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` —
|
||||
**>90%** on `app/`.
|
||||
- **No regressions:** existing story E2E suites (02–07) green in
|
||||
isolation after the change (the conftest threshold override is what
|
||||
keeps them green — verify each one).
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_retrieval_quality.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: multi-format fixture import (hidden doc
|
||||
excluded, `/api/docs` counts); "How did I install gitlab?" → grounded,
|
||||
not deflected, gitlab chip, `query_log` row; keyword-only question beats
|
||||
vector ranking (FTS-OR gate end to end); "sourdough" → deflected bubble +
|
||||
≥2 chips.
|
||||
|
||||
## Success criteria
|
||||
- [ ] live eval: "How did I install gitlab?" → `gitlab.md` is the top doc
|
||||
- [ ] zero dot-prefixed path components in `documents` after re-import
|
||||
- [ ] off-topic still deflects; on-topic still grounds (new + existing E2E)
|
||||
- [ ] unit + integration green, coverage >90%, ruff + pyright green
|
||||
- [ ] README documents formats / hidden-dir skip / eval / tuning
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(rag): hybrid FTS+vector retrieval and multi-format ingestion — name-your-tool questions find the right document"
|
||||
```
|
||||
@@ -0,0 +1,101 @@
|
||||
# Phase 10 — Story: Clickable Document Viewer
|
||||
|
||||
**Story:** `.agent/user_stories/document-viewer.md`
|
||||
**Context:** `.agent/PLAN.md` §4 (API), §7 (UI/UX)
|
||||
|
||||
## Goal
|
||||
Every document Brain cites — and every row in the Sources table — opens
|
||||
in the browser: a dark-themed viewer page fed by a stateless content
|
||||
endpoint served from the database.
|
||||
|
||||
## Dependencies
|
||||
08 (dark theme + tokens — the viewer inherits them) and 09 (multi-format
|
||||
corpus; content in every format must be viewable). 01–07 as the base.
|
||||
|
||||
## Locked decisions
|
||||
No anchors changed. Adds one stateless endpoint under `/api` (A10) —
|
||||
PLAN §4 was updated 2026-08-21 with owner permission. No new technology
|
||||
(A11): vanilla HTML/CSS/JS, content rendered by the existing local
|
||||
escape-first markdown renderer.
|
||||
|
||||
## Implementation steps
|
||||
1. **Schema + endpoint** (`app/schemas.py`, `app/api/docs.py`):
|
||||
`DocContent {source, path, title, format, content, indexed_at,
|
||||
chunks}`; `GET /api/documents/content?source=…&path=…` — look up
|
||||
`documents` by `(source, path)` **in the database only** (no
|
||||
filesystem access → no path-traversal surface; `../`-style values
|
||||
simply aren't rows → 404). `format` = lowercased path suffix. 404 →
|
||||
`{detail: "document not found"}`.
|
||||
2. **Shared renderer** — extract the ~60-line escape-first markdown
|
||||
renderer from `frontend/assets/app.js` into `frontend/assets/
|
||||
markdown.js` (local static — no CDN); `index.html` and the new viewer
|
||||
page both load it via relative `<script src>`. If a unit test
|
||||
inspects the renderer inside `app.js`, update it to inspect
|
||||
`markdown.js` (rendering behavior must not change).
|
||||
3. **Viewer page** `frontend/document.html` (+ small
|
||||
`frontend/assets/document.js`): read `source`/`path` query params,
|
||||
fetch the endpoint, render:
|
||||
- Header: back link (SVG arrow + "Sources"; `history.length > 1` →
|
||||
`history.back()`, else navigate to `/sources.html`), `#doc-title`,
|
||||
meta row (`#doc-meta`): source badge, `.format-badge` (mono), mono
|
||||
path, indexed date, chunk count.
|
||||
- Content `#doc-content`: `md`/`markdown` → shared renderer into a
|
||||
≤46rem centered column; any other format → escaped text in
|
||||
`<pre class="doc-raw">` (mono, `overflow-x: auto`, full width).
|
||||
- 404 → `#doc-not-found` card (no emoji) + "Open Sources" link.
|
||||
- A11y: landmarks (`<header>`/`<main>`), skip link, focus moved to
|
||||
main on load, `aria-live="polite"` around the load→content swap,
|
||||
visible labels, `:focus-visible` ring, Phase-08 tokens (all pairs
|
||||
already ≥4.5:1).
|
||||
4. **Chat chips** (`frontend/assets/app.js`):
|
||||
`chip.href = "/document.html?source=" + encodeURIComponent(s.source) +
|
||||
"&path=" + encodeURIComponent(s.path)`; `target="_blank"
|
||||
rel="noopener"`; keep the existing title/aria-label truncation logic.
|
||||
5. **Sources table** (`frontend/assets/sources.js`): path cell →
|
||||
`<a class="doc-link">` to the same URL, `target="_blank"
|
||||
rel="noopener"`, keep the `title` full-path attribute; style:
|
||||
`--brand-ink`, underline on hover/focus.
|
||||
6. **No-CDN integration test:** extend the existing local-asset test to
|
||||
cover `/document.html` (Phase 07 extended it to `/sources.html` —
|
||||
same pattern).
|
||||
7. **README:** UI section — cited documents open in the browser.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** viewer URL builder (query-encoding of paths containing
|
||||
spaces/slashes); 404 mapping; format-from-suffix (incl. `.markdown`
|
||||
and no-suffix fallback).
|
||||
- **Integration:** content endpoint 200 (all fields, seeded doc) / 404
|
||||
(unknown path; traversal-style `path=../../etc/passwd` → 404, no leak);
|
||||
no-CDN on `/document.html`; renderer extraction keeps the existing
|
||||
frontend tests green.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` —
|
||||
**>90%** on `app/`.
|
||||
- **No regressions:** existing story E2E suites (02–08) green in
|
||||
isolation (story 03's chip test asserts chip presence/text, not the
|
||||
href — verify; the new href behavior is covered by this story's suite).
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_document_viewer.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: source chip opens a **new tab** viewer with
|
||||
title/content/format badge; Sources row link opens the viewer (yaml in a
|
||||
`pre`); md `<script>` fixture renders escaped (no execution); unknown doc
|
||||
→ not-found state + Sources link; dark theme + all assets local.
|
||||
|
||||
## Success criteria
|
||||
- [ ] chip click → new tab → full document (any format)
|
||||
- [ ] Sources table path links work
|
||||
- [ ] XSS-safe rendering (escaped) proven in E2E
|
||||
- [ ] 404 state designed, no console crash
|
||||
- [ ] unit + integration green, coverage >90%, ruff + pyright green
|
||||
- [ ] existing story E2E suites green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(ui): clickable document viewer — open any cited document in the browser from chat chips and the sources table"
|
||||
```
|
||||
@@ -0,0 +1,52 @@
|
||||
# Phase 11 — Long Answers (No Truncation)
|
||||
|
||||
**Story:** `.agent/user_stories/long-answers.md`
|
||||
**Context:** owner report 2026-08-22 — "responses keep getting cut off.
|
||||
It should be allowed to respond up to 32768 tokens."
|
||||
|
||||
## Goal
|
||||
Remove the hard 700-token output cap on chat answers; the model may
|
||||
respond up to **32 768** tokens (`BOR_MAX_OUTPUT_TOKENS`, default
|
||||
32 768).
|
||||
|
||||
## Diagnosis
|
||||
`app/rag/llm.py::chat_stream` calls `chat.completions.create(...,
|
||||
max_tokens=700, ...)`. Long answers die mid-sentence at ~700 tokens.
|
||||
|
||||
## Implementation steps
|
||||
1. **Config** (`app/config.py`): `max_output_tokens: int = 32_768` in the
|
||||
RAG-tuning section (env `BOR_MAX_OUTPUT_TOKENS`); document in
|
||||
`.env.example`.
|
||||
2. **LLM client** (`app/rag/llm.py`): `chat_stream` passes
|
||||
`max_tokens=self.settings.max_output_tokens`.
|
||||
3. **E2E mock** (`tests/e2e/mock_llm.py`):
|
||||
- Honor `max_tokens` deterministically: token ≈ whitespace word; if
|
||||
the composed answer is longer, truncate to the first N words.
|
||||
(With the old 700 cap a long answer loses its tail — the mock now
|
||||
behaves like the real endpoint.)
|
||||
- New trigger: user message containing `write a long answer` →
|
||||
deterministic ~4 000-word numbered answer ending in a unique final
|
||||
line (`LONG-ANSWER-END`).
|
||||
- No behavior change for existing (short) answers: they fit under any
|
||||
sane cap.
|
||||
4. **Tests:**
|
||||
- Unit: settings default + env override (`test_config.py`);
|
||||
`chat_stream` forwards the configured `max_tokens` (fake client in
|
||||
`test_llm_client.py`).
|
||||
- E2E: `tests/e2e/test_long_answers.py` per the story mapping.
|
||||
|
||||
## Locked decisions
|
||||
None touched. A5 (aipi endpoint) unchanged; `turbo` accepts the larger
|
||||
cap per owner instruction.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit + integration green; `uv run pytest --cov=app --cov-report=term-missing`
|
||||
**>90%**; E2E in isolation:
|
||||
`uv run pytest tests/e2e/test_long_answers.py -v --no-cov`.
|
||||
- No regressions: `test_chat_rag.py` + `test_chat_api.py` green
|
||||
(short answers unaffected by the mock's new `max_tokens` honoring).
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ tests/ .env.example && git commit --no-gpg-sign -m "fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off"
|
||||
```
|
||||
@@ -0,0 +1,47 @@
|
||||
# Phase 12 — One Header, Same Size Everywhere
|
||||
|
||||
**Story:** `.agent/user_stories/header-consistency.md`
|
||||
**Context:** owner report 2026-08-22 — "the header changes size between
|
||||
sources and chat."
|
||||
|
||||
## Goal
|
||||
The sticky top bar is the **same height on every page**: Chat, Sources,
|
||||
and the document viewer (the page whose back button says "Sources" — the
|
||||
one users compare against chat).
|
||||
|
||||
## Diagnosis (measured 2026-08-22, headless Chromium)
|
||||
- Desktop: `.app-header` = 64px on `/` **and** `/sources.html`
|
||||
(identical markup/CSS); `.doc-header` = **66.375px** (content-sized:
|
||||
`padding-block: 0.7rem` around a 44px back pill + 2-line title block).
|
||||
- Mobile (≤640px): app header 58px; `.doc-header` = **63.19px** (and
|
||||
grows further when the title/meta wrap).
|
||||
So the "sources header" (viewer back button = "Sources") is visibly
|
||||
taller than the chat header — on desktop and mobile.
|
||||
|
||||
## Implementation steps
|
||||
1. **CSS** (`frontend/assets/styles.css`):
|
||||
- `.doc-header { height: var(--header-h); }` — same fixed box as
|
||||
`.app-header`.
|
||||
- `.doc-header-inner { height: 100%; padding-block: 0; }` (center
|
||||
vertically; the 44px pill + title/meta block fit in 64/58px).
|
||||
- Mobile: replace the wrapping rule with `flex-wrap: nowrap`;
|
||||
`.doc-meta { flex-wrap: nowrap; overflow: hidden; }` so the meta
|
||||
row clips (path already ellipsizes) instead of growing the header.
|
||||
`#doc-title` already ellipsizes on one line.
|
||||
2. No HTML/JS changes (ids/classes unchanged → phase-10 tests intact).
|
||||
3. **PLAN §7.1:** note the shared 64px header bar across all three pages.
|
||||
|
||||
## Locked decisions
|
||||
None. A11/Phase-08 tokens untouched (same colors, same hairline).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `tests/e2e/test_header_consistency.py` per the story mapping
|
||||
(desktop 1280px + mobile 375px: three pages, three identical boxes).
|
||||
- Regression: `test_document_viewer.py` green in isolation (viewer
|
||||
content/meta/back link unchanged).
|
||||
- Coverage gate unchanged (no `app/` code touched → stays >90%).
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/assets/styles.css tests/e2e/test_header_consistency.py && git commit --no-gpg-sign -m "fix(ui): uniform header bar height on chat, sources, and the document viewer"
|
||||
```
|
||||
@@ -0,0 +1,51 @@
|
||||
# Phase 13 — Document Back Button Returns to Where You Came From
|
||||
|
||||
**Story:** `.agent/user_stories/document-back-navigation.md`
|
||||
**Context:** owner report 2026-08-22 — "clicking a document from the
|
||||
chat tab pulls up the document correct, but the back button goes back to
|
||||
sources, not the chat."
|
||||
|
||||
## Goal
|
||||
The viewer's back button returns to **the page the document was opened
|
||||
from**: chat chip → Chat; Sources table → Sources.
|
||||
|
||||
## Diagnosis
|
||||
Chips/links open the viewer with `target="_blank"`. In the fresh tab
|
||||
`window.history.length` is 1, so `document.js`'s heuristic
|
||||
(`history.length > 1 ? history.back() : href`) always falls through to
|
||||
the static `href="/sources.html"` — wrong for chat-originated visits.
|
||||
|
||||
## Implementation steps
|
||||
1. **Chat chips** (`frontend/assets/app.js`): `documentUrl(source, path,
|
||||
back = "/")` — appends `&back=<encoded>`; chat passes `"/"`.
|
||||
Existing phase-10 E2E href assertion updates to include
|
||||
`&back=%2F`.
|
||||
2. **Sources links** — unchanged: no `back` param in the URL; the
|
||||
viewer's default target is `/sources.html`, so the phase-10
|
||||
assertion stays green.
|
||||
3. **Viewer** (`frontend/document.js`):
|
||||
- Resolve back target: `back` param wins **only** when it is a
|
||||
same-origin relative URL (starts with `/`, not `//`); otherwise
|
||||
`/sources.html`. (Rejects `https://…`, `//…`, `javascript:…`.)
|
||||
- Set `#doc-back` href + label: `/` → "Chat", `/sources.html` →
|
||||
"Sources", anything else relative → "Back".
|
||||
- Click: deterministic navigation to the resolved target (drop the
|
||||
`history.length` heuristic — both entry points are new tabs, and
|
||||
determinism is what the story demands).
|
||||
- The static `href="/sources.html"` in `document.html` remains the
|
||||
no-JS fallback.
|
||||
4. **E2E:** `tests/e2e/test_document_back_navigation.py` per the story
|
||||
mapping; update `test_document_viewer.py` chip-href assertion.
|
||||
|
||||
## Locked decisions
|
||||
None. A10/A11 untouched (no new endpoint, no new asset).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E in isolation (new file) + `test_document_viewer.py` regression in
|
||||
isolation.
|
||||
- Coverage gate unchanged (frontend-only phase).
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/e2e/ && git commit --no-gpg-sign -m "fix(ui): document viewer back button returns to the page you came from (chat or sources)"
|
||||
```
|
||||
@@ -0,0 +1,60 @@
|
||||
# Phase 14 — Chat Survives a Refresh (localStorage)
|
||||
|
||||
**Story:** `.agent/user_stories/chat-persistence.md`
|
||||
**Context:** owner report 2026-08-22 — "the chat disappears as soon as
|
||||
the browser refreshes. It should use local storage to track previous
|
||||
sessions."
|
||||
|
||||
## Goal
|
||||
The conversation is a **durable local session**: refresh, tab close, or
|
||||
a trip to Sources and back — the chat comes back exactly as left.
|
||||
|
||||
## Design
|
||||
- Storage key `bor.chat.v1` (versioned; a format bump = clean start).
|
||||
- Value: `{v: 1, messages: [{who: "user"|"brain", text, sources?,
|
||||
deflected?, suggestions?}]}` — **raw text** (re-rendered through the
|
||||
existing escape-first markdown on restore; never stored HTML).
|
||||
- Save points: user message on send; brain message on `done` (with
|
||||
sources/deflected/suggestions). A failed turn keeps the user message
|
||||
(the question is not lost) — consistent with the "never stale"
|
||||
contract.
|
||||
- Restore on load: re-render messages (user bubble; brain bubble with
|
||||
source chips, `is-deflected` styling, maybe-try chips), hide the empty
|
||||
state when non-empty.
|
||||
- Size bound: if the serialized state exceeds ~700k chars, drop oldest
|
||||
messages until it fits (localStorage quota is ~5MB; stay well under).
|
||||
- Failure-safe: every `localStorage` access in try/catch (private mode,
|
||||
quota) — chat keeps working with in-memory state only.
|
||||
- **"New chat"** button in the chat header (`#new-chat-btn`, ghost pill
|
||||
like a nav link, ≥44px, accessible name): clears the key + the message
|
||||
list, restores the empty state with suggestions.
|
||||
|
||||
## Implementation steps
|
||||
1. `frontend/assets/app.js`: conversation model + save/restore/clear as
|
||||
above; wire into `handleSend` (push+save user on send; push+save
|
||||
brain on `done`); "New chat" handler.
|
||||
2. `frontend/index.html`: `#new-chat-btn` in `.header-inner` after the
|
||||
nav (chat page only); live-region text reuse for clear confirmation.
|
||||
3. `frontend/assets/styles.css`: `.new-chat-btn` (Phase-08 tokens,
|
||||
focus-visible, ≥44px, hover like `.nav-link`).
|
||||
4. **PLAN §7.5:** new component ids (`#new-chat-btn`); §7.4 note:
|
||||
persistence is local-only (A10 stateless API unchanged — no server
|
||||
session).
|
||||
|
||||
## Locked decisions
|
||||
A10 (stateless API) untouched — persistence is browser-local only.
|
||||
A11 untouched (no library — raw `localStorage` JSON).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `tests/e2e/test_chat_persistence.py` per the story mapping
|
||||
(fresh page fixture = fresh context, so tests are isolated by
|
||||
construction).
|
||||
- Regression: `test_chat_rag.py`, `test_suggestion_chips.py`,
|
||||
`test_loading_feedback.py` green in isolation (fresh contexts start
|
||||
with the empty state exactly as before).
|
||||
- Coverage gate unchanged (frontend-only phase).
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/e2e/test_chat_persistence.py && git commit --no-gpg-sign -m "feat(ui): persist the chat conversation in localStorage — survives refresh and navigation, with a New chat reset"
|
||||
```
|
||||
@@ -0,0 +1,81 @@
|
||||
# Phase 15 — Tune How Brain Answers (Steering Notes)
|
||||
|
||||
**Story:** `.agent/user_stories/steering-notes.md`
|
||||
**Context:** owner report 2026-08-22 — "the answers are off. Add a
|
||||
feature that lets me 'tune' the output if I think an answer isn't quite
|
||||
right. This tuning should be added to the database and read in with the
|
||||
system prompt to steer the replies."
|
||||
|
||||
## Goal
|
||||
A first-class **steering** loop: under any answer, "Tune" → short
|
||||
instruction → stored in Postgres (`steering_notes`) → injected into the
|
||||
system prompt of **every** subsequent turn → observable in the reply.
|
||||
Notes are listed and deletable in a "Tuning" panel.
|
||||
|
||||
## Implementation steps
|
||||
1. **Schema** — `alembic/versions/0003_steering_notes.py`:
|
||||
`steering_notes(id UUID PK, note TEXT NOT NULL, created_at
|
||||
TIMESTAMPTZ NOT NULL DEFAULT now())`; downgrade drops the table.
|
||||
Model `SteeringNote` in `app/models.py`.
|
||||
2. **API** — `app/api/steering.py` (stateless, A10):
|
||||
- `GET /api/steering` → `{notes: [{id, note, created_at}]}` newest
|
||||
first.
|
||||
- `POST /api/steering` `{note}` (trimmed, 1–2000 chars; 422 on
|
||||
empty/over-long) → 201 with the created note.
|
||||
- `DELETE /api/steering/{note_id}` → 204; 404 unknown id.
|
||||
Mount in `app/main.py`; schemas in `app/schemas.py`.
|
||||
3. **Prompt** (`app/rag/prompts.py`):
|
||||
- `build_steering_section(notes: Sequence[str]) -> str` — `""` when
|
||||
empty; else `<tuning>…</tuning>` with numbered notes, capped at
|
||||
`steering_max_chars` (new setting, default 8000; `[…truncated…]`
|
||||
marker). **Both** HIGH and DEFLECT prompts carry it (after
|
||||
`<relevance>…</relevance>`, before `<documents>`/DEFLECT_MODE
|
||||
body). With zero notes the prompt is byte-identical to today.
|
||||
- Preserve the owner's working-tree persona edits (no "you've got
|
||||
this" / no mandated deflection opening — align
|
||||
`tests/unit/test_prompts.py` verbatim check with the current text;
|
||||
record as a PLAN §6 revision).
|
||||
4. **Chat turn** (`app/api/chat.py`): load notes (created_at asc),
|
||||
pass into `plan_turn` → prompts; per-turn log line gains
|
||||
`tuning=N` (PLAN §9).
|
||||
5. **E2E mock** (`tests/e2e/mock_llm.py`): when the system prompt
|
||||
contains `<tuning>`, the composed answer ends with
|
||||
` (tuning: <first note line>)` — makes prompt injection observable
|
||||
in the UI deterministically.
|
||||
6. **UI** (`frontend/index.html`, `app.js`, `styles.css`):
|
||||
- "Tune" button (`.tune-btn`, ghost, ≥44px) in the meta row of every
|
||||
completed brain bubble (deflected included).
|
||||
- Inline `.tune-form`: labeled textarea (maxlength 2000) + Save /
|
||||
Cancel → `POST /api/steering` → success `.tune-saved`
|
||||
(role=status: "Saved — future answers will follow this.") or
|
||||
inline error (role=alert), form kept on failure.
|
||||
- Header `#steering-toggle` "Tuning" + count badge; `#steering-panel`
|
||||
(region) above the messages: notes newest-first, per-note delete
|
||||
(labeled); empty text; count updates on add/delete.
|
||||
- a11y: aria-expanded/controls on the toggle, aria-live announcements
|
||||
for save/delete, focus-visible, Phase-08 tokens (all ≥4.5:1).
|
||||
7. **Docs:** README "Tuning your answers" section; PLAN §4/§5/§6/§9/§7.5
|
||||
+ roadmap rows 11–15.
|
||||
|
||||
## Locked decisions
|
||||
None broken: A10 (stateless endpoints), A13 (alembic), A16 (one E2E
|
||||
suite), A11 (no library). New table + new setting only.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** steering section (empty/one/many/budget truncation), prompt
|
||||
placement in both modes, persona-verbatim check aligned to the
|
||||
owner's current persona.
|
||||
- **Integration:** steering CRUD (201/200/204/404/422, ordering,
|
||||
validation); chat turn with a stored note → fake LLM's captured system
|
||||
prompt contains the note (HIGH **and** LOW); log line `tuning=N`.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing`
|
||||
**>90%**.
|
||||
- **E2E:** `tests/e2e/test_steering.py` per the story mapping.
|
||||
- **Regression:** `test_chat_rag.py`, `test_honest_deflection.py`,
|
||||
`test_retrieval_quality.py` green in isolation (zero-note prompt is
|
||||
byte-identical, so behavior is unchanged until a note exists).
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ alembic/ app/ frontend/ tests/ README.md && git commit --no-gpg-sign -m "feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt"
|
||||
```
|
||||
@@ -0,0 +1,188 @@
|
||||
# Phase 16 — Admin Sign-In (Single-Admin Auth)
|
||||
|
||||
**Story:** `.agent/user_stories/admin-auth.md`
|
||||
**Context:** owner request 2026-08-22 — "add authentication. Only the admin
|
||||
user (there will be only one admin user, me) should be able to tune the
|
||||
outputs and view the entire sources page. Anonymous users should only be
|
||||
able to chat and view relevant documents from that chat." Owner-confirmed
|
||||
choices: **plaintext `BOR_ADMIN_PASSWORD`**; **soft document rule** (anon
|
||||
may open any document by direct URL — the catalog, not the viewer, is
|
||||
gated); **UX**: header Sign in / Sign out, Sources shows a soft gate state
|
||||
(not a redirect), tuning UI completely hidden from anonymous.
|
||||
|
||||
## Objective
|
||||
Single-admin password login backed by a **signed session cookie** (no new
|
||||
services, no new packages, no DB tables): the admin can **tune** (Phase 15
|
||||
steering) and see the **full Sources catalog**; anonymous users keep
|
||||
**chat** + **document viewer**. Revises LOCKED **A10** with owner
|
||||
permission (2026-08-22) — the public API stays stateless; the session cookie
|
||||
is the only session state.
|
||||
|
||||
## Dependencies
|
||||
- All of `01`–`15` (complete). Specifically: `15_steering_notes` (the gated
|
||||
tuning feature), `10_story_document_viewer` (the anonymous document path),
|
||||
`02_story_import_documents` (Sources page being gated), `08_story_dark_tech_theme`
|
||||
(Phase-08 tokens for the login page), `12_header_consistency` (the auth
|
||||
link joins `.header-inner`), `14_chat_persistence` (restored messages must
|
||||
also omit Tune buttons for anonymous).
|
||||
|
||||
## Design
|
||||
- **Mechanism:** Starlette `SessionMiddleware` (ships with FastAPI;
|
||||
`itsdangerous` is already a starlette dependency) → **zero new packages,
|
||||
zero new services** (A12 untouched). Cookie `bor_session`,
|
||||
`same_site="lax"`, `https_only=False` (homelab HTTP — documented in
|
||||
README), max age 12 h sliding (`BOR_SESSION_MAX_AGE`, default 43200).
|
||||
- **Config (fail-loud, A6 spirit):** `admin_password` (`BOR_ADMIN_PASSWORD`,
|
||||
plaintext in gitignored `.env`) + `session_secret`
|
||||
(`BOR_SESSION_SECRET`, random hex; README one-liner
|
||||
`python -c 'import secrets;print(secrets.token_hex(32))'`). Either empty →
|
||||
`create_app()` raises `RuntimeError` naming the missing `BOR_`
|
||||
variable(s) **before the app serves anything**.
|
||||
- **Password check:** `secrets.compare_digest` (constant-time); one admin →
|
||||
one generic 401 `"invalid password"` (no user enumeration).
|
||||
- **API surface** (`app/api/auth.py`, new router):
|
||||
- `POST /api/login` `{password}` → 204 + session `{"admin": true}`; 401 on
|
||||
mismatch (no session set).
|
||||
- `POST /api/logout` → 204; clears the session (idempotent for anon).
|
||||
- `GET /api/whoami` → `{"authenticated": bool, "role": "admin"|"anonymous"}`
|
||||
(drives all UI gating; trivially testable).
|
||||
- `require_admin` dependency in `app/core/auth.py`: reads
|
||||
`request.session`, else **403** `{"detail": "admin only"}`.
|
||||
- **Gated:** `GET /api/docs` + the whole `/api/steering` router
|
||||
(router-level `dependencies=[Depends(require_admin)]`).
|
||||
- **Public (unchanged):** `/api/chat`, `/api/documents/content` (soft rule
|
||||
— note it in the docstring), `/api/suggestions`, `/api/health`, all
|
||||
static pages.
|
||||
- **UI (Phase-08 tokens, WCAG 2.1 AA, ≥44px targets, focus-visible,
|
||||
contrast ≥4.5:1):**
|
||||
- **`/login.html`** + `frontend/assets/login.js`: standard app frame +
|
||||
sticky header (brand + nav, same as other pages — Phase 12 consistency),
|
||||
centered card: `#login-form` with visually-hidden `<label>` +
|
||||
`#login-password` (`type=password`, `autocomplete="current-password"`),
|
||||
submit "Sign in", `#login-error` `role=alert`. On submit →
|
||||
`POST /api/login`; 204 → `location` to `?next` (same-origin `/…` only,
|
||||
default `/sources.html`); 401 → announce + keep form. On load:
|
||||
`GET /api/whoami` already-admin → redirect to `next` immediately.
|
||||
- **Chat (`index.html`, `app.js`):** `.header-inner` gains
|
||||
`#sign-in-link` (`<a>` → `/login.html?next=/sources.html`) and
|
||||
`#sign-out-btn` (`<button>`, aria-label "Sign out") — exactly one
|
||||
visible, decided by `/api/whoami` at load. Anonymous:
|
||||
`#steering-toggle` + `#steering-panel` `hidden`, no steering notes
|
||||
fetch, and **no `.tune-btn` injected on new or Phase-14-restored
|
||||
messages**. Sign out → `POST /api/logout` → `location.reload()`.
|
||||
- **Sources (`sources.html`, `sources.js`):** new `#sources-gate` block
|
||||
(heading "Sign in to view the full catalog", copy, sign-in link →
|
||||
`/login.html?next=/sources.html`, ≥44px). `sources.js` fetches whoami
|
||||
**before** `/api/docs`: anonymous → show gate, hide stat cards +
|
||||
`.sources-shell` table, skip the docs fetch; admin → today's behavior.
|
||||
- **Document viewer:** untouched (anonymous OK).
|
||||
- **Non-goals:** no rate limiting / lockout, no HTTPS enforcement, no
|
||||
multi-user, no per-user history, **no schema change / no migration**.
|
||||
|
||||
## Tasks
|
||||
1. `app/config.py` — add `admin_password: str = ""`, `session_secret: str =
|
||||
""`, `session_max_age: int = 43_200`, `session_cookie: str =
|
||||
"bor_session"` (documented as auth settings).
|
||||
2. `app/core/auth.py` (new) — `ensure_admin_configured(settings) -> None`
|
||||
(RuntimeError naming missing `BOR_ADMIN_PASSWORD` /
|
||||
`BOR_SESSION_SECRET`), `check_password(candidate, expected) -> bool`
|
||||
(`secrets.compare_digest`), `require_admin(request)` FastAPI dependency
|
||||
(403), `sign_in(session)` / `sign_out(session)` helpers.
|
||||
3. `app/api/auth.py` (new) — `login` / `logout` / `whoami` routes;
|
||||
`LoginRequest` schema in `app/schemas.py`; mount in `app/main.py`
|
||||
(with the other API routers, before the static catch-all).
|
||||
4. `app/main.py::create_app` — call `ensure_admin_configured(settings)`
|
||||
first; `app.add_middleware(SessionMiddleware, secret_key=…, max_age=…,
|
||||
same_site="lax")`.
|
||||
5. `app/api/docs.py`, `app/api/steering.py` — `require_admin` on
|
||||
`list_documents` and on the steering router; `get_document_content`
|
||||
stays public (docstring: soft rule, Phase 16).
|
||||
6. `frontend/login.html`, `frontend/assets/login.js`,
|
||||
`frontend/assets/styles.css` — login page per Design (landmarks,
|
||||
skip-link, label, live error, Phase-08 tokens).
|
||||
7. `frontend/index.html`, `frontend/assets/app.js`,
|
||||
`frontend/assets/styles.css` — whoami on load → `isAdmin`; auth link in
|
||||
`.header-inner`; gate steering toggle/panel + tune-button injection
|
||||
(incl. the restore path); sign-out handler.
|
||||
8. `frontend/sources.html`, `frontend/assets/sources.js`,
|
||||
`frontend/assets/styles.css` — `#sources-gate`; whoami-before-docs
|
||||
fetch; hide stats + table for anonymous.
|
||||
9. Docs & config — `.env.example`: `BOR_ADMIN_PASSWORD=`,
|
||||
`BOR_SESSION_SECRET=`, `# BOR_SESSION_MAX_AGE=43200`; README "Admin &
|
||||
sign-in" section (setup one-liner, fail-loud behavior, anon vs admin
|
||||
capability table); **PLAN revisions**: A10 → *revised 2026-08-22
|
||||
(owner permission): single-admin signed-cookie auth — public: chat /
|
||||
documents / suggestions / health; admin-only: docs catalog + steering*;
|
||||
§4 API table (+ `/api/login`, `/api/logout`, `/api/whoami`, auth column);
|
||||
§7.5 new ids (`#sign-in-link`, `#sign-out-btn`, `#login-form`,
|
||||
`#login-password`, `#login-error`, `#sources-gate`); §13 (auth hook now
|
||||
done — keep the multi-user/Valkey line); §12 roadmap row 16.
|
||||
10. Test fixtures — `tests/conftest.py`: `os.environ.setdefault`
|
||||
`BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (known test values) **before**
|
||||
the `app.main` import (same pattern as `BOR_RELEVANCE_THRESHOLD`);
|
||||
`tests/e2e/conftest.py`: set both in `app_server`'s env + export an
|
||||
`ADMIN_PASSWORD` constant for tests; `tests/e2e/auth_helpers.py` (new):
|
||||
`login(page, app_url, password=None, next=None)` performing the real
|
||||
form login (wrong-password variant for the error test).
|
||||
11. E2E regression adaptations — `tests/e2e/test_steering.py` (log in
|
||||
before any tuning), `tests/e2e/test_import_documents.py` +
|
||||
`tests/e2e/test_document_back_navigation.py` (log in for Sources-table
|
||||
assertions), `tests/e2e/test_header_consistency.py` (auth link
|
||||
presence/consistency; height assertions unchanged).
|
||||
|
||||
## Locked decisions
|
||||
- **A10 revised with owner permission (2026-08-22):** single-admin auth
|
||||
via signed cookie; public API endpoints remain stateless. Recorded as a
|
||||
PLAN §2 revision (owner permission noted), not a silent deviation.
|
||||
- **A12 untouched** — no new services (SessionMiddleware/itsdangerous ship
|
||||
with starlette). **A13 untouched** — no migration needed. **A16
|
||||
untouched** — one new story E2E suite + adapted regressions. **A11
|
||||
untouched** — vanilla frontend, no CDN. No other anchor changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit** (`tests/unit/test_auth.py`): `ensure_admin_configured`
|
||||
(missing password / missing secret / both set; plus `create_app` raising
|
||||
with `app.main.settings` monkeypatched invalid); `check_password`
|
||||
(match / mismatch / empty); `require_admin` (admin session passes,
|
||||
anonymous → 403); whoami payload shape; `sign_in`/`sign_out` session
|
||||
semantics.
|
||||
- **Integration** (`tests/integration/test_auth_api.py`): wrong password →
|
||||
401 and subsequent `/api/steering` still 403; correct → 204 + cookie →
|
||||
whoami admin, `/api/docs` 200, steering GET/POST/DELETE 201/200/204;
|
||||
logout → 403 again; **anonymous** `/api/documents/content` still 200
|
||||
(seeded doc) and `/api/chat` still streams (regression guard); `/api/whoami`
|
||||
anonymous shape.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` **>90%**
|
||||
on `app/`.
|
||||
- **E2E:** `tests/e2e/test_admin_auth.py` — the six scenarios in the story's
|
||||
Playwright Mapping Rule.
|
||||
- **Regression (each green in isolation):** `test_steering.py`,
|
||||
`test_import_documents.py`, `test_document_back_navigation.py`,
|
||||
`test_header_consistency.py`, `test_chat_rag.py`, `test_document_viewer.py`.
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run uvicorn app.main:app` with `BOR_ADMIN_PASSWORD` unset fails at
|
||||
startup naming the missing variable(s); with both auth vars set it
|
||||
serves.
|
||||
- [ ] Anonymous: `POST /api/chat` streams; `GET /api/docs` → 403;
|
||||
`GET /api/documents/content?source=…&path=…` → 200; `/sources.html`
|
||||
shows `#sources-gate`; chat UI has no Tune button / Tuning panel;
|
||||
header shows Sign in.
|
||||
- [ ] `POST /api/login` (correct) → 204 + cookie → `/api/whoami`
|
||||
`{"authenticated": true, "role": "admin"}` → Sources + tuning work →
|
||||
`POST /api/logout` → 403 again.
|
||||
- [ ] `uv run pytest --cov=app --cov-report=term-missing` > 90%;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `uv run pytest tests/e2e/test_admin_auth.py -v --no-cov` green in
|
||||
isolation; all regression suites above green in isolation.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): login page — landmarks,
|
||||
labeled control, contrast ≥4.5:1, focus-visible, `role=alert` error,
|
||||
centered card in the standard frame, no CDN tags.
|
||||
- [ ] One `--no-gpg-sign` commit (below); `.agent/phases/todo/16_admin_auth.md`
|
||||
moved to `.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ frontend/ tests/ README.md .env.example && git commit --no-gpg-sign -m "feat(auth): single-admin password login (signed cookie) — gate tuning + Sources catalog, keep chat and document viewer public"
|
||||
```
|
||||
@@ -0,0 +1,232 @@
|
||||
# Phase 17 — Model "Thinking" in the Chat UI
|
||||
|
||||
**Story:** `.agent/user_stories/thinking-display.md` (created by task 04)
|
||||
**Context:** PLAN §3/§4 (SSE chat transport, A15), §6 (locked prompt),
|
||||
§7.4/§7.5 (feedback contract + component inventory), §9 (per-turn log line);
|
||||
`app/rag/llm.py` (streaming client), `app/api/chat.py` (SSE mapping),
|
||||
`frontend/assets/app.js` (turn handler + phase-14 persistence),
|
||||
`tests/e2e/mock_llm.py` (deterministic mock).
|
||||
|
||||
## Objective
|
||||
The web interface **supports and shows the model's "thinking"**: the aipi
|
||||
`turbo` model streams its reasoning in `delta.reasoning_content` chunks
|
||||
before the answer (verified live 2026-08-23 — the app currently discards
|
||||
those tokens, so the user stares at "Thinking…" with no visibility). This
|
||||
phase plumbs reasoning through the SSE contract as a new `thinking` event
|
||||
type and renders it in a **collapsible "Thinking" block** above the answer
|
||||
bubble — streaming open, auto-collapsing when the answer starts,
|
||||
user-toggleable afterwards, and persisted with the message (phase 14).
|
||||
Models/turns that emit no reasoning render exactly as before.
|
||||
|
||||
## Dependencies
|
||||
- All of `01`–`16` (complete). Specifically: `03_story_chat_rag` (the SSE
|
||||
turn pipeline this extends), `06_story_loading_feedback` (the
|
||||
`idle → thinking → streaming → done | error → idle` state machine +
|
||||
120s guard the thinking display plugs into), `14_chat_persistence`
|
||||
(localStorage record shape gains an optional `thinking` field),
|
||||
`08_story_dark_tech_theme` (Phase-08 design tokens for the block),
|
||||
`16_admin_auth` (chat stays public — no gating change).
|
||||
|
||||
## Verified facts (probe, 2026-08-23, live aipi endpoint)
|
||||
- `POST /v1/chat/completions` with `stream=True` against `turbo` emits
|
||||
`choices[0].delta.reasoning_content` chunks **before** the first
|
||||
`delta.content` chunk (deepseek/litellm wire convention). No request-side
|
||||
flag is needed — the model thinks on its own; how much it thinks is the
|
||||
model's call.
|
||||
- Reasoning counts against `max_tokens`: at `max_tokens=300` a probe
|
||||
produced 300 reasoning tokens and **no answer content**. With the locked
|
||||
default `BOR_MAX_OUTPUT_TOKENS=32768` there is ample headroom, but an
|
||||
answer can in principle be empty — the UI must handle
|
||||
thinking-without-answer gracefully (existing empty-answer fallback).
|
||||
- `openai` SDK 2.54 (this repo's version) preserves unknown delta fields:
|
||||
`ChatCompletionChunk.model_validate(...)` keeps `reasoning_content` in
|
||||
`model_extra`, reachable via `getattr(delta, "reasoning_content", None)`.
|
||||
The design therefore needs no raw-HTTP parsing.
|
||||
|
||||
## Design
|
||||
- **SSE contract (PLAN §4 extension, owner permission 2026-08-23 = this
|
||||
request):** new event type `{"type":"thinking","text":"…"}`. Frames
|
||||
arrive before `delta` frames in practice (the model reasons first); the
|
||||
client must tolerate a late/interleaved `thinking` event defensively
|
||||
(append to the block, never reopen it once the answer started). The
|
||||
`done` event shape is **unchanged** (`deflected`, `sources`,
|
||||
`suggestions`) — thinking text never needs to travel again on `done`.
|
||||
- **Backend:**
|
||||
- `app/rag/llm.py` — new `StreamPiece` (frozen dataclass:
|
||||
`kind: "content" | "thinking"`, `text: str`); `chat_stream` yields
|
||||
`StreamPiece` instead of `str`. Per chunk: `delta.reasoning_content`
|
||||
(verified aipi field) → thinking piece; fallback `delta.reasoning`
|
||||
(future-proofing, same getattr pattern); `delta.content` → content
|
||||
piece. A chunk carrying both yields thinking **before** content.
|
||||
`LLMError` wrapping and generation params (model, `temperature=0.4`,
|
||||
`max_tokens`, `stream=True`) unchanged.
|
||||
- `app/schemas.py` — `ChatThinkingEvent` (`type="thinking"`, `text`),
|
||||
sibling of `ChatErrorEvent`/`ChatDoneEvent`.
|
||||
- `app/config.py` — `stream_thinking: bool = True`
|
||||
(`BOR_STREAM_THINKING`; `0`/`false` disables) — operator kill-switch.
|
||||
When off, thinking pieces are still **counted** for the log line but
|
||||
never emitted. (Reasoning is otherwise always on: the model emits it,
|
||||
and the whole point of this phase is to show it.)
|
||||
- `app/api/chat.py` — maps pieces to `thinking`/`delta` events;
|
||||
accumulates `thinking_chars` per turn; per-turn log line (PLAN §9)
|
||||
gains `thinking_chars=N` inserted immediately before `total_ms=N`.
|
||||
No schema change (A13 untouched), no new packages (A12 untouched).
|
||||
- **Frontend (`frontend/assets/app.js`, `styles.css`):**
|
||||
- **Block DOM** (dynamic — `index.html` unchanged): inside `.msg-body`,
|
||||
**before** `.bubble`:
|
||||
```html
|
||||
<details class="thinking" open>
|
||||
<summary>Thinking</summary>
|
||||
<div class="thinking-text"></div>
|
||||
</details>
|
||||
```
|
||||
`renderStoredMessage` renders the same block **collapsed** for
|
||||
restored messages carrying `thinking`.
|
||||
- **Turn handler** (`handleSend`): new turn-local state
|
||||
`thinkingAcc`, `sawThinking`, `sawDone`.
|
||||
- First `thinking` event: `clearTurnTimeout()` (the stream is alive —
|
||||
the 120s pre-token guard also clears on the first `delta`, as
|
||||
today); if no wrap exists yet, create it (`addMessage("brain", "")`);
|
||||
`ensureThinkingBlock(wrap)` (idempotent; creates the open
|
||||
`<details>` and returns it); the **typing indicator is removed**
|
||||
(the live block replaces it as the visible "thinking" feedback —
|
||||
the UI state stays `thinking`: button still disabled, label
|
||||
"Thinking…", `#send-status` still "Brain of Reese is thinking" —
|
||||
no state machine change); render
|
||||
`.thinking-text.innerHTML = renderMarkdown(thinkingAcc)` (escape-
|
||||
first renderer — XSS-safe; the model's scratchpad may contain
|
||||
markdown-ish formatting); while the block is open, pin
|
||||
`.thinking-text` scrolled to the bottom on each update.
|
||||
- First `delta`: if the UI state is still `thinking`, transition to
|
||||
`streaming` (replaces today's `if (!wrap)`-only transition so the
|
||||
label/status flip even when thinking created the wrap first);
|
||||
`closeThinkingBlock(wrap)` (idempotent — never reopens a block once
|
||||
the answer started); existing delta logic unchanged.
|
||||
- `done`: `sawDone = true`; existing logic (deflected styling, source
|
||||
chips, tune button) unchanged; the thinking block is closed if open;
|
||||
**thinking-without-answer:** when `acc` is empty but `sawThinking`,
|
||||
the bubble receives the existing empty-answer fallback string and
|
||||
that is what gets persisted (what the user saw is what is stored);
|
||||
persistence record gains `thinking: thinkingAcc` (only when
|
||||
non-empty — `bor.chat.v1` shape: brain messages may carry an
|
||||
optional `thinking` field; **no version bump**: old records without
|
||||
it restore exactly as before).
|
||||
- **Stream-drop guard (new, required now that streams run longer):**
|
||||
after `readSSE` completes, if `!sawDone && !aborted` and at least
|
||||
one `thinking`/`delta` frame arrived → `setUiState(error, "The
|
||||
stream ended before my answer finished — try again?")` (previously a
|
||||
severed stream settled silently into idle with a half bubble). Zero
|
||||
frames + no wrap keeps today's "came back empty" fallback.
|
||||
- **No live region on the thinking text** (it is a scratchpad —
|
||||
announcing every chunk would be hostile to screen readers); the
|
||||
existing `#send-status` region + native `<details>` open/closed
|
||||
announcements cover accessibility.
|
||||
- **Styling (Phase-08 tokens, WCAG AA):** `details.thinking` —
|
||||
`background: var(--surface)`, `border: 1px solid var(--line)`,
|
||||
`border-left: 3px solid var(--brand-soft)`,
|
||||
`border-radius: var(--radius-sm)`, `margin-bottom: 0.5rem`,
|
||||
`overflow: hidden`. `summary` — flex row, `padding: 0.5rem 0.75rem`,
|
||||
`min-height: 44px` (mobile touch target), `color: var(--brand-ink)`
|
||||
(8.7:1 on surface), `font-size: 0.9rem`, `cursor: pointer`,
|
||||
`list-style: none` (+ `::-webkit-details-marker {display:none}`),
|
||||
CSS chevron `::before` (`"▸"`, rotates 90° when open, 0.15s
|
||||
transform disabled under `prefers-reduced-motion`), `:focus-visible`
|
||||
3px `var(--brand)` outline offset 2px. `.thinking-text` —
|
||||
`padding: 0 0.75rem 0.75rem`, `color: var(--ink-soft)` (6.9:1 on
|
||||
surface), `font-size: 0.875rem`, `line-height: 1.55`,
|
||||
`max-height: 320px`, `overflow-y: auto`; its inner paragraphs get
|
||||
reduced margins. No background animation — `prefers-reduced-motion`
|
||||
respected by construction (only the chevron transition, gated).
|
||||
- **E2E mock (`tests/e2e/mock_llm.py`):** deterministic thinking via a new
|
||||
user-message trigger `THINKING_TRIGGER = "think out loud"` (same
|
||||
convention as the existing `"write a long answer"` /
|
||||
`"pretend to think slowly"` triggers). When the trigger is present the
|
||||
mock streams ~800 chars of deterministic `reasoning_content` chunks
|
||||
(a fixed "Step 1… Step 4…" scratchpad) **before** the normal
|
||||
`content` chunks; the non-streaming path includes `reasoning_content`
|
||||
in the message. Without the trigger the mock is byte-identical to
|
||||
today — every existing story suite is unaffected. (The suite is
|
||||
mock-only by design: `E2E_REAL_LLM=1` against the real `turbo` — which
|
||||
thinks on every turn — would break the "no thinking block" regression
|
||||
test. Note that in the file header.)
|
||||
- **Non-goals:** no thinking-budget/effort request parameters (the model
|
||||
decides; `BOR_MAX_OUTPUT_TOKENS` already bounds total output); no DB
|
||||
schema change (thinking is not stored server-side — `query_log` keeps
|
||||
its shape; only the log line counts chars); no in-UI toggle (the
|
||||
kill-switch is `BOR_STREAM_THINKING`); no `done`-event change; no
|
||||
changes to steering/suggestions/viewer.
|
||||
|
||||
## Tasks
|
||||
1. `01_backend_thinking_stream.md` — `StreamPiece` in the LLM client,
|
||||
`thinking` SSE event end-to-end (schema, settings, chat mapping,
|
||||
log line), backend tests.
|
||||
2. `02_frontend_thinking_block.md` — the collapsible thinking block in
|
||||
`app.js`/`styles.css` (streaming, auto-collapse, persistence,
|
||||
stream-drop guard), frontend unit pins.
|
||||
3. `03_e2e_mock_and_story_suite.md` — mock thinking trigger + the
|
||||
dedicated `test_thinking_display.py` Playwright suite (5 scenarios),
|
||||
run in isolation.
|
||||
4. `04_story_docs_plan_commit.md` — user story file, README section,
|
||||
`.env.example`, PLAN §2/§4/§7/§9/§12 revisions (owner permission
|
||||
recorded), the single atomic commit, phase move to complete/.
|
||||
|
||||
## Locked decisions
|
||||
- **A15 extended with owner permission (2026-08-23):** the SSE contract
|
||||
gains the `thinking` event type; `delta` + `done` shapes unchanged.
|
||||
Recorded as a PLAN §4 revision (owner permission noted), not a silent
|
||||
deviation.
|
||||
- **A5 untouched** — no new models/params sent to aipi; we only *read* a
|
||||
field the model already emits. **A11 untouched** — vanilla JS/CSS, no
|
||||
CDN (native `<details>/<summary>`). **A12/A13 untouched** — no new
|
||||
services, no migration. **A16 untouched** — one new story E2E suite +
|
||||
unit/integration extensions. **A10 untouched** — chat stays public;
|
||||
thinking is visible to everyone (homelab scope). No other anchor
|
||||
changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_llm_client.py` (StreamPiece mapping: content,
|
||||
`reasoning_content`, `reasoning` fallback, both-in-one-chunk ordering,
|
||||
empty-skip, failure wrapping), `tests/unit/test_sse_events.py`
|
||||
(thinking frame shape), `tests/unit/test_config.py` (default + env
|
||||
parse of `stream_thinking`), `tests/unit/test_frontend_feedback.py` +
|
||||
`tests/unit/test_chat_persistence.py` (source-level pins: thinking event
|
||||
handling, block markers, auto-collapse, `sawDone` guard, persisted
|
||||
`thinking` field, `.thinking` CSS rules).
|
||||
- **Integration:** `tests/integration/test_chat_api.py` — `FakeRagLLM`
|
||||
gains optional thinking; thinking frames precede all delta frames and
|
||||
reassemble; `BOR_STREAM_THINKING=0` suppresses thinking frames while
|
||||
deltas are unchanged; existing suites stay green.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing`
|
||||
**>90%** on `app/`.
|
||||
- **E2E:** `tests/e2e/test_thinking_display.py` — five scenarios (see
|
||||
task 03), **mock-only**, green **in isolation**
|
||||
(`uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (unit + integration);
|
||||
`uv run pytest --cov=app --cov-report=term-missing` > 90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
green in isolation (prereq: `podman compose up -d db`).
|
||||
- [ ] Regression suites green in isolation: `test_chat_rag.py`,
|
||||
`test_loading_feedback.py`, `test_chat_persistence.py`,
|
||||
`test_honest_deflection.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual live check (dev server, real aipi): a chat turn shows the
|
||||
thinking block streaming, collapsing when the answer starts,
|
||||
toggleable afterwards; with `BOR_STREAM_THINKING=0` no block.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): block uses Phase-08 tokens,
|
||||
all text contrast ≥4.5:1, summary is a real focusable control with
|
||||
≥44px target, `prefers-reduced-motion` respected, no CDN tags,
|
||||
chat column still 46rem.
|
||||
- [ ] `.agent/user_stories/thinking-display.md` exists; PLAN §2/§4/§7.4/
|
||||
§7.5/§9/§12 carry the revision notes (owner permission
|
||||
2026-08-23).
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/17_thinking_display/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ frontend/ tests/ README.md .env.example && git commit --no-gpg-sign -m "feat(chat): stream model thinking over SSE and show it in a collapsible block"
|
||||
```
|
||||
@@ -0,0 +1,120 @@
|
||||
# Task 01 — Backend: stream thinking pieces end-to-end
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agent/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
The chat pipeline carries the model's reasoning: `LLMClient.chat_stream`
|
||||
yields typed `StreamPiece`s (content vs thinking), and `POST /api/chat`
|
||||
emits a new `{"type":"thinking","text":…}` SSE event ahead of the
|
||||
`delta` events — counted in the per-turn log line, suppressible via
|
||||
`BOR_STREAM_THINKING=0`.
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py`
|
||||
- Add a module-level frozen dataclass `StreamPiece` with fields
|
||||
`kind: Literal["content", "thinking"]` and `text: str` (export it in
|
||||
the module — `from app.rag.llm import StreamPiece` must work).
|
||||
- Change `chat_stream` to return `AsyncIterator[StreamPiece]` (was
|
||||
`AsyncIterator[str]`). Per streamed chunk with at least one choice:
|
||||
read `delta = chunk.choices[0].delta`, then
|
||||
- `reasoning = getattr(delta, "reasoning_content", None)` (the field
|
||||
aipi's `turbo` emits — verified live 2026-08-23; the openai SDK
|
||||
keeps unknown delta fields in `model_extra`, so `getattr` is the
|
||||
right accessor); if falsy fall back to
|
||||
`getattr(delta, "reasoning", None)` (future-proofing).
|
||||
If truthy, `yield StreamPiece("thinking", reasoning)`.
|
||||
- If `delta.content` is truthy, `yield StreamPiece("content", delta.content)`.
|
||||
A chunk carrying both fields yields the thinking piece **first**.
|
||||
Chunks with no choices are skipped, as today. The
|
||||
`LLMError` wrap (`except Exception → LLMError`) and the generation
|
||||
params (`model`, `temperature=0.4`, `max_tokens=
|
||||
settings.max_output_tokens`, `stream=True`) are unchanged.
|
||||
- Update the module + method docstrings: name the verified wire
|
||||
convention (`delta.reasoning_content` before `delta.content`) and
|
||||
that reasoning counts against `max_tokens` (an answer can in
|
||||
principle be empty — the UI handles that).
|
||||
2. `app/schemas.py` — add `ChatThinkingEvent(BaseModel)`:
|
||||
`type: str = "thinking"`, `text: str`, with a docstring referencing
|
||||
the phase-17 PLAN §4 extension (sibling of `ChatErrorEvent`).
|
||||
3. `app/config.py` — in the LLM section add
|
||||
`stream_thinking: bool = True` with a comment: operator kill-switch
|
||||
for the `thinking` SSE events (phase 17); when off, pieces are still
|
||||
counted for the log line but never emitted. Env: `BOR_STREAM_THINKING`
|
||||
(`0`/`false` → False — pydantic-settings parses bools).
|
||||
4. `app/api/chat.py`
|
||||
- Import `ChatThinkingEvent` and `StreamPiece` (for typing).
|
||||
- In the step-3 streaming loop, replace
|
||||
`for piece in llm.chat_stream(messages): yield sse_event({"type": "delta", "text": piece})`
|
||||
with a loop over `StreamPiece`s:
|
||||
- `kind == "thinking"`: accumulate `thinking_chars += len(piece.text)`;
|
||||
emit `sse_event(ChatThinkingEvent(text=piece.text).model_dump())`
|
||||
**only when** `settings.stream_thinking` is true (`settings` is
|
||||
already in scope from step 2).
|
||||
- `kind == "content"`: emit the `delta` event exactly as today.
|
||||
- Per-turn log line (PLAN §9): insert `thinking_chars=%d`
|
||||
immediately before `total_ms=%d` (add `thinking_chars` to the
|
||||
`logger.info` args). Keep every existing field and order.
|
||||
- Update the module docstring: the turn now streams `thinking` events
|
||||
(phase 17, PLAN §4 extension) ahead of `delta` events, with
|
||||
`BOR_STREAM_THINKING=0` suppressing them.
|
||||
5. `.env.example` — add `BOR_STREAM_THINKING=1` with a short comment
|
||||
(stream the model's thinking as `thinking` SSE events; set `0` to
|
||||
suppress).
|
||||
|
||||
## Testing & Quality
|
||||
- `tests/unit/test_llm_client.py`
|
||||
- Extend the `_chunk` helper: `_chunk(content=…, reasoning=None)` —
|
||||
build the delta `SimpleNamespace` with `reasoning_content` present
|
||||
only when `reasoning is not None` (mirror the real wire: the field
|
||||
exists only when the model sends it).
|
||||
- Adapt the existing chat-stream tests to the new yield type: `_collect`
|
||||
returns pieces; assertions compare
|
||||
`(p.kind, p.text)` pairs (or map to text where the old intent was
|
||||
"deltas in order").
|
||||
- New tests:
|
||||
- `test_chat_stream_maps_reasoning_content_to_thinking_pieces`
|
||||
- `test_chat_stream_falls_back_to_reasoning_field`
|
||||
- `test_chat_stream_thinking_yields_before_content_in_chunk` (one
|
||||
chunk with both fields → thinking piece first)
|
||||
- `test_chat_stream_interleaved_thinking_and_content_order_preserved`
|
||||
(thinking chunks → content chunks → order of the piece sequence
|
||||
matches the chunk order)
|
||||
- Existing `test_chat_stream_skips_empty_deltas_and_choiceless_chunks`
|
||||
and `test_chat_stream_wraps_failures_as_llm_error` keep passing
|
||||
(adapted to pieces where needed).
|
||||
- `tests/unit/test_sse_events.py` — `test_thinking_frame_serializes_exactly`:
|
||||
`sse_event(ChatThinkingEvent(text="…").model_dump())` round-trips to
|
||||
`{"type": "thinking", "text": "…"}`.
|
||||
- `tests/unit/test_config.py` — `stream_thinking` defaults to `True`;
|
||||
`Settings(_env_file=None, stream_thinking=False)` / env `0` parse.
|
||||
- `tests/integration/test_chat_api.py`
|
||||
- `FakeRagLLM` — add `thinking: str = ""`; its `chat_stream` yields
|
||||
`StreamPiece("thinking", …)` slices of `self.thinking` (12-char
|
||||
slices, same cadence as content) **before** the content pieces; with
|
||||
the default `thinking=""` it yields content-only pieces (today's
|
||||
behavior, new yield type).
|
||||
- New `test_chat_streams_thinking_before_deltas` (override `get_llm`
|
||||
with `FakeRagLLM(thinking="…")`, assert: ≥1 `thinking` frame, every
|
||||
`thinking` frame precedes every `delta` frame, thinking text
|
||||
reassembles to the input, `done` still last, sources unchanged).
|
||||
- New `test_chat_thinking_suppressed_when_disabled` — monkeypatch
|
||||
`chat_api.get_settings` to a `Settings(_env_file=None, …)` with
|
||||
`stream_thinking=False` **and the same `relevance_threshold` the
|
||||
module/conftest already use** (read it from the existing settings —
|
||||
don't hardcode a different gate), then: no `thinking` frames, deltas
|
||||
identical to the thinking-free case. Restore via `monkeypatch`.
|
||||
- Coverage: **>90%** on the touched `app/` modules
|
||||
(`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_llm_client.py tests/unit/test_sse_events.py tests/unit/test_config.py tests/integration/test_chat_api.py -v --no-cov`
|
||||
green (integration needs `podman compose up -d db`).
|
||||
- [ ] `uv run pytest` fully green; coverage > 90%;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `curl -N -X POST localhost:8000/api/chat …` against the dev server
|
||||
+ real aipi shows `thinking` frames before `delta` frames (and the
|
||||
per-turn log line carries `thinking_chars=`); with
|
||||
`BOR_STREAM_THINKING=0` no `thinking` frames.
|
||||
- [ ] The frontend (phase-16 state, `app.js` untouched) still works:
|
||||
unknown `thinking` frames are ignored by its `readSSE` handler
|
||||
(it only branches on `delta`/`done`/`error`) — no chat regression.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Task 02 — Frontend: the collapsible Thinking block
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agent/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
`index.html` chat shows the model's thinking: a `<details class="thinking">`
|
||||
block above the answer bubble streams open while `thinking` events
|
||||
arrive, auto-collapses when the first answer token lands, stays
|
||||
user-toggleable, and survives reloads (phase-14 persistence gains an
|
||||
optional `thinking` field). Plus a stream-drop guard for severed
|
||||
streams.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js` — turn handler (`handleSend`):
|
||||
- New turn-local vars next to `acc`/`aborted`: `let thinkingAcc = "";
|
||||
let sawThinking = false; let sawDone = false;`
|
||||
- **Two small helpers** (near `addTyping`/`removeTyping`):
|
||||
- `ensureThinkingBlock(wrap)` — returns the existing
|
||||
`.thinking` `details` in `wrap`, or creates one:
|
||||
`details.thinking` with `open = true`, containing
|
||||
`<summary>Thinking</summary>` + `<div class="thinking-text">`,
|
||||
inserted into `.msg-body` **before** the `.bubble`
|
||||
(`body.insertBefore(el, body.querySelector(".bubble"))`).
|
||||
- `closeThinkingBlock(wrap)` — sets `.open = false` on
|
||||
`wrap.querySelector(".thinking")` when present (no-op otherwise).
|
||||
- **`readSSE` onEvent** — new branch `ev.type === "thinking"`:
|
||||
- `thinkingAcc += ev.text || ""; sawThinking = true;`
|
||||
- `clearTurnTimeout()` (the stream is alive — same role the first
|
||||
`delta` already plays; keep the existing delta-side clear as is).
|
||||
- If `!wrap`: `wrap = addMessage("brain", "");` — **do not call
|
||||
`setUiState` here**: the UI state stays `thinking` (button still
|
||||
disabled/"Thinking…", `#send-status` still
|
||||
"Brain of Reese is thinking" — all still true); remove the typing
|
||||
indicator via `removeTyping()` since the live block replaces it as
|
||||
the visible feedback.
|
||||
- `const block = ensureThinkingBlock(wrap);` then
|
||||
`block.querySelector(".thinking-text").innerHTML =
|
||||
renderMarkdown(thinkingAcc);` (escape-first renderer — XSS-safe;
|
||||
the scratchpad may contain markdown-ish text). While
|
||||
`block.open`, pin the text to the bottom
|
||||
(`textEl.scrollTop = textEl.scrollHeight`) and
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" })`.
|
||||
- **`delta` branch** — replace the `if (!wrap) { setUiState(streaming);
|
||||
wrap = addMessage("brain",""); }` transition with:
|
||||
`if (uiState === UI_STATE.thinking) setUiState(UI_STATE.streaming);`
|
||||
then `if (!wrap) wrap = addMessage("brain", "");` then
|
||||
`closeThinkingBlock(wrap);` (idempotent — never reopens once the
|
||||
answer started; a late/interleaved `thinking` event appends to the
|
||||
closed block without reopening it). Rest of the delta logic
|
||||
unchanged.
|
||||
- **`done` branch** — set `sawDone = true;` and
|
||||
`closeThinkingBlock(wrap)`; existing logic (deflected class,
|
||||
maybe-try, sources, tune button) unchanged. Then:
|
||||
- **thinking-without-answer:** `const finalText = acc ||
|
||||
(sawThinking ? EMPTY_ANSWER_FALLBACK : "");` where
|
||||
`EMPTY_ANSWER_FALLBACK` is the existing fallback string
|
||||
("Hmm — that came back empty. Ask me again?" — hoist it to a
|
||||
const shared by the `done` branch and the post-stream `!wrap`
|
||||
fallback). When `finalText` was substituted, set the bubble's
|
||||
innerHTML to `renderMarkdown(finalText)`.
|
||||
- `rememberBrainTurn(finalText || acc, { thinking:
|
||||
thinkingAcc || undefined, deflected: !!ev.deflected, sources:
|
||||
ev.sources, suggestions: ev.suggestions })` — `undefined` drops
|
||||
the key from the JSON, so turns without thinking persist exactly
|
||||
as before.
|
||||
- **Stream-drop guard** — after the `await readSSE(…)` call, before
|
||||
the existing `if (!aborted && !wrap)` fallback:
|
||||
`if (!sawDone && !aborted && (acc || thinkingAcc)) {
|
||||
setUiState(UI_STATE.error, "The stream ended before my answer
|
||||
finished — try again?"); }` (ERROR_HINT is appended by the banner).
|
||||
The zero-frame case falls through to the existing "came back empty"
|
||||
fallback, unchanged.
|
||||
- **Persistence/restore:**
|
||||
- `renderStoredMessage` (brain branch): after `addMessage`, if
|
||||
`m.thinking` — `const block = ensureThinkingBlock(wrap);
|
||||
block.open = false;
|
||||
block.querySelector(".thinking-text").innerHTML =
|
||||
renderMarkdown(m.thinking);`
|
||||
- `loadStoredConversation` filter: unchanged (raw `text` still
|
||||
required; `thinking` is optional).
|
||||
- Header doc comment: extend the phase-14 persistence note (brain
|
||||
records may carry `thinking`) and the loading-feedback comment
|
||||
(the thinking block is the visible feedback while `thinking`
|
||||
events stream; the 120s guard clears on the first thinking *or*
|
||||
delta event).
|
||||
2. `frontend/assets/styles.css` — new rules (Phase-08 tokens; place with
|
||||
the other `.msg` styles):
|
||||
- `details.thinking` — `background: var(--surface)`,
|
||||
`border: 1px solid var(--line)`,
|
||||
`border-left: 3px solid var(--brand-soft)`,
|
||||
`border-radius: var(--radius-sm)`, `margin: 0 0 0.5rem`,
|
||||
`overflow: hidden`.
|
||||
- `details.thinking summary` — `display: flex`,
|
||||
`align-items: center`, `gap: 0.5rem`,
|
||||
`padding: 0.5rem 0.75rem`, `min-height: 44px`,
|
||||
`color: var(--brand-ink)` (8.7:1 on `--surface`),
|
||||
`font-size: 0.9rem`, `cursor: pointer`, `list-style: none`;
|
||||
`summary::-webkit-details-marker { display: none; }`;
|
||||
chevron `summary::before { content: "▸"; display: inline-block;
|
||||
transition: transform 0.15s ease; }` and
|
||||
`details.thinking[open] summary::before { transform:
|
||||
rotate(90deg); }`; `summary:focus-visible` — 3px `var(--brand)`
|
||||
outline, 2px offset.
|
||||
- `details.thinking .thinking-text` —
|
||||
`padding: 0 0.75rem 0.75rem`, `color: var(--ink-soft)` (6.9:1 on
|
||||
`--surface` — keep the ratio comment in-line with the file's
|
||||
convention), `font-size: 0.875rem`, `line-height: 1.55`,
|
||||
`max-height: 320px`, `overflow-y: auto`; reduce the margins of its
|
||||
direct `p`/`ul` (e.g. `margin: 0 0 0.5rem`).
|
||||
- In the existing `@media (prefers-reduced-motion: reduce)` block(s),
|
||||
disable the summary chevron `transition`.
|
||||
3. Frontend unit pins (source-level, matching the existing files' style):
|
||||
- `tests/unit/test_frontend_feedback.py` — new tests:
|
||||
`ev.type === "thinking"` is handled in `app.js`;
|
||||
`ensureThinkingBlock` + `closeThinkingBlock` markers exist;
|
||||
the auto-collapse marker (`closeThinkingBlock(wrap)` in the delta
|
||||
branch); the `sawDone` stream-drop guard marker; the
|
||||
`uiState === UI_STATE.thinking` → streaming transition.
|
||||
- `tests/unit/test_chat_persistence.py` — new tests: the stored brain
|
||||
record carries `thinking:` in `rememberBrainTurn`'s meta
|
||||
(assert the `thinking:` marker in the save call site) and
|
||||
`renderStoredMessage` restores `m.thinking`; `styles.css` contains
|
||||
the `.thinking` rules (`details.thinking`, `.thinking-text`).
|
||||
|
||||
## Testing & Quality
|
||||
- The browser behavior is E2E-covered by task 03's suite; here the
|
||||
source-level pins above catch silent regressions without a browser.
|
||||
- No backend code changes in this task.
|
||||
- Coverage: unaffected on `app/` (frontend) — keep
|
||||
`uv run pytest --cov=app` ≥ today's number.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_frontend_feedback.py tests/unit/test_chat_persistence.py -v --no-cov`
|
||||
green; `uv run pytest` fully green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual (dev server + real aipi, or mock with a trigger once task
|
||||
03 lands): a turn with thinking shows the block open and streaming,
|
||||
collapsed after the answer; clicking the summary toggles it;
|
||||
reload restores the collapsed block with the same text; a plain
|
||||
turn (no thinking events) renders exactly as before (no block).
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): summary is a real focusable
|
||||
control (≥44px target, `:focus-visible` outline), text contrast
|
||||
≥4.5:1 (8.7:1 / 6.9:1 as specified), no CDN tags, chat column
|
||||
still 46rem, `prefers-reduced-motion` respected.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Task 03 — E2E: mock thinking trigger + the story suite
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agent/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
Deterministic E2E coverage of the thinking display: the mock LLM gains a
|
||||
`"think out loud"` trigger that streams `reasoning_content` chunks before
|
||||
the answer, and the new dedicated suite
|
||||
`tests/e2e/test_thinking_display.py` (one story → one file, A16) covers
|
||||
streaming, auto-collapse, toggle, persistence-restore, deflection
|
||||
coexistence, and the no-thinking regression — green in isolation.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/mock_llm.py`
|
||||
- Module docstring: add the trigger to the documented list —
|
||||
*user message containing ``think out loud`` → the answer is
|
||||
preceded by ~800 chars of deterministic `reasoning_content`
|
||||
chunks (thinking-display story, phase 17).*
|
||||
- `THINKING_TRIGGER = "think out loud"` (checked case-insensitively
|
||||
on the user message — same convention as
|
||||
`LONG_ANSWER_TRIGGER`).
|
||||
- `compose_thinking(body) -> str` — deterministic scratchpad text of
|
||||
roughly 700–900 chars built from a fixed 4–5 line "Step 1… Step
|
||||
N:" template that quotes the first ~60 chars of the user question
|
||||
(unique per question, stable across runs). It must contain the
|
||||
line fragment `Step 2: Check my notes` (tests key off it).
|
||||
- `_sse_stream(answer, delay)` — extend to
|
||||
`_sse_stream(answer, delay, thinking="")`: when `thinking` is
|
||||
non-empty, first yield its 12-char slices as
|
||||
`choices[0].delta = {"reasoning_content": piece}` frames (same
|
||||
0.02s cadence, same `chunk_id`/envelope shape as content frames),
|
||||
then the content frames exactly as today. Without `thinking` the
|
||||
output is byte-identical to today.
|
||||
- `chat_completions` — streaming: pass
|
||||
`compose_thinking(body) if THINKING_TRIGGER in _user(body).lower()
|
||||
else ""` as `thinking`. Non-streaming: when the trigger is present,
|
||||
include `"reasoning_content": <same text>` in the message dict
|
||||
(harmless future-proofing; the app only uses streaming).
|
||||
- Confirm no trigger collision: existing E2E questions do not contain
|
||||
the substring `"think out loud"` (the loading-feedback trigger is
|
||||
`"pretend to think slowly"` — distinct).
|
||||
2. `tests/e2e/test_thinking_display.py` (new)
|
||||
- Header comment: **mock-only suite** — `E2E_REAL_LLM=1` is not
|
||||
supported here because the real `turbo` thinks on every turn and
|
||||
would break the no-thinking regression test.
|
||||
- Fixtures mirroring `tests/e2e/test_chat_persistence.py`:
|
||||
`seeded_kb` (truncate `chunks, documents, query_log`, import
|
||||
`tests/fixtures/docs` through the real `LLMClient` +
|
||||
`import_sources`, assert `summary.added == 8`, truncate in
|
||||
teardown), `db_ready`-style skip when Postgres is down (reuse the
|
||||
conftest `db_ready` fixture).
|
||||
- Helpers: `THINK_QUESTION = "think out loud — how is my kubernetes
|
||||
cluster set up?"` (on-topic → grounded answer + thinking);
|
||||
`PLAIN_QUESTION = "How is my Kubernetes cluster set up?"`;
|
||||
`THINK_DEFLECT_QUESTION = "think out loud — tell me about quantum
|
||||
wormhole cooling"` (off-topic → deflected + thinking).
|
||||
- `send_and_wait(page, question)` — type into `#message-input`,
|
||||
submit via `#composer`, then `expect` the last `.msg.brain` to
|
||||
settle (send button re-enabled) with a 30s timeout (the mock
|
||||
streams at 0.02s/chunk; thinking + answer ≈ a few seconds).
|
||||
- **The five scenarios** (also the story's Playwright Mapping Rule):
|
||||
1. `test_thinking_block_streams_open_then_collapses` — submit
|
||||
`THINK_QUESTION`. Assert `details.thinking` inside the last
|
||||
`.msg.brain` attaches within 10s (it appears at the first
|
||||
`thinking` event); immediately after attach, assert it is open
|
||||
(the mock's ~800-char thinking stream gives a multi-second
|
||||
open window — see determinism note) and `.thinking-text` is
|
||||
non-empty; once `.bubble` text is non-empty, assert the block is
|
||||
**closed**; at settle: `.thinking-text` contains `Step 2: Check
|
||||
my notes`, the bubble contains the mock's deterministic answer
|
||||
sentence, `.source-chip` count ≥ 1, send button re-enabled.
|
||||
2. `test_thinking_toggle_after_done` — after a settled
|
||||
`THINK_QUESTION` turn, the block is closed; click `summary` →
|
||||
`details[open]` and the full thinking text is visible; click
|
||||
again → closed. (Real keyboard-focusable control.)
|
||||
3. `test_thinking_restored_after_reload` — settle a
|
||||
`THINK_QUESTION` turn; capture the thinking text;
|
||||
`page.reload()`; the restored conversation contains the brain
|
||||
message with a **closed** `details.thinking` whose
|
||||
`.thinking-text` matches the captured text, and the answer
|
||||
bubble + source chips are intact (phase-14 restore path).
|
||||
4. `test_no_thinking_block_without_trigger` — submit
|
||||
`PLAIN_QUESTION`; at settle: `page.locator("details.thinking")`
|
||||
count is 0 (a model that doesn't think renders exactly as
|
||||
before — no layout regression).
|
||||
5. `test_thinking_with_deflection` — submit
|
||||
`THINK_DEFLECT_QUESTION`; at settle: the brain message has
|
||||
`.is-deflected`, a `.maybe-try` group with chips, and a closed
|
||||
`details.thinking` whose text contains `Step 2: Check my notes`
|
||||
(thinking and the honesty gate coexist).
|
||||
- Determinism note (comment in the file): the mock paces every SSE
|
||||
frame at 0.02s and the thinking text is ~700–900 chars (≈ 60–75
|
||||
frames ≈ 1.2–1.5s) before the first content frame, so
|
||||
"attach → assert open" runs well inside the open window on
|
||||
headless Chromium; all other assertions are made after the send
|
||||
button re-enables (fully settled state).
|
||||
3. Regression pass (run each **in isolation**, per A16):
|
||||
`uv run pytest tests/e2e/test_chat_rag.py -v --no-cov`,
|
||||
`… test_loading_feedback.py …`, `… test_chat_persistence.py …`,
|
||||
`… test_honest_deflection.py …` — all green (the mock is
|
||||
trigger-gated, so their behavior is unchanged; this pass proves it).
|
||||
|
||||
## Testing & Quality
|
||||
- The new suite is this story's Playwright gate (A16): it must pass in
|
||||
isolation: `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
(prereq: `podman compose up -d db`, Chromium installed).
|
||||
- No `app/` or `frontend/` code changes in this task — if a test exposes
|
||||
a real bug in tasks 01/02, fix it in the owning file (app/ vs
|
||||
frontend/) and re-run that task's tests; note the fix in the commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Without the trigger, mock behavior is byte-identical to before —
|
||||
the four regression suites from the Work section each green **one
|
||||
command at a time** (isolation is the A16 invariant).
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
green in isolation (5/5).
|
||||
- [ ] `uv run pytest` (unit + integration) still green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Task 04 — Story file, docs, PLAN revisions, the phase commit
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agent/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
Record the feature where it belongs: the user story file (AGENTS.md rule
|
||||
4 — one story per phase), the README section, `.env.example` parity, the
|
||||
PLAN.md revisions (owner permission noted per the A10-revision
|
||||
precedent of phase 16), and the single atomic `--no-gpg-sign` commit with
|
||||
the phase moved to `complete/`.
|
||||
|
||||
## Work
|
||||
1. `.agent/user_stories/thinking-display.md` (new — match the format of
|
||||
the sibling stories, e.g. `loading-feedback.md`):
|
||||
- Header: `**Phase:** 17_thinking_display · **E2E:**
|
||||
tests/e2e/test_thinking_display.py`.
|
||||
- Narrative: as a user, local reasoning models "think" before they
|
||||
answer (10–30s of silence today). I want to *see* Brain think —
|
||||
its reasoning streaming live, tucked away once the answer starts —
|
||||
so long turns feel transparent instead of frozen.
|
||||
- Acceptance criteria:
|
||||
1. Turns whose model stream carries reasoning show a "Thinking"
|
||||
block (collapsible, above the answer bubble) that streams open
|
||||
and auto-collapses on the first answer token; always
|
||||
user-toggleable afterwards.
|
||||
2. Turns without reasoning render exactly as before (no block, no
|
||||
layout shift).
|
||||
3. Thinking-without-answer (reasoning exhausted the token budget)
|
||||
shows the existing empty-answer fallback with the thinking
|
||||
block preserved.
|
||||
4. Thinking persists with the message (phase 14) and restores
|
||||
collapsed after reload; "New chat" clears it with everything
|
||||
else.
|
||||
5. Deflected turns show the amber bubble + "Maybe try" chips
|
||||
alongside the thinking block (honesty gate untouched).
|
||||
6. A stream that dies mid-thinking/mid-answer ends in the error
|
||||
state (retry hint) — never a silent half bubble.
|
||||
7. `BOR_STREAM_THINKING=0` suppresses `thinking` events server-side
|
||||
(log line still counts `thinking_chars`).
|
||||
- UI Visualization & Structure: the DOM contract
|
||||
(`details.thinking` > `summary` + `.thinking-text`, before
|
||||
`.bubble`), Phase-08 token values + computed contrasts (summary
|
||||
8.7:1, text 6.9:1), 44px summary target, native
|
||||
`<details>/<summary>` accessibility (no live region on the
|
||||
scratchpad — `#send-status` announces state), `max-height: 320px`
|
||||
scroll, reduced-motion note.
|
||||
- Playwright Mapping Rule: the five scenarios of
|
||||
`tests/e2e/test_thinking_display.py` verbatim from task 03.
|
||||
2. `README.md` — add a short "Thinking" section in the chat/features
|
||||
area (find the natural neighbor — e.g. after the description of the
|
||||
chat UI / loading feedback): what it is (the model's reasoning,
|
||||
streamed as `thinking` SSE events, shown in a collapsible block),
|
||||
that how much it thinks is the model's call, and the
|
||||
`BOR_STREAM_THINKING=0` kill-switch. No CDN rule, no other edits.
|
||||
3. `.env.example` — verify the task-01 line (`BOR_STREAM_THINKING=1`)
|
||||
is present with its comment; add nothing new.
|
||||
4. `.agent/PLAN.md` revisions — **record owner permission
|
||||
(2026-08-23) in each note**, exactly the style phase 16 used for the
|
||||
A10 revision:
|
||||
- Header revisions line: append
|
||||
`; thinking display (Phase 17)`.
|
||||
- **§4 SSE contract:** extend the example with
|
||||
`data: {"type":"thinking","text":"…"}` frames before the `delta`
|
||||
frames, and add the client rule: *render `thinking` text in a
|
||||
collapsible block above the answer; auto-collapse on the first
|
||||
`delta`; tolerate interleaved `thinking` events; the `done` shape
|
||||
is unchanged.*
|
||||
- **§7.4 table:** new row — **Thinking (model reasoning)**:
|
||||
collapsible `.thinking` block streams open (replaces the typing
|
||||
dots as the live indicator), auto-collapses on the first answer
|
||||
token, toggleable afterwards, persisted with the message (phase
|
||||
14); 120s guard clears on the first `thinking` *or* `delta` event.
|
||||
- **§7.5 component inventory:** add `.thinking`, `.thinking-text`
|
||||
(collapsible thinking block; plain `<summary>`, no id).
|
||||
- **§9 per-turn log line:**
|
||||
`question=… embed_ms=… top_score=… fts_hits=… tuning=N
|
||||
threshold=… deflected=… sources=… thinking_chars=… total_ms=…`
|
||||
- **§12 roadmap:** new row 17 — `17_thinking_display` /
|
||||
`thinking-display.md` / `tests/e2e/test_thinking_display.py`.
|
||||
- Do **not** touch the locked anchors themselves (A15's decision text
|
||||
stays; the §4 note carries the extension) and do not renumber
|
||||
anything.
|
||||
5. Final validation pass (all gates, AGENTS.md rules 5 + 9):
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` (> 90%),
|
||||
- `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
(in isolation), plus the four regression E2E suites (one command
|
||||
each, in isolation),
|
||||
- `uv run ruff check . && uv run pyright`,
|
||||
- confirm no CDN tags were introduced
|
||||
(`tests/integration/test_api.py::test_index_html_served_locally`
|
||||
covers index; the thinking block is dynamic JS, no template
|
||||
change).
|
||||
6. Commit + phase move (last step, only when all gates are green):
|
||||
```bash
|
||||
git add -A .agent/ app/ frontend/ tests/ README.md .env.example
|
||||
git commit --no-gpg-sign -m "feat(chat): stream model thinking over SSE and show it in a collapsible block"
|
||||
mv .agent/phases/todo/17_thinking_display .agent/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- No new logic in this task — it is the record-keeping + validation pass
|
||||
of the phase; the gates above are the phase's final proof.
|
||||
- If validation fails, fix in the owning task's files, re-run that task's
|
||||
tests, and only then commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `.agent/user_stories/thinking-display.md` exists with all five
|
||||
sections (header, narrative, acceptance, UI visualization,
|
||||
Playwright Mapping Rule).
|
||||
- [ ] README "Thinking" section + `.env.example` line present;
|
||||
`.agent/PLAN.md` carries the §2-revision/§4/§7.4/§7.5/§9/§12 notes
|
||||
with the 2026-08-23 owner-permission wording; no anchor text
|
||||
altered.
|
||||
- [ ] All gates green (coverage > 90%, story E2E + 4 regressions in
|
||||
isolation, ruff + pyright clean).
|
||||
- [ ] Exactly one new commit, conventional, `--no-gpg-sign`;
|
||||
`.agent/phases/todo/17_thinking_display/` is now under
|
||||
`.agent/phases/complete/`.
|
||||
@@ -0,0 +1,160 @@
|
||||
# Phase 18 — Scroll Control: Follow the Bottom (No Yank While Reading)
|
||||
|
||||
**Story:** `.agent/user_stories/follow-bottom-scroll.md` (created by task 03)
|
||||
**Context:** `frontend/assets/app.js` — every `scrollIntoView` call site
|
||||
(`addMessage`, `addTyping`, the streaming `delta` branch, and the
|
||||
phase-17 `thinking` branch), the `SCROLL`/`reducedMotion` constants, and
|
||||
the phase-14 restore path; PLAN §7.4 ("never stale" feedback contract).
|
||||
|
||||
## Objective
|
||||
The chat must stop yanking the viewport. Today `app.js` calls
|
||||
`scrollIntoView` on **every message added, on the typing indicator, and
|
||||
on every streaming delta** — so a user who scrolls up to read earlier
|
||||
messages (or the top of a long thinking block) is dragged back to the
|
||||
bottom token by token. This phase implements the owner-chosen
|
||||
**follow-the-bottom** contract (owner choice 2026-08-23, option 1 of the
|
||||
two presented — no "↓ new content" pill): the page auto-scrolls *only
|
||||
while the user is already pinned at the bottom*; submitting a question
|
||||
still reveals the user's own message; once the user scrolls up, nothing
|
||||
auto-scrolls for the rest of the turn (thinking or answer); a restored
|
||||
conversation still lands on the latest message.
|
||||
|
||||
## Dependencies
|
||||
- `17_thinking_display` (**todo — must complete first**): its task 02
|
||||
adds the `thinking` branch that also scrolls per chunk; this phase
|
||||
gates that call site too. The pipeline runs phases in numeric order, so
|
||||
17 lands before 18 by construction.
|
||||
- `14_chat_persistence` (complete) — the restore path keeps its
|
||||
one-shot "land on the latest message" behavior (now via forced
|
||||
reveals).
|
||||
- `06_story_loading_feedback` (complete) — the state machine and the
|
||||
`SCROLL` smooth/auto constant are reused, not changed.
|
||||
- `08_story_dark_tech_theme` (complete) — no new UI surface, so no new
|
||||
tokens.
|
||||
|
||||
## Design
|
||||
- **Scroller:** the document itself (there is no inner scroll container —
|
||||
`body` is `min-height: 100dvh` and the page scrolls on the window).
|
||||
All measurements go through `window.scrollY` /
|
||||
`document.documentElement.scrollHeight` / `window.innerHeight`.
|
||||
- **New constants/helpers in `frontend/assets/app.js`:**
|
||||
- `export const NEAR_BOTTOM_PX = 200;` — the "pinned to the bottom"
|
||||
band (exported + unit-pinned, same pattern as `TURN_TIMEOUT_MS`).
|
||||
200px ≈ the composer zone (the textarea auto-grows to 192px plus
|
||||
the button row), so "the composer is fully in view" counts as
|
||||
pinned — exactly where the user sits when they submit. Scrolling
|
||||
up into the conversation (≫200px from the bottom) leaves the band.
|
||||
- `function isNearBottom()` —
|
||||
`document.documentElement.scrollHeight - window.scrollY -
|
||||
window.innerHeight <= NEAR_BOTTOM_PX`.
|
||||
- `function scrollReveal(wrap, behavior = SCROLL, force = false)` —
|
||||
the **single** scroll call site:
|
||||
`if (force || isNearBottom()) wrap.scrollIntoView({ behavior,
|
||||
block: "end" });` — `force` is used only by the phase-14 restore
|
||||
landing (one-shot, load-time).
|
||||
The existing `SCROLL` constant (smooth, or `auto` under
|
||||
`prefers-reduced-motion` — "calm, don't remove") still controls the
|
||||
*feel* of a follow scroll; reduced-motion handling is untouched.
|
||||
- **Call-site wiring (all in `app.js`):**
|
||||
- `addMessage(who, html, scrollBehavior = SCROLL, force = false)` —
|
||||
body ends with `scrollReveal(wrap, scrollBehavior, force)` (replaces
|
||||
the unconditional `wrap.scrollIntoView(…)`).
|
||||
- **Submit** (`handleSend`): the user's own message is revealed
|
||||
through the **same gate** (no force): `addMessage("user",
|
||||
renderMarkdown(text))`. In real use the user submits from the
|
||||
composer — i.e. they are pinned (within the 200px band) — so the
|
||||
message appears in view, per the owner's option-1 contract ("on
|
||||
send you still see your message and the answer appear"); a submit
|
||||
that happens with the viewport away from the bottom (only reachable
|
||||
artificially) does **not** yank it.
|
||||
- **Typing indicator** (`addTyping`): `scrollReveal(wrap)` — right
|
||||
after submit the user is pinned (visible); if they scroll up during
|
||||
pre-token "Thinking…", the indicator no longer drags them down.
|
||||
- **Streaming `delta` branch:** replace
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" })` with
|
||||
`scrollReveal(wrap)` — follows only while pinned.
|
||||
- **Phase-17 `thinking` branch:** same replacement
|
||||
(`scrollReveal(wrap)`); the block's internal
|
||||
`.thinking-text` bottom-pinning (scrollTop of the block's own
|
||||
overflow) stays as-is — that is the user's element, not the page.
|
||||
- **Restore (phase 14):** both restore call sites become
|
||||
`addMessage(…, "auto", true)` — one-shot, non-smooth, lands on the
|
||||
last restored message exactly as today (the only remaining `force`
|
||||
users; a load-time landing, not continuous auto-scroll —
|
||||
intentional, pinned by E2E test 5).
|
||||
- **Fallbacks** (empty-answer bubble, `done`-without-wrap "…"
|
||||
placeholder): `scrollReveal` with no force — appears if pinned,
|
||||
never yanks.
|
||||
- `startNewChat`: no change (the document shrinks; the browser clamps
|
||||
the stale scrollTop).
|
||||
- **After this phase, `scrollIntoView` appears exactly once in
|
||||
`app.js`** (inside `scrollReveal`) — the regression pin.
|
||||
- **Non-goals:** no "↓ new content" pill (owner opted against extras,
|
||||
2026-08-23 — a follow-up phase if ever wanted); no scroll-position
|
||||
persistence across reloads; no changes to other pages (Sources,
|
||||
document viewer, login have no vertical auto-scroll); no virtualized
|
||||
message list; no new DOM nodes, no CSS changes, no backend changes.
|
||||
|
||||
## Tasks
|
||||
1. `01_follow_bottom_scroll.md` — `app.js`: `NEAR_BOTTOM_PX`,
|
||||
`isNearBottom`, `scrollReveal`, all call sites rewired; new
|
||||
source-level unit pins.
|
||||
2. `02_e2e_story_suite.md` — `tests/e2e/test_follow_bottom_scroll.py`
|
||||
(5 scenarios, isolated run) + regression suites (incl. phase 17's
|
||||
`test_thinking_display.py` and phase 14's `test_chat_persistence.py`).
|
||||
3. `03_docs_plan_commit.md` — user story file, PLAN revisions
|
||||
(owner-permission noted), final validation, the single atomic commit,
|
||||
phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **No anchor changes.** A11 (vanilla JS, no CDN) — pure `app.js`
|
||||
logic. A15/A16 — one new story E2E suite + unit pins, no API change.
|
||||
A10/A12/A13 — untouched. This is a UI behavior change only.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_frontend_scroll.py` (new) — source-level
|
||||
pins: `export const NEAR_BOTTOM_PX = 200`; `function isNearBottom`;
|
||||
`function scrollReveal`; `scrollIntoView` occurs **exactly once** in
|
||||
`app.js` and only inside `scrollReveal`; the submit call is the plain
|
||||
default (gated, no force); restore call sites pass
|
||||
`("auto", true)`.
|
||||
- **Integration:** none (no `app/` code changes) —
|
||||
`uv run pytest --cov=app` must stay ≥ today's number.
|
||||
- **Coverage:** frontend-only phase; the `app/` >90% gate is unaffected
|
||||
but re-run to prove it.
|
||||
- **E2E:** `tests/e2e/test_follow_bottom_scroll.py` — five scenarios
|
||||
(see task 02), green **in isolation**
|
||||
(`uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov`,
|
||||
prereq `podman compose up -d db`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's coverage (frontend-only
|
||||
phase — expect no change).
|
||||
- [ ] `uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov`
|
||||
green in isolation (5/5).
|
||||
- [ ] Regression suites green **in isolation** (one command each):
|
||||
`test_thinking_display.py` (phase 17 — the thinking scroll it
|
||||
added is now gated), `test_chat_persistence.py` (restore landing),
|
||||
`test_loading_feedback.py`, `test_chat_rag.py`,
|
||||
`test_suggestion_chips.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual check (dev server): submit a question → own message +
|
||||
answer follow into view; scroll up mid-stream (or mid-thinking
|
||||
with a real aipi turn) → viewport holds still until the turn
|
||||
ends; "New chat" and reload behave as before.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): no new UI surface —
|
||||
verify no CSS/HTML template change was needed;
|
||||
`prefers-reduced-motion` still respected (the `SCROLL` constant
|
||||
is untouched).
|
||||
- [ ] PLAN carries the revisions with the 2026-08-23 owner-choice
|
||||
wording; `.agent/user_stories/follow-bottom-scroll.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/18_follow_bottom_scroll/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport"
|
||||
```
|
||||
@@ -0,0 +1,106 @@
|
||||
# Task 01 — app.js: single scroll gate (follow-the-bottom)
|
||||
|
||||
**Phase:** `18_follow_bottom_scroll` · **Story:** `.agent/user_stories/follow-bottom-scroll.md`
|
||||
|
||||
## Objective
|
||||
All chat-page scrolling goes through one gate — `scrollReveal` — which
|
||||
only fires when the user is pinned to the bottom (the 200px composer-zone
|
||||
band, `NEAR_BOTTOM_PX`) or when a call site forces it (restore only).
|
||||
The per-delta / per-thinking-chunk / typing-indicator scrolls that yank
|
||||
the viewport disappear.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js`
|
||||
- Next to the `SCROLL` constant, add:
|
||||
```js
|
||||
/* Follow-the-bottom scroll contract (phase 18, owner choice
|
||||
* 2026-08-23): the page auto-scrolls only while the user is pinned
|
||||
* at the bottom — the 200px band covers the composer zone (the
|
||||
* textarea auto-grows to 192px + the button row), i.e. "the
|
||||
* composer is in view". Exported so the band is unit-pinned (same
|
||||
* pattern as TURN_TIMEOUT_MS). */
|
||||
export const NEAR_BOTTOM_PX = 200;
|
||||
|
||||
function isNearBottom() {
|
||||
const doc = document.documentElement;
|
||||
return doc.scrollHeight - window.scrollY - window.innerHeight <= NEAR_BOTTOM_PX;
|
||||
}
|
||||
|
||||
/* The ONE scroll call site in this file. `force` is used only by
|
||||
* the phase-14 restore landing (one-shot, load-time). */
|
||||
function scrollReveal(wrap, behavior = SCROLL, force = false) {
|
||||
if (force || isNearBottom()) {
|
||||
wrap.scrollIntoView({ behavior, block: "end" });
|
||||
}
|
||||
}
|
||||
```
|
||||
- `addMessage(who, html, scrollBehavior = SCROLL, force = false)` —
|
||||
replace the trailing `wrap.scrollIntoView({ behavior: scrollBehavior,
|
||||
block: "end" });` with `scrollReveal(wrap, scrollBehavior, force);`
|
||||
(update the function's comment: scrolling is now conditional).
|
||||
- `addTyping()` — replace its
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" });` with
|
||||
`scrollReveal(wrap);`.
|
||||
- `handleSend` — the user-message call stays
|
||||
`addMessage("user", renderMarkdown(text));` (default: gated, no
|
||||
force). The user submits from the composer, so they are pinned and
|
||||
the message reveals; a submit with the viewport away from the
|
||||
bottom does not yank it.
|
||||
- Streaming `delta` branch — replace
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" });` with
|
||||
`scrollReveal(wrap);`.
|
||||
- Phase-17 `thinking` branch — replace its
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" });` with
|
||||
`scrollReveal(wrap);` (keep the `.thinking-text` internal
|
||||
`scrollTop = scrollHeight` bottom-pinning — that scrolls the
|
||||
block's own overflow, not the page). Phase 17 is complete by the
|
||||
time this task runs (pipeline numeric order), so the branch exists.
|
||||
- `renderStoredMessage` (both user and brain branches) — the restore
|
||||
calls become `addMessage("user", renderMarkdown(m.text), "auto",
|
||||
true)` / `addMessage("brain", renderMarkdown(m.text), "auto",
|
||||
true)` (one-shot, non-smooth landing on the last restored message —
|
||||
phase-14 behavior preserved).
|
||||
- `startNewChat`, fallback/placeholder `addMessage` calls: no change
|
||||
(they now go through the gated default — no force).
|
||||
- Header doc comment: add a short "Scroll (phase 18)" paragraph
|
||||
describing the follow-the-bottom contract and naming
|
||||
`NEAR_BOTTOM_PX` / `scrollReveal` as the single gate.
|
||||
2. `tests/unit/test_frontend_scroll.py` (new file — same source-level
|
||||
style as `test_frontend_feedback.py`, reading
|
||||
`frontend/assets/app.js`):
|
||||
- `test_near_bottom_constant_exported_at_200px` — regex
|
||||
`export\s+const\s+NEAR_BOTTOM_PX\s*=\s*200\s*;`.
|
||||
- `test_is_near_bottom_uses_document_scroller` — `isNearBottom`
|
||||
defined; references `documentElement.scrollHeight`,
|
||||
`window.scrollY`, `window.innerHeight`, `NEAR_BOTTOM_PX`.
|
||||
- `test_single_scroll_gate` — `function scrollReveal` exists; its
|
||||
body guards with `force || isNearBottom()`;
|
||||
`app.js.count("scrollIntoView") == 1` (exactly one occurrence,
|
||||
inside `scrollReveal`); `block: "end"` still used.
|
||||
- `test_submit_reveal_is_gated` — the send handler's user-message
|
||||
call is the plain default `addMessage("user", renderMarkdown(text))`
|
||||
(no force argument — the gate decides, and it does in real use
|
||||
because the composer being visible means pinned).
|
||||
- `test_restore_force_landing` — both restore call sites pass
|
||||
`("auto", true)`.
|
||||
- `test_streaming_scrolls_only_through_gate` — the delta branch and
|
||||
the thinking branch contain `scrollReveal(wrap)` and **no**
|
||||
raw `scrollIntoView` (covered by the count==1 test, but pin the
|
||||
two call-site markers explicitly for locality).
|
||||
|
||||
## Testing & Quality
|
||||
- No `app/` code changes — `uv run pytest --cov=app
|
||||
--cov-report=term-missing` must report the same coverage as before the
|
||||
task (gate stays green).
|
||||
- `uv run pytest tests/unit/test_frontend_scroll.py -v --no-cov` green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `rg -c "scrollIntoView" frontend/assets/app.js` reports exactly 1.
|
||||
- [ ] `uv run pytest tests/unit/test_frontend_scroll.py -v --no-cov`
|
||||
green; `uv run pytest` fully green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual smoke (dev server, real or mock LLM): submit → own message
|
||||
+ typing + answer follow into view while at the bottom; scroll to
|
||||
the top mid-stream → the viewport does not move for the rest of
|
||||
the turn; reload → lands on the latest message.
|
||||
- [ ] No changes to `index.html`, `styles.css`, or any other page.
|
||||
@@ -0,0 +1,106 @@
|
||||
# Task 02 — E2E: the follow-the-bottom story suite
|
||||
|
||||
**Phase:** `18_follow_bottom_scroll` · **Story:** `.agent/user_stories/follow-bottom-scroll.md`
|
||||
|
||||
## Objective
|
||||
Dedicated Playwright gate for the scroll contract (A16 — one story, one
|
||||
file, run in isolation): submit reveals the user's message, streaming
|
||||
follows while pinned at the bottom, and — the heart of the story — the
|
||||
viewport **holds still** while the user is scrolled up, whether the
|
||||
stream is thinking (phase 17) or answering; restore still lands on the
|
||||
latest message.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_follow_bottom_scroll.py` (new)
|
||||
- Header comment: story, prereq (`podman compose up -d db`), and the
|
||||
measurement convention — the scroller is the **document**
|
||||
(no inner scroll container): read
|
||||
`{ y: window.scrollY, sh: document.documentElement.scrollHeight,
|
||||
ch: window.innerHeight }` via `page.evaluate`; "near bottom" =
|
||||
`sh - y - ch <= 200` (mirrors `NEAR_BOTTOM_PX`); scroll to the top
|
||||
with `page.evaluate("() => window.scrollTo(0, 0)")`.
|
||||
- Real-user flow note: the user submits from the composer (pinned at
|
||||
the bottom — normal `fill` + `Enter`), and only **after** the
|
||||
stream starts do they scroll up to read. The no-yank scenarios
|
||||
follow exactly that sequence, so no off-screen input manipulation
|
||||
is needed.
|
||||
- Fixtures mirroring `tests/e2e/test_chat_persistence.py`:
|
||||
`seeded_kb` (truncate, import `tests/fixtures/docs` through the
|
||||
real `LLMClient` + `import_sources`, assert `summary.added == 8`,
|
||||
truncate in teardown); reuse the conftest `db_ready` skip.
|
||||
- Helpers:
|
||||
- `LONG_QUESTION = "write a long answer about my kubernetes cluster"`
|
||||
(mock long-answer trigger ≈ 900 words ≈ 8s of streaming — a wide,
|
||||
deterministic window to scroll away in).
|
||||
- `THINK_LONG_QUESTION = "think out loud — write a long answer about my kubernetes cluster"`
|
||||
(phase-17 thinking prefix + long answer; both mock triggers fire
|
||||
independently).
|
||||
- `scroll_state(page)` → the evaluate above; `near_bottom(state)`
|
||||
helper with the 200px band.
|
||||
- `wait_settled(page)` — `expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)`.
|
||||
- **The five scenarios** (also the story's Playwright Mapping Rule):
|
||||
1. `test_submit_reveals_new_message` — fresh chat page (pinned at
|
||||
the bottom by default); submit `LONG_QUESTION`; once settled,
|
||||
the last `.msg.brain` is inside the viewport (bounding box) and
|
||||
`near_bottom(scroll_state(page))`.
|
||||
2. `test_stream_follows_while_pinned_at_bottom` — fresh page;
|
||||
submit `LONG_QUESTION`; ~2s into the stream (poll until the
|
||||
last brain bubble's text length > 200), assert
|
||||
`near_bottom(scroll_state(page))` — the follow behavior is
|
||||
alive, not accidentally removed; at settle, still near bottom.
|
||||
3. `test_no_yank_while_scrolled_up_during_answer_stream` — submit
|
||||
`LONG_QUESTION` (fresh page, pinned — normal flow; once
|
||||
settled, the document overflows the 800px viewport — assert
|
||||
`sh > ch`); submit a second `LONG_QUESTION`; wait until the
|
||||
new brain bubble's text length > 200 (streaming has started);
|
||||
`window.scrollTo(0, 0)` (the user goes up to read); wait until
|
||||
the bubble's text length > 600 (the stream kept running while
|
||||
the viewport was at the top); assert
|
||||
`scroll_state(page)["y"] <= 5` (viewport held); wait for
|
||||
settle; assert `y <= 5` again (no scroll happened for the rest
|
||||
of the turn — the answer finished off-screen below, by
|
||||
design).
|
||||
4. `test_no_yank_while_scrolled_up_during_thinking` — one settled
|
||||
turn first (overflow exists); submit `THINK_LONG_QUESTION`
|
||||
(normal flow, pinned); wait for `details.thinking` in the last
|
||||
`.msg.brain` to attach (thinking is streaming — phase-17
|
||||
behavior) and is open; `window.scrollTo(0, 0)`; wait until the
|
||||
answer `.bubble` text is non-empty (the whole thinking stream
|
||||
plus the answer's start happened at the top); assert
|
||||
`y <= 5`; wait for settle; assert `y <= 5` and that the
|
||||
thinking text contains `Step 2: Check my notes` and the bubble
|
||||
is filled (all present but off-screen — the point of the
|
||||
story).
|
||||
5. `test_restore_lands_on_latest_message` — two settled turns
|
||||
(user + brain × 2, overflow); `page.reload()`; after restore,
|
||||
the last `.msg.brain` is inside the viewport and
|
||||
`near_bottom(scroll_state(page))` (phase-14 one-shot landing
|
||||
preserved — pinned so a future "remove all scrolling" change
|
||||
fails loudly instead of silently).
|
||||
- Determinism note (comment in the file): the mock paces SSE frames
|
||||
at 0.02s; the long answer (~900 words) streams for several seconds,
|
||||
so "mid-stream" assertions land comfortably inside the window on
|
||||
headless Chromium; every "held still" assertion compares against
|
||||
the exact `scrollTo(0, 0)` position (tolerance 5px for rounding).
|
||||
2. Regression pass — each **in isolation** (A16):
|
||||
`test_thinking_display.py` (phase 17 — its thinking scroll is now
|
||||
gated; its assertions attach/visible from a pinned-at-bottom fresh
|
||||
page, so they must still pass), `test_chat_persistence.py`
|
||||
(restore behavior), `test_loading_feedback.py` (pre-token/streaming
|
||||
feedback), `test_chat_rag.py` (core turn), `test_suggestion_chips.py`
|
||||
(chip row is horizontal — unaffected, cheap to include).
|
||||
|
||||
## Testing & Quality
|
||||
- This suite is the story's gate:
|
||||
`uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov`
|
||||
green in isolation.
|
||||
- No `app/` or mock changes in this task. If a scenario exposes a real
|
||||
bug, fix it in `frontend/assets/app.js` (the owning file) and re-run
|
||||
task 01's unit pins + this suite.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov`
|
||||
green in isolation (5/5).
|
||||
- [ ] All five regression suites above green, one command each.
|
||||
- [ ] `uv run pytest` (unit + integration) still green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Task 03 — Story file, PLAN revisions, the phase commit
|
||||
|
||||
**Phase:** `18_follow_bottom_scroll` · **Story:** `.agent/user_stories/follow-bottom-scroll.md`
|
||||
|
||||
## Objective
|
||||
Record the change: the user story file (AGENTS.md rule 4), the PLAN
|
||||
revisions with the owner-choice noted, and the single atomic
|
||||
`--no-gpg-sign` commit with the phase moved to `complete/`.
|
||||
|
||||
## Work
|
||||
1. `.agent/user_stories/follow-bottom-scroll.md` (new — match the
|
||||
format of the sibling stories, e.g. `loading-feedback.md`):
|
||||
- Header: `**Phase:** 18_follow_bottom_scroll · **E2E:**
|
||||
tests/e2e/test_follow_bottom_scroll.py`.
|
||||
- Narrative: as a user, Brain's answers (and thinking) stream for
|
||||
10–30s. I want to scroll up and read without the page dragging me
|
||||
back to the bottom token by token — but when I submit, I should
|
||||
still see my message and the reply appear.
|
||||
- Acceptance criteria:
|
||||
1. Submitting a question always reveals the user's own message
|
||||
(explicit action, unconditional).
|
||||
2. While the user is pinned to the bottom (within 200px of it —
|
||||
the composer zone), the typing indicator, thinking chunks, and
|
||||
answer deltas follow into view (smooth, or instant under
|
||||
`prefers-reduced-motion`).
|
||||
3. Once the user scrolls up (more than 200px from the bottom),
|
||||
nothing auto-scrolls for the rest of the turn — thinking or
|
||||
answer; the viewport position is unchanged at turn end.
|
||||
4. The thinking block's *internal* text still bottom-pins itself
|
||||
while open (that is the block's own overflow, not the page).
|
||||
5. Restoring a stored conversation still lands on the latest
|
||||
message (one-shot, non-smooth).
|
||||
6. No new UI surface (no "new content" pill — owner chose the
|
||||
minimal contract, 2026-08-23); no other page is affected.
|
||||
- UI Visualization & Structure: the scroller is the document (no
|
||||
inner overflow container); `NEAR_BOTTOM_PX = 200` (exported,
|
||||
unit-pinned); `scrollReveal` is the single `scrollIntoView` call
|
||||
site; the `SCROLL` smooth/auto constant (reduced-motion) is reused
|
||||
unchanged.
|
||||
- Playwright Mapping Rule: the five scenarios of
|
||||
`tests/e2e/test_follow_bottom_scroll.py` verbatim from task 02.
|
||||
2. `.agent/PLAN.md` revisions — **record the owner choice
|
||||
(2026-08-23, "option 1: follow-the-bottom, no pill")** in each note,
|
||||
style per the phase-16 A10-revision precedent:
|
||||
- Header revisions line: append
|
||||
`; follow-the-bottom scroll (Phase 18)`.
|
||||
- **§7.4 table:** new row — **Scroll (follow-the-bottom, phase
|
||||
18)**: the page auto-scrolls only while the user is pinned to the
|
||||
bottom (≤200px band, `NEAR_BOTTOM_PX` — the composer zone; submit
|
||||
reveals the user's message through the same gate, which holds in
|
||||
real use); scrolling up holds the viewport for the rest of the
|
||||
turn (thinking and answer alike); restore lands one-shot on the
|
||||
latest message.
|
||||
- **§12 roadmap:** new row 18 — `18_follow_bottom_scroll` /
|
||||
`follow-bottom-scroll.md` / `test_follow_bottom_scroll.py`.
|
||||
- Do not alter any locked anchor or renumber anything. (No §9/§4/§7.5
|
||||
changes — no log, API, or new component surface.)
|
||||
3. `README.md` — no change (no operator-facing behavior; the README
|
||||
does not document chat scroll behavior today).
|
||||
4. Final validation pass (all gates, AGENTS.md rules 5 + 9):
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` (≥ today's
|
||||
coverage),
|
||||
- `uv run pytest tests/e2e/test_follow_bottom_scroll.py -v --no-cov`
|
||||
in isolation, plus the five regression suites from task 02 (one
|
||||
command each, in isolation),
|
||||
- `uv run ruff check . && uv run pyright`,
|
||||
- `rg -c "scrollIntoView" frontend/assets/app.js` → 1.
|
||||
5. Commit + phase move (last step, only when all gates are green):
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/
|
||||
git commit --no-gpg-sign -m "feat(ui): chat auto-scrolls only while pinned to the bottom — submitting reveals your message, scrolling up holds the viewport"
|
||||
mv .agent/phases/todo/18_follow_bottom_scroll .agent/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- No new logic — this is the record-keeping + validation pass; the
|
||||
gates above are the phase's final proof. If validation fails, fix in
|
||||
the owning task's files, re-run that task's tests, then commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `.agent/user_stories/follow-bottom-scroll.md` exists with all
|
||||
five sections (header, narrative, acceptance, UI visualization,
|
||||
Playwright Mapping Rule).
|
||||
- [ ] `.agent/PLAN.md` carries the header-revision/§7.4/§12 notes with
|
||||
the 2026-08-23 owner-choice wording; no anchor text altered.
|
||||
- [ ] All gates green (coverage, story E2E + 5 regressions in
|
||||
isolation, ruff + pyright, single `scrollIntoView`).
|
||||
- [ ] Exactly one new commit, conventional, `--no-gpg-sign`;
|
||||
`.agent/phases/todo/18_follow_bottom_scroll/` is now under
|
||||
`.agent/phases/complete/`.
|
||||
@@ -0,0 +1,187 @@
|
||||
# Phase 19 — Shared Header: auth + New Chat on every page, Sources link admin-only
|
||||
|
||||
**Story:** `.agent/user_stories/shared-header.md` (created by task 03)
|
||||
**Context:** the four page headers (`frontend/index.html`,
|
||||
`sources.html`, `document.html`, `login.html` — each hand-rolled, which is
|
||||
exactly why the controls "disappear" between pages), `frontend/assets/
|
||||
app.js` (chat's whoami gating + New Chat + sign-out handlers),
|
||||
`sources.js` (whoami-before-docs gate), `document.js`, phase 12's exact
|
||||
header-height contract (64px / 58px — `test_header_consistency.py`),
|
||||
phase 16's auth model (A10 revised).
|
||||
|
||||
## Objective
|
||||
Make the title bar actually shared. Owner report 2026-08-23: clicking
|
||||
"Sources" makes **New Chat** and **Sign in** vanish — the user expects
|
||||
one consistent bar on every page. This phase puts the same header
|
||||
controls on **Chat, Sources, and the document viewer** (Sign in /
|
||||
Sign out + New Chat, via one shared module), and — per the same owner
|
||||
instruction — **hides the "Sources" nav link from anonymous users**
|
||||
(revises the phase-16 UX choice "show the link, soft-gate the page";
|
||||
the soft gate itself stays for direct-URL visitors, and the API rules
|
||||
of the A10 revision are untouched).
|
||||
|
||||
## Owner-confirmed changes (2026-08-23, this request)
|
||||
1. **Sign in / Sign out + New Chat are always visible** on chat,
|
||||
sources, and the document viewer (anonymous AND admin — Sign in vs
|
||||
Sign out per whoami). The login page gets **no** chat controls (it is
|
||||
the auth page, not an app page) — noted boundary, owner may overrule.
|
||||
2. **The "Sources" nav link is hidden for anonymous users** on every
|
||||
page that has a nav (chat, sources, login). `/sources.html` keeps
|
||||
its phase-16 soft gate for direct-URL access; `GET /api/docs` stays
|
||||
403 for anonymous (A10 revision unchanged — this is UI visibility,
|
||||
not API access).
|
||||
|
||||
## Design
|
||||
- **Shared module `frontend/assets/header.js` (new, ES module — all
|
||||
pages already load JS as `type="module"`):**
|
||||
- `export function fetchIsAdmin(): Promise<boolean>` — one
|
||||
`GET /api/whoami`, cached in a module-level promise (anonymous-safe:
|
||||
network failure → `false`). Every page's whoami goes through this
|
||||
single function, so the chat page makes exactly one request
|
||||
(app.js swaps its private `loadAuthState` fetch for this import).
|
||||
- `export async function initSharedHeader()` — awaits
|
||||
`fetchIsAdmin()`, then toggles **only the elements that exist on
|
||||
the page** (missing → no-op, which is how the login page reuses it
|
||||
without gaining controls):
|
||||
- `#sign-in-link` hidden when admin, `#sign-out-btn` shown when
|
||||
admin (exactly one visible — phase-16 semantics);
|
||||
- `#nav-sources` (new id on the Sources nav link, every page with
|
||||
a nav) **hidden for anonymous, shown for admin** — new
|
||||
anonymous-safe default: the link ships with the `hidden`
|
||||
attribute (phase-16 "absent, not hidden" spirit) and appears when
|
||||
whoami says admin.
|
||||
- `export function clearChatStorage()` — removes the `bor.chat.v1`
|
||||
key in a try/catch (mirrors app.js's `clearStoredConversation`).
|
||||
- `#sign-out-btn` binding lives here (POST `/api/logout`, disable
|
||||
during the call, `location.reload()`) — `app.js` deletes its own
|
||||
copy so there is exactly one implementation.
|
||||
- `#new-chat-btn` binding: the chat page keeps `app.js`'s
|
||||
`startNewChat` (in-place reset + focus + announce). On **non-chat
|
||||
pages** (sources.js / document.js, ~4 lines each):
|
||||
click → `clearChatStorage()` → `location.href = "/"` (a new chat
|
||||
means going to the chat).
|
||||
- **HTML wiring:**
|
||||
- `index.html` — add `id="nav-sources"` to the Sources nav link
|
||||
(`hidden` by default); load `header.js` before `app.js`.
|
||||
- `sources.html` — add `id="nav-sources"` (`hidden`) to its Sources
|
||||
nav link; append to `.header-inner` the New Chat button +
|
||||
`#sign-in-link` (`/login.html?next=/sources.html`) +
|
||||
`#sign-out-btn` — markup copied from `index.html` (same classes,
|
||||
ids, aria-labels, ≥44px targets); load `header.js` before
|
||||
`sources.js`; sources.js calls `initSharedHeader()` at boot and
|
||||
binds its New Chat button. (sources.js's existing `isAdmin()`
|
||||
whoami helper keeps working — it can be reimplemented on top of
|
||||
`fetchIsAdmin()` to avoid a second request.)
|
||||
- `document.html` — append a `.doc-header-actions` wrapper (New Chat
|
||||
+ `#sign-in-link` `/login.html?next=/document.html` +
|
||||
`#sign-out-btn`) to the right of `.doc-header-inner` (the viewer
|
||||
has no nav — no `#nav-sources` there); load `header.js` before
|
||||
`document.js`; document.js calls `initSharedHeader()` and binds New
|
||||
Chat.
|
||||
- `login.html` — add `id="nav-sources"` (`hidden`) + load
|
||||
`header.js` (init only — it toggles the nav link; no chat controls
|
||||
are added, so none appear).
|
||||
- **CSS (`styles.css`):** the new controls reuse the existing
|
||||
`.new-chat-btn` / `.auth-link` classes, so the phase-14/16 mobile
|
||||
icon-only rules (labels hidden, 16px icon shown) apply automatically.
|
||||
New work is the **viewer bar only**: `.doc-header-actions {
|
||||
margin-left: auto; display: flex; gap: 0.5rem; align-items: center;
|
||||
}`; the title block gets `min-width: 0` so `#doc-title`/`#doc-meta`
|
||||
keep truncating (phase-12 "clip, don't wrap") while the two pills
|
||||
fit; the bar must still measure exactly `--header-h` (64px desktop,
|
||||
58px ≤640px) and produce **no horizontal overflow at 360px**
|
||||
(`test_responsive_polish` pins `scrollWidth <= clientWidth`). The
|
||||
sources bar already fits this exact control set (the chat bar does —
|
||||
it even carries the steering toggle), so no sources CSS is expected.
|
||||
- **`app.js` (chat) adaptations:** boot calls `initSharedHeader()`
|
||||
(toggles nav-sources + auth links) before `restoreConversation()`;
|
||||
its `isAdmin` value comes from the shared `fetchIsAdmin()` (cached —
|
||||
still one whoami per load); delete the now-duplicated sign-out
|
||||
listener. Everything else (steering gating, tune buttons) unchanged.
|
||||
- **Non-goals:** no server-side header (still static templates — A11);
|
||||
no API changes (A10 revision untouched); no login-page chat controls;
|
||||
no change to the document viewer's back-link/title contract (phase
|
||||
13); the "Sources" **page** soft gate and `#sources-gate` are
|
||||
unchanged; no `next`-param changes in `login.js`.
|
||||
|
||||
## Dependencies
|
||||
- `16_admin_auth` (complete) — the whoami/session model and
|
||||
`auth_helpers.login(page, app_url, next=…)` E2E helper.
|
||||
- `14_chat_persistence` (complete) — the `bor.chat.v1` key the
|
||||
non-chat New Chat buttons clear.
|
||||
- `12_header_consistency` (complete) — the 64/58px height contract the
|
||||
new controls must fit inside.
|
||||
- `10_story_document_viewer` + `13_document_back_navigation`
|
||||
(complete) — the viewer header being extended.
|
||||
- `17_thinking_display` / `18_follow_bottom_scroll` (todo) — no code
|
||||
overlap (chat-page turn rendering only); independent order.
|
||||
|
||||
## Tasks
|
||||
1. `01_shared_header_module.md` — `header.js` module, HTML wiring on
|
||||
all four pages, viewer-bar CSS, app.js/sources.js/document.js
|
||||
adaptations, source-level unit pins.
|
||||
2. `02_e2e_story_suite.md` — `tests/e2e/test_shared_header.py` (the
|
||||
story gate, isolated) + the regression suites (header consistency,
|
||||
responsive polish, admin auth, document back navigation, chat
|
||||
persistence).
|
||||
3. `03_docs_plan_commit.md` — story file, PLAN revisions (owner
|
||||
permission noted), final validation, the single atomic commit,
|
||||
phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Phase-16 UX revision with owner permission (2026-08-23):** the
|
||||
"Sources" nav link is hidden for anonymous (before: shown, page
|
||||
soft-gated). The **soft gate page and the A10 API split are
|
||||
unchanged** — recorded as a PLAN §7 revision note, not an anchor
|
||||
change.
|
||||
- **A11 untouched** — vanilla JS, no CDN, static templates. **A10
|
||||
untouched** — endpoint access unchanged. **A16 untouched** — one new
|
||||
story E2E suite + adapted regressions. No other anchor changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit (source-level, new `tests/unit/test_shared_header.py`):**
|
||||
`header.js` exports `fetchIsAdmin` / `initSharedHeader` /
|
||||
`clearChatStorage`; the whoami fetch is cached (single promise);
|
||||
`#nav-sources` present with initial `hidden` in index/sources/login
|
||||
HTML; sources + document HTML carry `#sign-in-link`, `#sign-out-btn`,
|
||||
`#new-chat-btn`; `app.js` no longer owns the sign-out binding
|
||||
(no `signOutBtn.addEventListener` in app.js) and imports
|
||||
`fetchIsAdmin`; `styles.css` has `.doc-header-actions`.
|
||||
- **Integration:** none (no `app/` changes) — `uv run pytest
|
||||
--cov=app` must stay at today's number.
|
||||
- **Coverage:** frontend-only; the >90% `app/` gate is unaffected,
|
||||
re-run to prove it.
|
||||
- **E2E:** `tests/e2e/test_shared_header.py` — six scenarios (task 02),
|
||||
green **in isolation** (prereq `podman compose up -d db`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Anonymous, on **chat, sources, and the viewer**: Sign in + New
|
||||
Chat visible, `#nav-sources` hidden. Admin, on all three: Sign
|
||||
out + New Chat + `#nav-sources` (chat/sources) visible.
|
||||
- [ ] New Chat from sources/viewer clears `bor.chat.v1` and lands on
|
||||
the chat empty state; New Chat on chat behaves exactly as before
|
||||
(in-place reset).
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number.
|
||||
- [ ] `uv run pytest tests/e2e/test_shared_header.py -v --no-cov` green
|
||||
in isolation (6/6); regressions green in isolation (one command
|
||||
each): `test_header_consistency.py` (64/58px with the new pills
|
||||
on sources + viewer), `test_responsive_polish.py` (no 360px
|
||||
overflow), `test_admin_auth.py`, `test_document_back_navigation.py`,
|
||||
`test_chat_persistence.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): all new controls reuse
|
||||
labeled ≥44px patterns (aria-labels on icon-only mobile),
|
||||
focus-visible, no CDN tags, one header bar per page, heights
|
||||
unchanged.
|
||||
- [ ] PLAN carries the revisions with the 2026-08-23 owner-permission
|
||||
wording; `.agent/user_stories/shared-header.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/19_shared_header/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(ui): shared header — Sign in/Sign out and New Chat on every page; hide the Sources nav link from anonymous users"
|
||||
```
|
||||
@@ -0,0 +1,138 @@
|
||||
# Task 01 — header.js shared module + page wiring
|
||||
|
||||
**Phase:** `19_shared_header` · **Story:** `.agent/user_stories/shared-header.md`
|
||||
|
||||
## Objective
|
||||
One shared header module drives the auth controls and the Sources nav
|
||||
link on every page; Sources and the document viewer gain the New Chat /
|
||||
Sign in / Sign out controls; the Sources nav link is hidden for
|
||||
anonymous users everywhere.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/header.js` (new ES module — same style as
|
||||
`app.js`: header doc comment citing the phase, no dependencies):
|
||||
- `let adminPromise: Promise<boolean> | null = null;`
|
||||
`export function fetchIsAdmin(): Promise<boolean>` — first call
|
||||
stores `fetch("/api/whoami")` → `.then(r => r.ok && (r.json()…
|
||||
.authenticated === true))` with a catch → `false` (anonymous-safe,
|
||||
mirrors app.js's current `loadAuthState`); subsequent calls return
|
||||
the same promise.
|
||||
- `export async function initSharedHeader(): Promise<boolean>` —
|
||||
`const admin = await fetchIsAdmin();` then, **only when the
|
||||
element exists** (`document.querySelector`, null-safe):
|
||||
- `#sign-in-link` → `hidden = admin`;
|
||||
- `#sign-out-btn` → `hidden = !admin`;
|
||||
- `#nav-sources` → `hidden = !admin` (the link ships hidden —
|
||||
anonymous-safe default, appears for admin).
|
||||
Returns `admin` (callers may reuse it).
|
||||
- `export function clearChatStorage(): void` —
|
||||
`try { localStorage.removeItem("bor.chat.v1"); } catch {}` (same
|
||||
key + fail-silence contract as app.js's `clearStoredConversation`).
|
||||
- Sign-out binding (runs at module import, so every page that loads
|
||||
header.js gets it): if `#sign-out-btn` exists — click →
|
||||
`disabled = true`, `fetch("/api/logout", {method:"POST"})`
|
||||
(catch ignored — the reload resets UI), `window.location.reload()`.
|
||||
2. `frontend/index.html` (chat)
|
||||
- Sources nav link: add `id="nav-sources"` and the `hidden`
|
||||
attribute (appears once whoami says admin).
|
||||
- `<script type="module" src="/assets/header.js"></script>` before
|
||||
the `app.js` script tag.
|
||||
3. `frontend/assets/app.js`
|
||||
- `import { fetchIsAdmin, initSharedHeader } from "/assets/header.js";`
|
||||
- Boot (the trailing IIFE): replace `await loadAuthState();` with
|
||||
`isAdmin = await initSharedHeader();` (initSharedHeader returns
|
||||
admin and already toggled `#sign-in-link` / `#sign-out-btn` /
|
||||
`#nav-sources`); `loadAuthState`'s body reduces to calling
|
||||
`fetchIsAdmin()` + `applyAuthState()` — or, simpler, delete
|
||||
`loadAuthState` and inline: `isAdmin = await fetchIsAdmin();
|
||||
applyAuthState();` **after** `initSharedHeader()` (the cached
|
||||
promise means still exactly one whoami per page load).
|
||||
- **Delete the sign-out listener** (`if (signOutBtn) { …
|
||||
addEventListener("click", …) }`) — header.js owns it now. Keep the
|
||||
`signOutBtn` query only if `applyAuthState` still uses it
|
||||
(it does — for `hidden` toggling — which header.js also does;
|
||||
`applyAuthState` may keep its toggling, it's idempotent).
|
||||
- `startNewChat` unchanged (chat-page in-place reset).
|
||||
4. `frontend/sources.html`
|
||||
- Sources nav link: add `id="nav-sources"` + `hidden`.
|
||||
- Append to `.header-inner` (after the nav), copied from
|
||||
`index.html`: the New Chat button (`#new-chat-btn`, same svg +
|
||||
`aria-label="New chat"` + `.new-chat-label` span), the
|
||||
`#sign-in-link` `<a href="/login.html?next=/sources.html" hidden>`
|
||||
and the `#sign-out-btn` button — **both** start `hidden`, exactly
|
||||
as in index.html; `initSharedHeader` reveals one after whoami.
|
||||
- `<script type="module" src="/assets/header.js"></script>` before
|
||||
the `sources.js` tag.
|
||||
5. `frontend/assets/sources.js`
|
||||
- `import { fetchIsAdmin, initSharedHeader, clearChatStorage }
|
||||
from "/assets/header.js";`
|
||||
- Boot: call `await initSharedHeader()` (before the docs fetch,
|
||||
alongside the existing gate check); reimplement the local
|
||||
`isAdmin()` on top of `fetchIsAdmin()` (drop the private fetch —
|
||||
one request per page).
|
||||
- Bind `#new-chat-btn`: click → `clearChatStorage()` →
|
||||
`window.location.href = "/"`.
|
||||
6. `frontend/document.html`
|
||||
- Inside `.doc-header-inner`, after the title block:
|
||||
`<div class="doc-header-actions">` containing the New Chat button,
|
||||
`#sign-in-link` (`/login.html?next=/document.html`) and
|
||||
`#sign-out-btn` — **both** start `hidden` (initSharedHeader
|
||||
reveals one after whoami) — same markup/aria as index.html.
|
||||
- `<script type="module" src="/assets/header.js"></script>` before
|
||||
the `document.js` tag.
|
||||
7. `frontend/assets/document.js`
|
||||
- Import the same three header.js exports; at boot (before or after
|
||||
the doc fetch — independent) `await initSharedHeader()`; bind
|
||||
`#new-chat-btn` exactly like sources.js.
|
||||
8. `frontend/login.html`
|
||||
- Sources nav link: add `id="nav-sources"` + `hidden`; load
|
||||
`header.js` and call `initSharedHeader()` from `login.js` boot
|
||||
(login.js already fetches whoami for the redirect — switch it to
|
||||
the shared `fetchIsAdmin()` so the page makes one request, and
|
||||
keep its existing "already admin → redirect to `next`" behavior).
|
||||
9. `frontend/assets/styles.css` — viewer bar only:
|
||||
- `.doc-header-actions { margin-left: auto; display: flex; gap:
|
||||
0.5rem; align-items: center; }`
|
||||
- `.doc-title-block { min-width: 0; }` (title/meta keep their
|
||||
existing truncation — phase-12 "clip, don't wrap").
|
||||
- The reused `.new-chat-btn` / `.auth-link` classes already carry
|
||||
the desktop + ≤640px icon-only rules; if the 360px bar overflows
|
||||
(the E2E will say), tighten `.doc-header-actions` padding there —
|
||||
but do NOT change `--header-h` (64/58 are pinned).
|
||||
10. `tests/unit/test_shared_header.py` (new — source-level, same style
|
||||
as `test_frontend_feedback.py`):
|
||||
- `header.js` exists and exports `fetchIsAdmin`,
|
||||
`initSharedHeader`, `clearChatStorage`; the whoami fetch is
|
||||
cached (a module-level promise variable — pin the
|
||||
`adminPromise` marker); `clearChatStorage` references
|
||||
`"bor.chat.v1"` inside a try/catch.
|
||||
- `#nav-sources` present and initially `hidden` in index.html,
|
||||
sources.html, login.html; NOT in document.html.
|
||||
- sources.html AND document.html contain `#sign-in-link`,
|
||||
`#sign-out-btn`, `#new-chat-btn`, and load `header.js`.
|
||||
- `app.js` imports `fetchIsAdmin`/`initSharedHeader` from
|
||||
`header.js` and contains **no** `signOutBtn.addEventListener`
|
||||
(the binding moved to the shared module); `login.js` imports
|
||||
`fetchIsAdmin`.
|
||||
- `styles.css` defines `.doc-header-actions`.
|
||||
|
||||
## Testing & Quality
|
||||
- No `app/` changes — `uv run pytest --cov=app` stays at today's
|
||||
number.
|
||||
- `uv run pytest tests/unit/test_shared_header.py -v --no-cov` green;
|
||||
full `uv run pytest` green (the phase-16 unit tests for auth config
|
||||
are untouched).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_shared_header.py -v --no-cov`
|
||||
green; `uv run pytest` fully green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual (dev server): anonymous — chat: Sign in + New Chat shown,
|
||||
Sources nav link absent; sources page: same + soft gate; viewer:
|
||||
same + back/title intact. Log in — all three pages show Sign out
|
||||
+ Sources link (where a nav exists). New Chat from sources →
|
||||
conversation cleared, lands on chat empty state. Sign out from
|
||||
the viewer → reload → Sign in back.
|
||||
- [ ] Chat page still makes exactly one `/api/whoami` request per load
|
||||
(Network tab) and the steering/tune gating is unchanged.
|
||||
- [ ] No changes under `app/`, no CDN tags, no new assets.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Task 02 — E2E: the shared-header story suite
|
||||
|
||||
**Phase:** `19_shared_header` · **Story:** `.agent/user_stories/shared-header.md`
|
||||
|
||||
## Objective
|
||||
Dedicated Playwright gate (A16 — one story, one file, isolated): the
|
||||
shared bar contract on chat / sources / viewer in both auth states, the
|
||||
anonymous Sources-link hiding, New Chat from non-chat pages, and sign
|
||||
out from a non-chat page — plus the header-height and overflow
|
||||
regressions that this change puts at risk.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_shared_header.py` (new)
|
||||
- Header comment: story, prereq (`podman compose up -d db`), and the
|
||||
contract under test (one bar per page: brand + nav [Chat,
|
||||
Sources-admin-only] + New Chat + Sign in/Sign out on chat &
|
||||
sources; back + title + New Chat + Sign in/Sign out on the
|
||||
viewer; heights 64px/58px per phase 12).
|
||||
- Fixtures mirroring `tests/e2e/test_header_consistency.py`:
|
||||
seed the DB with the fixture docs (needed for the viewer URL and
|
||||
the sources catalog); reuse `e2e.auth_helpers.login(page, app_url,
|
||||
next=…)` for real form logins and the conftest `ADMIN_PASSWORD`.
|
||||
- Constants: `VIEWER_URL` (a seeded doc, URL-encoded),
|
||||
`SOURCES_URL = "/sources.html"`.
|
||||
- Helper `assert_shared_bar(page, admin: bool, page_kind:
|
||||
"chat"|"sources"|"viewer")` — the heart of the suite, asserting
|
||||
per kind:
|
||||
- `#new-chat-btn` visible (all kinds);
|
||||
- admin → `#sign-out-btn` visible + `#sign-in-link` hidden, else
|
||||
the inverse (all kinds);
|
||||
- chat/sources → `#nav-sources` visible iff admin; viewer → no
|
||||
`#nav-sources` in DOM (`count() == 0`);
|
||||
- bar height: chat/sources `.app-header` == 64 (viewport 1280) /
|
||||
58 (≤640), viewer `.doc-header` == same value (bounding boxes,
|
||||
phase-12 measurement convention).
|
||||
- **The six scenarios** (also the story's Playwright Mapping Rule):
|
||||
1. `test_anonymous_bar_on_all_pages` — fresh (anonymous) page:
|
||||
`assert_shared_bar` for chat, sources, and viewer, admin=False
|
||||
(desktop viewport).
|
||||
2. `test_admin_bar_on_all_pages` — `login(page, app_url, next="/")`;
|
||||
`assert_shared_bar` for all three pages, admin=True. (Also
|
||||
proves the login → `next` flow still lands right.)
|
||||
3. `test_sources_nav_hidden_for_anonymous_everywhere` —
|
||||
anonymous: on chat, sources, and the login page
|
||||
(`/login.html`), `#nav-sources` is hidden; after login on the
|
||||
chat page, `#nav-sources` is visible (toggle works, not just
|
||||
initial state).
|
||||
4. `test_new_chat_from_sources_clears_and_navigates` — anonymous
|
||||
is fine: seed a conversation via
|
||||
`page.add_init_script` setting `localStorage["bor.chat.v1"] =
|
||||
JSON.stringify({v:1, messages:[{who:"user",
|
||||
text:"hello brain"},{who:"brain", text:"hey there"}]})` (or
|
||||
drive it through the chat UI — either, deterministic); go to
|
||||
sources; click `#new-chat-btn`; expect navigation to `/` with
|
||||
the empty state visible and `bor.chat.v1` removed
|
||||
(`page.evaluate` reads localStorage).
|
||||
5. `test_sign_out_from_viewer_returns_to_anonymous` — login with
|
||||
`next=/sources.html` (lands on sources, admin); open the
|
||||
viewer URL directly; `assert_shared_bar(… admin=True,
|
||||
"viewer")`; click `#sign-out-btn`; after the reload,
|
||||
`assert_shared_bar(… admin=False, "viewer")`.
|
||||
6. `test_mobile_bar_fits_and_heights_held` — viewport 375×812,
|
||||
anonymous: on all three pages the bar height is 58 and
|
||||
`documentElement.scrollWidth <= clientWidth` (no horizontal
|
||||
overflow — the pills are icon-only per the existing mobile
|
||||
rules); repeat the three heights after login (Sign out +
|
||||
Sources link present) — the bar never grows.
|
||||
- Determinism note: all assertions are settled-state (no streaming
|
||||
involved in this story — the chat page is opened at most for its
|
||||
header; no turn is submitted except where a scenario says so).
|
||||
2. Regression pass — each **in isolation** (A16), one command each:
|
||||
`test_header_consistency.py` (64/58px on the three pages — now with
|
||||
the new pills on sources + viewer, both auth states on chat),
|
||||
`test_responsive_polish.py` (360px overflow guards),
|
||||
`test_admin_auth.py` (phase-16 flows: chat header auth, sources
|
||||
gate, login, sign out — its assertions must still hold with
|
||||
`#nav-sources` hidden for anonymous),
|
||||
`test_document_back_navigation.py` (viewer header back-link + title
|
||||
contract with the new actions wrapper),
|
||||
`test_chat_persistence.py` (chat New Chat in-place behavior +
|
||||
restore — untouched code path).
|
||||
|
||||
## Testing & Quality
|
||||
- This suite is the story's gate:
|
||||
`uv run pytest tests/e2e/test_shared_header.py -v --no-cov` green in
|
||||
isolation.
|
||||
- No `app/` or mock changes. If a scenario exposes a real bug, fix it
|
||||
in the owning frontend file and re-run task 01's unit pins + this
|
||||
suite.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_shared_header.py -v --no-cov` green
|
||||
in isolation (6/6).
|
||||
- [ ] All five regression suites above green, one command each.
|
||||
- [ ] `uv run pytest` (unit + integration) still green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Task 03 — Story file, PLAN revisions, the phase commit
|
||||
|
||||
**Phase:** `19_shared_header` · **Story:** `.agent/user_stories/shared-header.md`
|
||||
|
||||
## Objective
|
||||
Record the change: the user story file (AGENTS.md rule 4), the PLAN
|
||||
revisions with the owner permission noted (this revises a phase-16 UX
|
||||
choice — say so explicitly), and the single atomic `--no-gpg-sign`
|
||||
commit with the phase moved to `complete/`.
|
||||
|
||||
## Work
|
||||
1. `.agent/user_stories/shared-header.md` (new — match the sibling
|
||||
story format):
|
||||
- Header: `**Phase:** 19_shared_header · **E2E:**
|
||||
tests/e2e/test_shared_header.py`.
|
||||
- Narrative: as a user, the top bar should feel like one shared
|
||||
component: when I move to Sources or open a document, I should
|
||||
still see Sign in (or Sign out) and New Chat — and the Sources
|
||||
link should not offer me a page I can't use until I sign in.
|
||||
- Acceptance criteria:
|
||||
1. Chat, Sources, and the document viewer each show New Chat +
|
||||
Sign in (anonymous) or Sign out (admin) in the header; the
|
||||
login page shows neither (auth page, not an app page).
|
||||
2. The "Sources" nav link is hidden for anonymous users on every
|
||||
page that has a nav, and visible for the admin.
|
||||
3. Anonymous direct-URL access to `/sources.html` still shows the
|
||||
phase-16 soft gate (link hidden, gate intact); the API split
|
||||
is unchanged (`/api/docs` 403 anonymous).
|
||||
4. New Chat on chat: in-place reset (unchanged). New Chat on
|
||||
sources/viewer: clears the local conversation (`bor.chat.v1`)
|
||||
and navigates to the chat page.
|
||||
5. Sign out works from any page (logout + reload → anonymous
|
||||
state restored on that page).
|
||||
6. The bar stays exactly 64px (desktop) / 58px (≤640px) on all
|
||||
three pages in both auth states, with no horizontal overflow
|
||||
at 360px (phase-12 contract, phase-07 overflow guard).
|
||||
7. Exactly one `/api/whoami` request per page load (shared
|
||||
cached fetch).
|
||||
- UI Visualization & Structure: `header.js` shared module
|
||||
(`fetchIsAdmin` cached promise, `initSharedHeader` toggles
|
||||
existing elements only, `clearChatStorage`); element ids
|
||||
(`#nav-sources` hidden-by-default; `#sign-in-link`
|
||||
visible-by-default — existing patterns); the viewer's
|
||||
`.doc-header-actions` wrapper; reused `.new-chat-btn` /
|
||||
`.auth-link` mobile icon-only rules; login page boundary.
|
||||
- Playwright Mapping Rule: the six scenarios of
|
||||
`tests/e2e/test_shared_header.py` verbatim from task 02.
|
||||
2. `.agent/PLAN.md` revisions — **owner permission 2026-08-23 (this
|
||||
request)** in each note, phase-16-revision style:
|
||||
- Header revisions line: append
|
||||
`; shared header (Phase 19)`.
|
||||
- **§2 A10 note** (append to the existing 2026-08-22 revision text,
|
||||
do NOT change the decision itself): *UI revision 2026-08-23
|
||||
(owner permission): the "Sources" nav link is hidden from
|
||||
anonymous users on all pages — the soft-gate page and the API
|
||||
split above are unchanged.*
|
||||
- **§7.1 layout:** note that the header is a shared contract across
|
||||
chat / sources / viewer (one bar per page, same controls; the
|
||||
viewer bar = back + title + actions).
|
||||
- **§7.5 component inventory:** add `#nav-sources` (Sources nav
|
||||
link, hidden for anonymous), `#new-chat-btn` + `#sign-in-link` +
|
||||
`#sign-out-btn` on sources and viewer pages (ids shared with
|
||||
chat), `.doc-header-actions` (viewer).
|
||||
- **§12 roadmap:** new row 19 — `19_shared_header` /
|
||||
`shared-header.md` / `test_shared_header.py`.
|
||||
- Do not renumber anything or touch other anchors.
|
||||
3. `README.md` — no change (no operator-facing change).
|
||||
4. Final validation pass (all gates, AGENTS.md rules 5 + 9):
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` (≥ today's
|
||||
number),
|
||||
- `uv run pytest tests/e2e/test_shared_header.py -v --no-cov` in
|
||||
isolation, plus the five regression suites from task 02 (one
|
||||
command each, in isolation),
|
||||
- `uv run ruff check . && uv run pyright`.
|
||||
5. Commit + phase move (last step, only when all gates are green):
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/
|
||||
git commit --no-gpg-sign -m "feat(ui): shared header — Sign in/Sign out and New Chat on every page; hide the Sources nav link from anonymous users"
|
||||
mv .agent/phases/todo/19_shared_header .agent/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- No new logic — record-keeping + validation pass; the gates above are
|
||||
the phase's final proof. If validation fails, fix in the owning
|
||||
task's files, re-run that task's tests, then commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `.agent/user_stories/shared-header.md` exists with all five
|
||||
sections (header, narrative, acceptance, UI visualization,
|
||||
Playwright Mapping Rule).
|
||||
- [ ] `.agent/PLAN.md` carries the header-revision/§2-A10-note/§7.1/
|
||||
§7.5/§12 notes with the 2026-08-23 owner-permission wording; the
|
||||
A10 decision text itself is unaltered.
|
||||
- [ ] All gates green (coverage, story E2E + 5 regressions in
|
||||
isolation, ruff + pyright).
|
||||
- [ ] Exactly one new commit, conventional, `--no-gpg-sign`;
|
||||
`.agent/phases/todo/19_shared_header/` is now under
|
||||
`.agent/phases/complete/`.
|
||||
@@ -0,0 +1,122 @@
|
||||
# Phase 20 — Sources Mid-Stream: an in-flight answer is not lost on navigation
|
||||
|
||||
**Source:** `TODO.md` L3 — *"Clicking "sources" while chat is generating
|
||||
clears chat and result will never show up"*
|
||||
**Story:** `.agent/user_stories/sources-midstream.md` (created by task 02)
|
||||
**Context:** `frontend/assets/app.js` — the phase-14 persistence block
|
||||
(`STORAGE_KEY = "bor.chat.v1"`, `conversation`, `saveConversation`,
|
||||
`rememberBrainTurn`), the turn state machine (`UI_STATE.thinking` /
|
||||
`.streaming`), the streaming accumulators (`acc` / `thinkingAcc` /
|
||||
`sawThinking`), and the phase-14 restore path; `frontend/index.html`
|
||||
(`#nav-sources` link); `frontend/assets/header.js` (`clearChatStorage` —
|
||||
the deliberate New-Chat clear, NOT this bug).
|
||||
|
||||
## Objective
|
||||
When the user leaves the chat page (the Sources nav link, the document
|
||||
viewer, any link) while a turn is still in flight, the answer generated so
|
||||
far must not vanish. Today the brain message is persisted only on `done`,
|
||||
so navigating away aborts the stream and the partial answer is lost — the
|
||||
user returns to their own question with no result, ever. After this phase,
|
||||
returning to the chat shows the question **and** the partial answer that
|
||||
had already streamed (rendered like any brain message, thinking block
|
||||
restored if any).
|
||||
|
||||
## Owner-confirmed (2026-08-24, roadmap A1)
|
||||
1. **A partial answer is persisted as a plain brain message** — no
|
||||
"(partial)" marker, no sources/suggestions (the turn is dead; the user
|
||||
can re-ask for the full answer).
|
||||
2. Navigation **before the first answer token** (pure thinking) persists
|
||||
nothing brain-side: the question is restored, no empty/partial bubble.
|
||||
3. The deliberate **New Chat** `clearChatStorage()` (sources/viewer pages)
|
||||
is untouched — that clear is by design (phase 14/19).
|
||||
4. No server-side resume (A10 stays stateless) and no
|
||||
"leave page?" confirmation dialog.
|
||||
|
||||
## Design
|
||||
- **`app.js` — one new `pagehide` handler** (`window.addEventListener(
|
||||
"pagehide", …)` — fires on navigate-away and bfcache store):
|
||||
- Guard: only when a turn is in flight (current `uiState` is
|
||||
`UI_STATE.thinking` or `UI_STATE.streaming`) **and** `acc` is
|
||||
non-empty.
|
||||
- Action: `rememberBrainTurn(acc, { thinking: thinkingAcc ||
|
||||
undefined })` — reuse the existing save-point helper, so the partial
|
||||
text is stored raw (the restore path re-renders through the
|
||||
escape-first markdown renderer; the phase-17 `thinking` field
|
||||
restores the collapsed Thinking block).
|
||||
- **Idempotency guard:** a turn-local `persistedOnLeave` flag so a
|
||||
second `pagehide` (or bfcache store+restore churn) never appends the
|
||||
same partial message twice. The `done` save point is unaffected
|
||||
(navigation means the stream is dead; if the user returns via
|
||||
bfcache the turn is already aborted by the unloading page).
|
||||
- **Restore path:** unchanged — a stored partial message is a well-formed
|
||||
brain message and renders exactly like a completed one (minus
|
||||
sources/deflection, which it simply doesn't carry).
|
||||
- **Non-goals:** no resume of the SSE stream, no API changes, no changes
|
||||
to the New Chat buttons, sign-out, or the document-viewer back link.
|
||||
|
||||
## Dependencies
|
||||
- `14_chat_persistence` (complete) — `bor.chat.v1` shape, save points,
|
||||
restore, and the `rememberBrainTurn` helper this phase reuses.
|
||||
- `17_thinking_display` (complete) — the `thinking` field on persisted
|
||||
brain messages and the `thinkingAcc` accumulator.
|
||||
- `19_shared_header` (complete) — the `#nav-sources` link (admin-only)
|
||||
the bug report clicks.
|
||||
- `18_follow_bottom_scroll` (complete) — no overlap (scroll gating only).
|
||||
|
||||
## Tasks
|
||||
1. `01_persist_inflight_turn.md` — the `pagehide` partial-persistence
|
||||
handler in `app.js` + source-level unit pins.
|
||||
2. `02_e2e_story_suite_commit.md` — `tests/e2e/test_sources_midstream_bug.py`
|
||||
(the story gate, isolated), regression suites, story file, final
|
||||
validation, the single atomic commit, phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A10 untouched** — API stays stateless; no resume. **A11 untouched** —
|
||||
vanilla JS, no CDN. **A16 honored** — one new story E2E suite +
|
||||
adapted regressions. No anchor changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit (source-level, new `tests/unit/test_sources_midstream.py`,
|
||||
following the repo's source-pin pattern):** `app.js` registers a
|
||||
`pagehide` listener; the guard references the in-flight `uiState` and a
|
||||
non-empty `acc`; the partial path calls `rememberBrainTurn` with
|
||||
`thinking: thinkingAcc || undefined`; a turn-local idempotency flag
|
||||
exists; `STORAGE_KEY`/save-point comments updated to list the new
|
||||
save point.
|
||||
- **Integration:** none (no `app/` changes) — the `uv run pytest
|
||||
--cov=app` number must stay at today's.
|
||||
- **Coverage:** frontend-only; the >90% `app/` gate is unaffected,
|
||||
re-run to prove it.
|
||||
- **E2E:** `tests/e2e/test_sources_midstream_bug.py` (task 02), green
|
||||
**in isolation** (prereq `podman compose up -d db`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin, mid-stream, clicks **Sources** → returns to `/`: the
|
||||
question **and** the already-streamed partial answer are both
|
||||
rendered; no error banner; `bor.chat.v1` holds the partial brain
|
||||
message.
|
||||
- [ ] Navigate away before the first token → back: question restored,
|
||||
no empty/partial brain bubble.
|
||||
- [ ] A completed turn is persisted exactly as before (sources,
|
||||
deflection, suggestions intact).
|
||||
- [ ] New Chat from the sources page still clears the conversation.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number.
|
||||
- [ ] `uv run pytest tests/e2e/test_sources_midstream_bug.py -v --no-cov`
|
||||
green in isolation; regressions green in isolation (one command
|
||||
each): `test_chat_persistence.py`, `test_thinking_display.py`,
|
||||
`test_shared_header.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): no new UI surface — the
|
||||
restored partial renders through the existing bubble/thinking
|
||||
contract.
|
||||
- [ ] `.agent/user_stories/sources-midstream.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/20_sources_midstream_bug/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "fix(chat): keep the in-flight answer when navigating away mid-turn — partial answer restored on return"
|
||||
```
|
||||
@@ -0,0 +1,83 @@
|
||||
# Task 01 — app.js: persist the partial answer on navigate-away (pagehide)
|
||||
|
||||
**Phase:** `20_sources_midstream_bug` · **Source:** `TODO.md` L3 —
|
||||
*"Clicking "sources" while chat is generating clears chat and result will
|
||||
never show up"*
|
||||
|
||||
## Objective
|
||||
`frontend/assets/app.js` gains exactly one new save point: on `pagehide`
|
||||
(navigate-away / bfcache), if a turn is in flight and answer text has
|
||||
already streamed, the partial answer is persisted via the existing
|
||||
`rememberBrainTurn` helper — so returning to the chat restores the
|
||||
question **and** what had been generated.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js`
|
||||
- Next to the other turn-local state (around the `acc` /
|
||||
`thinkingAcc` / `sawThinking` / `aborted` declarations, ~line 920+),
|
||||
declare a turn-local flag:
|
||||
```js
|
||||
let persistedOnLeave = false; // pagehide partial-persist at most once
|
||||
```
|
||||
and reset it to `false` at the top of `runTurn` (where `acc`,
|
||||
`thinkingAcc`, `sawThinking`, `wrap` are initialized), so it is
|
||||
turn-scoped like the rest.
|
||||
- Register one handler at module boot (next to the other
|
||||
`window.addEventListener` calls):
|
||||
```js
|
||||
/* Navigate-away save point (phase 20, owner choice 2026-08-24 A1):
|
||||
* leaving the chat mid-turn would otherwise drop the in-flight
|
||||
* answer — the brain message persists only on `done`, and
|
||||
* navigation aborts the stream. On `pagehide`, if a turn is in
|
||||
* flight and answer text has streamed, persist the partial raw text
|
||||
* (reusing the save-point helper, so restore re-renders it exactly
|
||||
* like a completed answer — no "(partial)" marker, no sources).
|
||||
* Thinking-only (no answer tokens yet) persists nothing brain-side:
|
||||
* the question is already saved on send and the user can re-ask.
|
||||
* `persistedOnLeave` makes this idempotent across pagehide/bfcache
|
||||
* churn. */
|
||||
window.addEventListener("pagehide", () => {
|
||||
if (persistedOnLeave) return;
|
||||
if (uiState !== UI_STATE.thinking && uiState !== UI_STATE.streaming)
|
||||
return;
|
||||
if (!acc) return; // nothing brain-side to save yet
|
||||
persistedOnLeave = true;
|
||||
rememberBrainTurn(acc, { thinking: thinkingAcc || undefined });
|
||||
});
|
||||
```
|
||||
**ASSUMPTION (owner-confirmed A1):** the partial is a plain brain
|
||||
message (no marker, no sources); the variables `uiState`, `acc`,
|
||||
`thinkingAcc` are the turn's existing ones — match the names actually
|
||||
in scope (the file keeps `let acc = ""` / `let thinkingAcc = ""`
|
||||
turn-locals inside `runTurn`; if the handler needs them outside that
|
||||
scope, hoist the turn-locals to module scope *without* changing any
|
||||
behavior — smallest diff wins).
|
||||
- Update the phase-14 persistence block comment (~line 631): the
|
||||
"Save points:" sentence now lists three — the user message on send,
|
||||
the brain message on `done`, and the **partial** brain message on
|
||||
navigate-away (`pagehide`, phase 20).
|
||||
2. `tests/unit/test_sources_midstream.py` (new — follow the repo's
|
||||
source-level pin pattern used by `tests/unit/test_shared_header.py`):
|
||||
- `app.js` contains `addEventListener("pagehide"` exactly once.
|
||||
- The pagehide body is guarded by the in-flight states (`thinking`
|
||||
and `streaming`) and a non-empty `acc` check.
|
||||
- The pagehide body calls `rememberBrainTurn(acc,` with
|
||||
`thinking: thinkingAcc || undefined`.
|
||||
- `persistedOnLeave` is declared and reset in `runTurn`.
|
||||
- The persistence block comment lists the `pagehide` save point.
|
||||
- `clearChatStorage` (header.js) is untouched — still the only
|
||||
deliberate clear.
|
||||
|
||||
## Testing & Quality
|
||||
- `uv run pytest tests/unit/test_sources_midstream.py -v` green.
|
||||
- `uv run ruff check . && uv run pyright` clean.
|
||||
- Manual smoke (dev server, DEBUGPY optional): send a question against
|
||||
the real LLM (or any slow stream), click Sources mid-stream, return —
|
||||
the partial answer is visible.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The `pagehide` handler exists, is turn-scoped and idempotent, and
|
||||
reuses `rememberBrainTurn` (no duplicated storage code).
|
||||
- [ ] No other save point changed: send + `done` behave byte-identically
|
||||
to before (existing persistence unit pins still pass).
|
||||
- [ ] Unit pins green; lint/types clean.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Task 02 — E2E story suite, story file, validation, commit
|
||||
|
||||
**Phase:** `20_sources_midstream_bug` · **Source:** `TODO.md` L3
|
||||
|
||||
## Objective
|
||||
The story gate: `tests/e2e/test_sources_midstream_bug.py` proves the bug
|
||||
is fixed end-to-end (navigate away mid-stream, come back, the partial
|
||||
answer is there), plus the regression suites, the story file, final
|
||||
validation, and the single atomic commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_sources_midstream_bug.py` (new — mirror
|
||||
`test_chat_persistence.py`'s scaffolding: fixture import via
|
||||
`_import_fixtures`/`_run_in_thread`, mock LLM on a thread, `login`
|
||||
from `e2e.auth_helpers`). The mock LLM must stream **slowly enough**
|
||||
that the turn is still in flight when the test navigates (reuse the
|
||||
streaming pattern from `test_thinking_display.py` / `mock_llm.py`;
|
||||
tune the per-chunk delay until the navigation lands mid-stream).
|
||||
Tests (Playwright Mapping Rule — one per numbered scenario):
|
||||
1. `test_partial_answer_survives_sources_nav_midstream` — admin
|
||||
(`login(page, app_url, next="/")`), send a question, wait for the
|
||||
first streamed chunk to render (expect the first chunk's text in
|
||||
the answer bubble), **click `#nav-sources`** (the actual nav link —
|
||||
admin sees it), land on `/sources.html`, then `page.goto("/")`:
|
||||
expect the question text AND the first-chunk text present, no
|
||||
`role="alert"` banner. Read `localStorage` `bor.chat.v1`: the
|
||||
messages contain a brain message whose text starts with the first
|
||||
chunk.
|
||||
2. `test_no_orphan_brain_message_when_navigated_before_first_token` —
|
||||
mock streams a `thinking` event, then a long pre-token pause;
|
||||
navigate (direct `page.goto("/sources.html")` is fine here) during
|
||||
the pause, return to `/`: the question is present, exactly one
|
||||
user message and **zero** brain messages in both the DOM and
|
||||
`bor.chat.v1`.
|
||||
3. `test_completed_turn_unaffected` — a turn that finishes normally
|
||||
(`done`), then navigate to sources and back: full answer, sources
|
||||
chips, and the done-metadata (sources array) intact in storage.
|
||||
4. `test_new_chat_still_clears_conversation` — regression: completed
|
||||
turn → `/sources.html` → click the sources-page New Chat button →
|
||||
lands on `/` with the empty state and `bor.chat.v1` removed.
|
||||
2. `.agent/user_stories/sources-midstream.md` (new) — the short story
|
||||
file matching the repo's story format (goal, the bug report verbatim
|
||||
from `TODO.md` L3, the owner-confirmed A1 decisions from
|
||||
`00_phase.md`, the E2E mapping table test-name → scenario).
|
||||
3. Run the suite **in isolation** (prereq `podman compose up -d db`):
|
||||
`uv run pytest tests/e2e/test_sources_midstream_bug.py -v --no-cov`.
|
||||
4. Regressions, in isolation, one command each (all must stay green):
|
||||
- `uv run pytest tests/e2e/test_chat_persistence.py -v --no-cov`
|
||||
- `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
- `uv run pytest tests/e2e/test_shared_header.py -v --no-cov`
|
||||
5. Final validation: `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number (>90% gate);
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
6. **UI Structure Check** (AGENTS.md rule 5): the restored partial
|
||||
renders through the existing bubble/thinking contract — no new
|
||||
surface, no new ids, focus/contrast unchanged.
|
||||
7. Write the phase report
|
||||
(`.agent/reports/20_sources_midstream_bug/` — what changed, E2E
|
||||
results, the manual-smoke note from task 01).
|
||||
8. Commit (one atomic commit) and move the phase:
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/
|
||||
git commit --no-gpg-sign -m "fix(chat): keep the in-flight answer when navigating away mid-turn — partial answer restored on return"
|
||||
mv .agent/phases/todo/20_sources_midstream_bug .agent/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- Story suite green **in isolation**; the three regression suites green
|
||||
in isolation; full unit+integration suite green; `app/` coverage at or
|
||||
above today's number (>90%); ruff + pyright clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `test_sources_midstream_bug.py` 4/4 in isolation.
|
||||
- [ ] Regressions (persistence, thinking display, shared header) green.
|
||||
- [ ] Story file + phase report exist.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory in `complete/`.
|
||||
@@ -0,0 +1,104 @@
|
||||
# Phase 21 — Thinking Window: No Scroll Back, Just Live
|
||||
|
||||
**Source:** `TODO.md` L4 — *"Disable scroll in the thinking window.
|
||||
Users don't need to scroll back through thinking, just see it live."*
|
||||
**Story:** `.agent/user_stories/thinking-no-scroll.md` (created by task 02)
|
||||
**Context:** `frontend/assets/styles.css` —
|
||||
`details.thinking .thinking-text` (today: `max-height: 320px;
|
||||
overflow-y: auto;`, i.e. a user-scrollable 320px window, ~line 444);
|
||||
`frontend/assets/app.js` — the phase-17 thinking block (`ensureThinkingBlock`,
|
||||
the streaming `thinking` branch that appends chunks and already pins the
|
||||
stream to the bottom: `textEl.scrollTop = textEl.scrollHeight`, ~line 886),
|
||||
and the phase-14 restore path (stored `thinking` re-renders a
|
||||
**collapsed** block).
|
||||
|
||||
## Objective
|
||||
The live Thinking block is a scratchpad, not a transcript. The user must
|
||||
not be able to scroll back through it — the 320px window always shows the
|
||||
**live tail** of the reasoning stream (the existing per-chunk
|
||||
bottom-pinning stays). Wheel, drag, and keyboard scrolling on
|
||||
`.thinking-text` stop working; the stream itself keeps pinning to the
|
||||
bottom as chunks arrive.
|
||||
|
||||
## Owner-confirmed (2026-08-24, roadmap A2)
|
||||
1. **Keep the 320px clip** — "just see it live" means the window stays a
|
||||
fixed 320px viewport showing the newest lines; no auto-height growth,
|
||||
no "↓ more" affordance.
|
||||
2. **The answer bubble is untouched** — final answers keep their existing
|
||||
scroll behavior (phase 11 long answers).
|
||||
3. **Restored (collapsed) Thinking blocks are untouched** — the phase-14
|
||||
restore renders them collapsed, where overflow is moot.
|
||||
|
||||
## Design
|
||||
- **CSS (the whole functional change):**
|
||||
`details.thinking .thinking-text` — `overflow-y: auto` →
|
||||
`overflow-y: hidden`; keep `max-height: 320px`.
|
||||
`overflow: hidden` still permits **programmatic** scrolling
|
||||
(`scrollTop`), so the phase-17 pin
|
||||
(`textEl.scrollTop = textEl.scrollHeight` on every thinking chunk)
|
||||
keeps the window glued to the live tail — no JS change needed.
|
||||
Add a CSS comment: *no user scroll back (owner choice 2026-08-24):
|
||||
the window is a live tail only — the JS bottom-pin is the sole
|
||||
scroller*.
|
||||
- **No JS change** — the pin already exists; nothing else touches
|
||||
`.thinking-text` scroll.
|
||||
- **Non-goals:** no change to the answer bubble, the collapsed restore
|
||||
state, the summary/chevron, or the auto-collapse on first delta
|
||||
(phase 17).
|
||||
|
||||
## Dependencies
|
||||
- `17_thinking_display` (complete) — the block, the pin, the restore.
|
||||
- `18_follow_bottom_scroll` (complete) — no overlap (chat-page scroll
|
||||
gate only; the thinking window is a separate inner element).
|
||||
- `11_long_answers` (complete) — the untouched answer-bubble behavior.
|
||||
|
||||
## Tasks
|
||||
1. `01_disable_thinking_scroll.md` — the CSS change + source-level unit
|
||||
pins.
|
||||
2. `02_e2e_story_suite_commit.md` — `tests/e2e/test_thinking_no_scroll.py`
|
||||
(the story gate, isolated), regression suites, story file, final
|
||||
validation, the single atomic commit, phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A11 untouched** — no CDN, pure CSS. **A16 honored** — one new story
|
||||
E2E suite + adapted regressions. No anchor changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit (source-level, new `tests/unit/test_thinking_no_scroll.py`,
|
||||
repo source-pin pattern):** `styles.css` carries
|
||||
`details.thinking .thinking-text` with `overflow-y: hidden` and
|
||||
`max-height: 320px`; the phase-17 pin line
|
||||
(`textEl.scrollTop = textEl.scrollHeight`) still present in `app.js`
|
||||
(the live-tail mechanism must not be lost).
|
||||
- **Integration:** none (no `app/` changes).
|
||||
- **Coverage:** frontend-only; the >90% `app/` gate is unaffected.
|
||||
- **E2E:** `tests/e2e/test_thinking_no_scroll.py` (task 02), green
|
||||
**in isolation** (prereq `podman compose up -d db`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With a long thinking stream, wheel/mouse-drag/keyboard on
|
||||
`.thinking-text` do **not** move it; the visible content is always
|
||||
the live tail (`scrollTop === scrollHeight` after each chunk,
|
||||
within 1px).
|
||||
- [ ] Computed style: `overflow-y: hidden`, `max-height: 320px`.
|
||||
- [ ] A long **answer** bubble still scrolls normally; a restored
|
||||
collapsed Thinking block still renders (phase 17 regression).
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number.
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_no_scroll.py -v --no-cov`
|
||||
green in isolation; regressions green in isolation:
|
||||
`test_thinking_display.py`, `test_long_answers.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): no new surface; the block
|
||||
keeps its focus-visible summary, aria contract, and reduced-motion
|
||||
behavior.
|
||||
- [ ] `.agent/user_stories/thinking-no-scroll.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/21_thinking_no_scroll/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "fix(ui): thinking window no longer scrolls — live 320px view pinned to the stream tail"
|
||||
```
|
||||
@@ -0,0 +1,57 @@
|
||||
# Task 01 — styles.css: `.thinking-text` overflow hidden (no user scroll)
|
||||
|
||||
**Phase:** `21_thinking_no_scroll` · **Source:** `TODO.md` L4 —
|
||||
*"Disable scroll in the thinking window. Users don't need to scroll back
|
||||
through thinking, just see it live."*
|
||||
|
||||
## Objective
|
||||
One CSS property change makes the Thinking window a live-tail-only view:
|
||||
`overflow-y: hidden` instead of `auto`, keeping the 320px clip. The
|
||||
phase-17 JS bottom-pin (which keeps working under `overflow: hidden`) is
|
||||
the sole scroller.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/styles.css` — in the phase-17 thinking block
|
||||
section (~line 444):
|
||||
```css
|
||||
details.thinking .thinking-text {
|
||||
padding: 0 0.75rem 0.75rem;
|
||||
color: var(--ink-soft); /* 6.9:1 on --surface */
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.55;
|
||||
max-height: 320px;
|
||||
overflow-y: hidden; /* no scroll back (owner choice 2026-08-24):
|
||||
the window is a live tail only — the phase-17
|
||||
JS bottom-pin (scrollTop = scrollHeight per
|
||||
chunk) is the sole scroller */
|
||||
}
|
||||
```
|
||||
(Only the `overflow-y` value + comment change; every other declaration
|
||||
stays byte-identical.)
|
||||
2. `frontend/assets/app.js` — **no change expected.** Verify the pin is
|
||||
intact: the streaming `thinking` branch still does
|
||||
`textEl.scrollTop = textEl.scrollHeight` on every chunk (~line 886).
|
||||
If (and only if) the pin were missing/broken, fix it — do not remove
|
||||
or alter any other scrolling behavior.
|
||||
3. `tests/unit/test_thinking_no_scroll.py` (new — repo source-pin
|
||||
pattern):
|
||||
- `styles.css`: the `details.thinking .thinking-text` rule contains
|
||||
`overflow-y: hidden` and `max-height: 320px` (no `overflow-y: auto`
|
||||
left in that rule).
|
||||
- `app.js`: the bottom-pin line
|
||||
`textEl.scrollTop = textEl.scrollHeight` is still present (the
|
||||
live-tail mechanism).
|
||||
4. Manual smoke (dev server): stream a long thinking turn; try to wheel /
|
||||
drag / Tab+ArrowDown inside the Thinking block — it must not move;
|
||||
the newest chunk is always the one visible at the bottom.
|
||||
|
||||
## Testing & Quality
|
||||
- `uv run pytest tests/unit/test_thinking_no_scroll.py -v` green.
|
||||
- `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `.thinking-text` is `overflow-y: hidden`, `max-height: 320px`, with
|
||||
the owner-choice comment.
|
||||
- [ ] The JS bottom-pin is verified intact (no app.js diff unless the
|
||||
pin was broken).
|
||||
- [ ] Unit pins green; lint/types clean; manual smoke passed.
|
||||
@@ -0,0 +1,114 @@
|
||||
# Phase 22 — Animated Background: Make It Actually Animate
|
||||
|
||||
**Source:** `TODO.md` L5 — *"Fix background animation not working, just
|
||||
blinking"*
|
||||
**Story:** `.agent/user_stories/background-animation.md` (created by task 02)
|
||||
**Context:** `frontend/assets/styles.css` — the phase-08 animated
|
||||
background block (~lines 60–100): `body::before` (44px drifting grid,
|
||||
1px lines at ~35% `--line` alpha, radial mask
|
||||
`radial-gradient(120% 90% at 50% 0%, black 25%, transparent 78%)`,
|
||||
`animation: bg-grid-drift 60s linear infinite`) and `body::after` (two
|
||||
soft radial glows, `animation: bg-glow-breathe 14s ease-in-out infinite
|
||||
alternate`, opacity 0.65↔1 + scale 1↔1.05). Both layers are
|
||||
`position: fixed; inset: 0; z-index: -1; pointer-events: none`. `html`
|
||||
owns the `var(--bg)` canvas and `body` is `background: transparent`
|
||||
(~lines 42–54) — if any later rule occludes that, the layers vanish.
|
||||
The **phase-08 design comments are the spec** for what "working" means.
|
||||
|
||||
## Objective
|
||||
Owner report 2026-08-24: the background "just blinks" — i.e. the motion
|
||||
the phase-08 design promised (a slow, seamless grid drift + a gentle
|
||||
glow breathe) is not perceived; at most a flicker/blink is visible.
|
||||
Diagnose which layer(s) actually fail in a real Chromium viewport, fix
|
||||
the CSS, and leave a background that visibly and smoothly animates as
|
||||
designed — no blink, no static frame, no jank.
|
||||
|
||||
## Owner-confirmed (2026-08-24, roadmap A3)
|
||||
1. **Intended effect = the phase-08 design comments:** seamless 60s grid
|
||||
drift (one cell per loop) + 14s ease glow breathing. The fix serves
|
||||
that design, not a redesign.
|
||||
2. **Pure CSS, zero JS** (phase-08 anchor) — no animation JS, no new
|
||||
assets, no `filter: blur` (perf note in the block).
|
||||
|
||||
## Design / diagnostic plan
|
||||
The fix is found, not guessed — work through this checklist in a real
|
||||
Chromium window (dev server, full page, ~15s of observation):
|
||||
1. **Per-layer visibility:** toggle each pseudo-element (DevTools
|
||||
generated-content / a temp outline) and screenshot — is the grid
|
||||
visible at all? Is only the glow (the "blink" the user perceives)
|
||||
alive?
|
||||
2. **Grid layer:** sample `background-position` on `body::before` at two
|
||||
timestamps — is it actually moving? Is the radial mask fading the
|
||||
visible region so small that the 44px/60s drift is imperceptible?
|
||||
(If the drift is real but too faint: raise the grid line alpha and/or
|
||||
the mask's visible radius — smallest change that reads as "smooth
|
||||
drift".)
|
||||
3. **Glow layer:** is the 14s breathe reading as a *blink*? (If the
|
||||
opacity swing 0.65↔1 is perceived as pulsing: lengthen the period
|
||||
and/or narrow the opacity delta so it reads as breathing.)
|
||||
4. **Occlusion check:** confirm nothing later in `styles.css` (or in
|
||||
`html`/`body` rules) paints an opaque background over the
|
||||
`z-index: -1` layers — the phase-08 comment at ~line 42 is the
|
||||
contract.
|
||||
5. **Apply the fix in `styles.css`** — document the found root cause in
|
||||
the phase report (screenshot before/after in
|
||||
`.agent/screenshots/22_background_animation/`).
|
||||
|
||||
## Dependencies
|
||||
- `08_story_dark_tech_theme` (complete) — owns the layers, the palette,
|
||||
and the "pure CSS, zero JS" anchor this phase must respect.
|
||||
- `07_story_responsive_polish` (complete) — no new overflow at 360px
|
||||
(both layers are `fixed; inset: 0` — keep it that way).
|
||||
|
||||
## Tasks
|
||||
1. `01_fix_background_animation.md` — diagnosis + the CSS fix +
|
||||
source-level unit pins.
|
||||
2. `02_e2e_story_suite_commit.md` — `tests/e2e/test_background_animation.py`
|
||||
(the story gate, isolated), regression suites, story file, final
|
||||
validation, the single atomic commit, phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Phase-08 anchor honored** — pure CSS, zero JS, no `filter: blur`,
|
||||
WCAG AA palette untouched (background layers carry no text).
|
||||
**A11 untouched** — no CDN, no new assets. **A16 honored** — one new
|
||||
story E2E suite + adapted regressions. No anchor changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit (source-level, new `tests/unit/test_background_animation.py`,
|
||||
repo source-pin pattern):** `styles.css` still defines
|
||||
`@keyframes bg-grid-drift` and `@keyframes bg-glow-breathe`;
|
||||
`body::before` animates `bg-grid-drift` with `linear infinite`;
|
||||
`body::after` animates `bg-glow-breathe`; both layers remain
|
||||
`position: fixed; z-index: -1; pointer-events: none`; `html` keeps
|
||||
`background: var(--bg)` and `body` keeps `background: transparent`
|
||||
(the no-occlusion contract). Pin the **final** values the fix lands
|
||||
on (durations/opacities may move per the design plan).
|
||||
- **Integration:** none (no `app/` changes).
|
||||
- **Coverage:** frontend-only; the >90% `app/` gate is unaffected.
|
||||
- **E2E:** `tests/e2e/test_background_animation.py` (task 02), green
|
||||
**in isolation** (prereq `podman compose up -d db`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] In a real Chromium viewport, the background visibly and smoothly
|
||||
animates (grid drift + glow breathe) — screenshot before/after in
|
||||
the phase report; owner's "just blinking" perception gone.
|
||||
- [ ] Root cause documented in `.agent/reports/22_background_animation/`.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number.
|
||||
- [ ] `uv run pytest tests/e2e/test_background_animation.py -v --no-cov`
|
||||
green in isolation; regressions green in isolation:
|
||||
`test_dark_tech_theme.py`, `test_responsive_polish.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): layers stay behind content
|
||||
(`z-index: -1`, `pointer-events: none`), no text/contrast impact,
|
||||
no 360px overflow.
|
||||
- [ ] `.agent/user_stories/background-animation.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agent/phases/todo/22_background_animation/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "fix(ui): animated background actually animates — grid drift and glow breathe per the phase-08 design"
|
||||
```
|
||||
@@ -0,0 +1,68 @@
|
||||
# Task 02 — E2E story suite, story file, validation, commit
|
||||
|
||||
**Phase:** `22_background_animation` · **Source:** `TODO.md` L5
|
||||
|
||||
## Objective
|
||||
The story gate: `tests/e2e/test_background_animation.py` proves both
|
||||
background layers are actually running animations (not just declared),
|
||||
plus regressions, story file, final validation, and the single atomic
|
||||
commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_background_animation.py` (new). The layers are CSS
|
||||
pseudo-elements, so assert via computed style + the Web Animations
|
||||
API (Chromium reports pseudo-element CSS animations through
|
||||
`element.getAnimations()`):
|
||||
1. `test_grid_layer_animation_running` —
|
||||
`getComputedStyle(document.body, "::before").animationName` is the
|
||||
grid-drift keyframe (final name from task 01), timing function
|
||||
`linear`, iteration count `infinite`; and a matching entry in
|
||||
`document.body.getAnimations()` with `playState === "running"`.
|
||||
2. `test_glow_layer_animation_running` — same for `"::after"` with
|
||||
the glow-breathe keyframe; `playState === "running"`.
|
||||
3. `test_animations_advance` — sample `animation.currentTime` (or
|
||||
the `getAnimations()` entry's `currentTime`) for both layers,
|
||||
wait ~500ms (`page.wait_for_timeout`), assert both advanced —
|
||||
the animations are truly running, not paused.
|
||||
4. `test_background_layers_contracts` — both pseudo-elements:
|
||||
`position: fixed`, `z-index: -1`, `pointer-events: none`;
|
||||
`document.documentElement` computed `background-color` is the
|
||||
palette bg (the canvas stays on `html`); `document.body` computed
|
||||
`background-color` is `rgba(0, 0, 0, 0)` (no occlusion).
|
||||
5. `test_no_horizontal_overflow_with_layers` (regression, 360px) —
|
||||
viewport 360px: `document.documentElement.scrollWidth <=
|
||||
clientWidth` (the phase-07 pin, replicated locally).
|
||||
2. `.agent/user_stories/background-animation.md` (new) — story file per
|
||||
the repo format: goal, the bug report verbatim from `TODO.md` L5, the
|
||||
owner-confirmed A3 decisions + the found root cause (from task 01's
|
||||
report), E2E mapping table.
|
||||
3. Run the suite **in isolation** (prereq `podman compose up -d db`):
|
||||
`uv run pytest tests/e2e/test_background_animation.py -v --no-cov`.
|
||||
4. Regressions, in isolation, one command each:
|
||||
- `uv run pytest tests/e2e/test_dark_tech_theme.py -v --no-cov`
|
||||
- `uv run pytest tests/e2e/test_responsive_polish.py -v --no-cov`
|
||||
5. Final validation: `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` ≥ today's number (>90% gate);
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
6. **UI Structure Check** (AGENTS.md rule 5): layers stay behind
|
||||
content, no text/contrast impact, no overflow at 360px.
|
||||
7. Finish the phase report (`.agent/reports/22_background_animation/` —
|
||||
E2E results + the task-01 screenshots).
|
||||
8. Commit (one atomic commit) and move the phase:
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/
|
||||
git commit --no-gpg-sign -m "fix(ui): animated background actually animates — grid drift and glow breathe per the phase-08 design"
|
||||
mv .agent/phases/todo/22_background_animation .agent/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- Story suite green **in isolation**; both regression suites green in
|
||||
isolation; full unit+integration suite green; `app/` coverage at or
|
||||
above today's number (>90%); ruff + pyright clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `test_background_animation.py` 5/5 in isolation.
|
||||
- [ ] Regressions (dark tech theme, responsive polish) green in
|
||||
isolation.
|
||||
- [ ] Story file + phase report (with screenshots) exist.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory in `complete/`.
|
||||
@@ -0,0 +1,180 @@
|
||||
# Phase 25 — Background: No Motion, Only Fading Light
|
||||
|
||||
**Source:** Owner report (2026-08-25, chat): the background "jitters down
|
||||
and to the right every second and it slowly blinks brighter and darker. It
|
||||
should be smooth, fluxuating, dimming and brightening, but not moving.
|
||||
Different bright spots should slowly fade in and out."
|
||||
**Story:** `.agent/user_stories/background-no-motion.md` (created by task 02)
|
||||
**Context:** `frontend/assets/styles.css` — the background block:
|
||||
`body::before` (44px grid, 1px lines at 60% `--line` alpha, widened radial
|
||||
mask, `animation: bg-grid-drift 60s linear infinite` → `0 0` →
|
||||
`44px 44px` ≈ 0.73px/s down-right) and `body::after` (indigo + cyan radial
|
||||
glows, `animation: bg-glow-breathe 14s ease-in-out infinite alternate` →
|
||||
opacity 0.85↔1 + scale 1↔1.05). Both are
|
||||
`position: fixed; inset: 0; z-index: -1; pointer-events: none`; `<html>`
|
||||
owns the `var(--bg)` canvas, `<body>` stays transparent (no-occlusion
|
||||
contract). Pins to adapt: `tests/unit/test_background_animation.py`,
|
||||
`tests/e2e/test_background_animation.py`,
|
||||
`tests/e2e/test_dark_tech_theme.py` (`test_animated_background` pins
|
||||
60s/14s; `test_reduced_motion_honored` pins the two body pseudo-layers).
|
||||
|
||||
## Objective
|
||||
Stop the background from moving entirely, and replace the uniform
|
||||
whole-layer "blink" with **different bright spots that slowly fade in and
|
||||
out**: the grid becomes a static texture, and three soft glow spots
|
||||
(phase-08 colors/positions, plus a third spot) each run their own
|
||||
slow, **opacity-only** fade cycle at a different period, so the
|
||||
background's brightness fluxuates smoothly and irregularly — no blink, no
|
||||
jitter, no motion.
|
||||
|
||||
## Root cause (found from the code, 2026-08-25)
|
||||
1. **"Jitters down and to the right every second"** = `bg-grid-drift`:
|
||||
44px/60s (≈0.73px/s) in the diagonal `44px 44px` direction (exactly
|
||||
down-right). A 1px grid line translated sub-pixel-by-sub-pixel is
|
||||
rasterized with per-frame stepping/shimmer — perceived as a once-per-
|
||||
second jitter, not smooth drift. Phase 22 made the drift *visible*;
|
||||
that is precisely why it now reads as jitter.
|
||||
2. **"Slowly blinks brighter and darker"** = `bg-glow-breathe`: a uniform
|
||||
whole-layer opacity swing 0.85↔1 over 14s (alternate) plus
|
||||
`scale(1)↔scale(1.05)` — the entire background pulses in unison (the
|
||||
scale adds a faint zoom). One synchronized pulse reads as a blink; the
|
||||
owner wants independent spots instead.
|
||||
|
||||
Phase 22 served the phase-08 design intent (grid drift + whole-layer
|
||||
breathe). The owner now supersedes that design intent — this is an
|
||||
owner revision of a **design comment**, not of any LOCKED anchor: A1–A17
|
||||
are untouched, and the phase-08 anchors this phase must still honor are
|
||||
pure CSS / zero JS / no CDN / no new assets / no `filter: blur` (A11 +
|
||||
the block's perf note).
|
||||
|
||||
## Owner direction (2026-08-25, verbatim)
|
||||
> "It should be smooth, fluxuating, dimming and brightening, but not
|
||||
> moving. Different bright spots should slowly fade in and out."
|
||||
|
||||
1. **No movement** — no grid drift, no `scale`/`transform`, no
|
||||
`background-position` animation, anywhere in the background.
|
||||
2. **Fluxuating brightness** — overall page brightness varies smoothly and
|
||||
irregularly (not one synchronized pulse).
|
||||
3. **Different bright spots** — multiple glow spots, each fading in and
|
||||
out on its own slow cycle.
|
||||
4. **The static grid stays** — the owner rejected the grid's *motion*, not
|
||||
the grid; it remains as a still texture. (If the owner later wants the
|
||||
grid gone, that is a follow-up, not this phase.)
|
||||
|
||||
## Design (pure CSS, zero JS — A11 anchor)
|
||||
- `body::before` — grid: **remove** the `animation`; **delete**
|
||||
`@keyframes bg-grid-drift`. Keep the 44px cells, 60% `--line` alpha
|
||||
lines, and widened radial mask (the static texture).
|
||||
- Three glow-spot layers, one soft radial gradient each, **opacity-only**
|
||||
keyframes (`0%,100%` low → `50%` 1, `ease-in-out`, `infinite`), with
|
||||
different durations + negative delays so the cycles are out of phase
|
||||
(periods 26/34/42s → LCM 4641s, the composite pattern effectively never
|
||||
repeats within a viewing session):
|
||||
|
||||
| layer | spot (gradient) | keyframes | cycle |
|
||||
|---|---|---|---|
|
||||
| `body::after` | indigo `rgb(109 120 242 / 0.14)`, circle 56rem at 12% 8% (phase-08) | `bg-glow-a` | 26s, low opacity 0.25 |
|
||||
| `html::before` | cyan `rgb(34 211 238 / 0.10)`, circle 60rem at 88% 92% (phase-08) | `bg-glow-b` | 34s, delay −12s, low 0.20 |
|
||||
| `html::after` | indigo `rgb(109 120 242 / 0.09)`, circle 52rem at 14% 86% | `bg-glow-c` | 42s, delay −23s, low 0.15 |
|
||||
|
||||
- `html::before` / `html::after` join `body::after` as background layers:
|
||||
`<html>` is the root stacking context — its `z-index: -1`
|
||||
pseudo-elements paint **above** the `var(--bg)` canvas and **below**
|
||||
the transparent, non-stacking `<body>`'s content, so the no-occlusion
|
||||
contract holds unchanged (verify in the E2E, not just assume).
|
||||
- All four layers keep: `content: ""; position: fixed; inset: 0;
|
||||
z-index: -1; pointer-events: none;`
|
||||
- `prefers-reduced-motion: reduce` stills **all four** layers
|
||||
(`animation: none`).
|
||||
- No `filter` (phase-08 no-blur perf anchor), no JS, no new assets; the
|
||||
WCAG palette and every text contrast pair are untouched (the layers
|
||||
carry no text). Opacity-only keyframes stay compositor-friendly.
|
||||
|
||||
## Dependencies
|
||||
- `08_story_dark_tech_theme` (complete) — the layers, the palette, the
|
||||
pure-CSS / no-blur / no-CDN anchors, and the 60s/14s pins in
|
||||
`tests/e2e/test_dark_tech_theme.py` this phase adapts.
|
||||
- `22_background_animation` (complete) — the current implementation and
|
||||
the unit/E2E pins this phase supersedes.
|
||||
- `07_story_responsive_polish` (complete) — the 360px overflow pin (the
|
||||
layers are `fixed; inset: 0` — they must add no width).
|
||||
|
||||
## Tasks
|
||||
1. `01_still_background_css.md` — the CSS redesign (static grid + three
|
||||
opacity-only glow fades) + unit source pins (new
|
||||
`tests/unit/test_background_no_motion.py`; the phase-22 pins in
|
||||
`tests/unit/test_background_animation.py` adapted).
|
||||
2. `02_e2e_story_suite_commit.md` — `tests/e2e/test_background_no_motion.py`
|
||||
(the story gate, isolated), the regression suites adapted
|
||||
(`test_background_animation.py`, `test_dark_tech_theme.py`), the story
|
||||
file, the phase report + screenshots, final validation, the single
|
||||
atomic commit, phase move to `complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A11 untouched** — vanilla HTML/CSS/JS in git, no CDN, zero JS, system
|
||||
fonts; the whole change is CSS. **No `filter`/`blur`** (phase-08 perf
|
||||
anchor). No new assets. **No anchor changed** — the superseded spec is
|
||||
the phase-08 *design intent* (grid drift + whole-layer breathe), not a
|
||||
LOCKED decision; the owner's 2026-08-25 direction is recorded above as
|
||||
the revision. **A16 honored** — one new story E2E suite + adapted
|
||||
regressions. **A17 honored** — one atomic `--no-gpg-sign` commit.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit (source-level, repo source-pin pattern):**
|
||||
- New `tests/unit/test_background_no_motion.py` — pins the full new
|
||||
contract (task 01, step 6).
|
||||
- Adapted `tests/unit/test_background_animation.py` — the phase-22
|
||||
drift/breathe pins flip to the new contract; the generic
|
||||
layer-plumbing and no-blur/no-JS sections stay (task 01, step 7).
|
||||
- **Integration:** none (no `app/` changes).
|
||||
- **Coverage:** frontend-only; the >90% `app/` gate is unaffected (TOTAL
|
||||
must stay ≥ the pre-change number).
|
||||
- **E2E:** `tests/e2e/test_background_no_motion.py` (task 02), green
|
||||
**in isolation** (prereq `podman compose up -d db`); regressions green
|
||||
in isolation after adaptation: `test_background_animation.py`,
|
||||
`test_dark_tech_theme.py`, `test_responsive_polish.py`.
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] No movement: the grid is static (`bg-grid-drift` gone, no
|
||||
`animation` on `body::before`) and **no `bg-*` keyframe animates
|
||||
anything but `opacity`** — audited in a real Chromium via
|
||||
`document.styleSheets` (E2E test 3).
|
||||
- [ ] Three distinct bright spots (`body::after`, `html::before`,
|
||||
`html::after`) run distinct slow opacity fades (26s/34s/42s,
|
||||
out of phase); the timelines advance; the layer opacity and a
|
||||
clipped screenshot of the glow region measurably change within a
|
||||
few seconds (E2E tests 2, 4, 5).
|
||||
- [ ] Contracts hold: all four layers `fixed; inset: 0; z-index: -1;
|
||||
pointer-events: none`; `<html>` keeps the `var(--bg)` canvas and
|
||||
`<body>` stays transparent (no occlusion); `prefers-reduced-motion`
|
||||
stills all four; no horizontal overflow at 360px (E2E tests 6–8).
|
||||
- [ ] `.agent/reports/25_background_no_motion/` documents the two root
|
||||
causes with before/after screenshot pairs
|
||||
(`.agent/screenshots/25_background_no_motion/`).
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app
|
||||
--cov-report=term-missing` TOTAL ≥ pre-change number (gate >90%).
|
||||
- [ ] `uv run pytest tests/e2e/test_background_no_motion.py -v --no-cov`
|
||||
green in isolation; `test_background_animation.py`,
|
||||
`test_dark_tech_theme.py`, `test_responsive_polish.py` green in
|
||||
isolation after adaptation.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): layers stay behind content,
|
||||
no text/contrast impact, no 360px overflow.
|
||||
- [ ] `.agent/user_stories/background-no-motion.md` exists; the old
|
||||
`background-animation.md` story carries a supersession note.
|
||||
- [ ] One `--no-gpg-sign` commit staging **only this phase's files** (the
|
||||
unrelated dirty `TODO.md` must NOT be staged);
|
||||
`.agent/phases/todo/25_background_no_motion/` moved to
|
||||
`.agent/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -f .agent/phases/complete/25_background_no_motion \
|
||||
.agent/user_stories/background-no-motion.md \
|
||||
.agent/user_stories/background-animation.md \
|
||||
.agent/reports/25_background_no_motion \
|
||||
.agent/screenshots/25_background_no_motion
|
||||
git add frontend/assets/styles.css tests/unit tests/e2e
|
||||
git commit --no-gpg-sign -m "fix(ui): background no longer moves — static grid, three glow spots fading in and out on their own slow cycles (owner 2026-08-25)"
|
||||
```
|
||||
@@ -0,0 +1,149 @@
|
||||
# Task 02 — Story E2E Suite, Regression Adaptations, Story File, Commit
|
||||
|
||||
**Phase:** `25_background_no_motion` · **Story:** `.agent/user_stories/background-no-motion.md`
|
||||
|
||||
## Objective
|
||||
Prove the new behavior in a real Chromium viewport (no movement,
|
||||
independent slow fades, all layer contracts), adapt the two
|
||||
phase-08/22 E2E suites to the new contract, write the story file and the
|
||||
phase report, run the final validation, and land the single atomic
|
||||
commit.
|
||||
|
||||
## Work
|
||||
1. **New `tests/e2e/test_background_no_motion.py`** — the story gate,
|
||||
run in isolation (fixtures as in `tests/e2e/test_background_animation.py`:
|
||||
`page`, `app_url`, `db_ready`; keep that file's Chromium notes:
|
||||
pseudo-element CSS animations are enumerated by
|
||||
`document.getAnimations()`, and the html pseudo-layers' computed
|
||||
styles come from
|
||||
`getComputedStyle(document.documentElement, "::before"/"::after")`).
|
||||
Tests (8):
|
||||
1. `test_grid_layer_is_static` — computed `animationName` of
|
||||
`body::before` is `"none"`; no `bg-grid-drift` entry in
|
||||
`document.getAnimations()`; the grid `backgroundImage` is still
|
||||
present (the static texture survives).
|
||||
2. `test_three_glow_layers_run_distinct_fades` — `body::after` →
|
||||
`bg-glow-a` (26s), `documentElement::before` → `bg-glow-b` (34s),
|
||||
`documentElement::after` → `bg-glow-c` (42s); each
|
||||
`ease-in-out` + `infinite`, with a matching
|
||||
`playState === "running"` entry in the document animation list;
|
||||
the three durations are pairwise distinct.
|
||||
3. `test_no_motion_properties_in_background_keyframes` — walk
|
||||
`document.styleSheets`; for every `CSSRule.KEYFRAMES_RULE` whose
|
||||
name starts with `bg-`, collect the declared property names of
|
||||
every keyframe frame (iterate `frame.style`); the set across all
|
||||
frames is exactly `{"opacity"}` — the deterministic no-movement
|
||||
proof (no `transform`/`background-position` anywhere).
|
||||
4. `test_glow_timelines_advance` — poll until all three timelines
|
||||
report `currentTime > 0` (the phase-22 pattern: headless Chromium
|
||||
starts the document timeline ~1s after load), sample all three,
|
||||
wait ~500ms, each advanced ≥ 200ms.
|
||||
5. `test_background_light_actually_changes` — (a) read the computed
|
||||
`opacity` of `body::after` (or `documentElement::before`) at t0 and
|
||||
poll up to ~8s until |Δ| ≥ 0.05 (a real fade, not a frozen frame);
|
||||
(b) take two clipped screenshots ~4s apart of the bottom-left glow
|
||||
region (the `html::after` spot at 14%/86%, e.g. clip
|
||||
`{"x": 0, "y": 500, "width": 500, "height": 300}`) and assert the
|
||||
bytes differ — the light visibly changes while nothing moves
|
||||
(a fresh `/` page has no other animation, so the diff is the
|
||||
background's).
|
||||
6. `test_background_layers_contracts` — all four pseudo-elements
|
||||
(`body::before`, `body::after`, `documentElement::before`,
|
||||
`documentElement::after`): `position: fixed`, `z-index: -1`,
|
||||
`pointer-events: none`, top/right/bottom/left all `0px`;
|
||||
`documentElement` computed background is `rgb(10, 14, 23)`
|
||||
(`var(--bg)` — the canvas stays on `html`); `document.body`
|
||||
computed background is `rgba(0, 0, 0, 0)` (no occlusion).
|
||||
7. `test_reduced_motion_stills_all_layers` — `reduced_motion="reduce"`
|
||||
context: all four pseudo-layers report computed `animationName`
|
||||
`"none"` and still carry a `backgroundImage` (static background
|
||||
remains visible).
|
||||
8. `test_no_horizontal_overflow_with_layers` — 360px viewport:
|
||||
`documentElement.scrollWidth <= clientWidth` (the phase-07 pin).
|
||||
2. **Adapt `tests/e2e/test_background_animation.py`** (the phase-22 story
|
||||
suite — now a regression): replace `test_grid_layer_animation_running`,
|
||||
`test_glow_layer_animation_running`, `test_animations_advance` with the
|
||||
new-contract equivalents (grid static; `body::after` runs
|
||||
`bg-glow-a` running; the glow timelines advance — delegate to the
|
||||
same JS-report pattern); keep and extend
|
||||
`test_background_layers_contracts` to the two new html pseudo-layers;
|
||||
keep `test_no_horizontal_overflow_with_layers`; rewrite the module
|
||||
docstring (phase 25 supersedes the phase-22 pins; pointer to
|
||||
`background-no-motion.md`).
|
||||
3. **Adapt `tests/e2e/test_dark_tech_theme.py`:**
|
||||
- `test_animated_background` — the 60s/14s recipe is gone; new
|
||||
contract: `body::before` `animationName == "none"` with its grid
|
||||
image present (static texture); the three glow layers run
|
||||
`bg-glow-a/b/c` at 26s/34s/42s (read the html pseudo-layers off
|
||||
`documentElement`); update the docstring/AC text accordingly.
|
||||
- `test_reduced_motion_honored` — assert **all four** pseudo-layers
|
||||
(body `::before`/`::after` + documentElement `::before`/`::after`)
|
||||
report `animationName` `"none"` and keep their images.
|
||||
4. **`.agent/user_stories/background-no-motion.md`** — the story file, in
|
||||
the repo's story format (model it on `background-animation.md`):
|
||||
verbatim owner report (2026-08-25); Given/When/Then narrative; root
|
||||
causes (sub-pixel 0.73px/s grid drift = once-per-second down-right
|
||||
jitter; whole-layer 0.85↔1 + scale breathe = uniform blink); the fix
|
||||
table (phase-22 → phase-25: grid static; three spots, opacity-only,
|
||||
26/34/42s, delays 0/−12s/−23s, lows 0.25/0.20/0.15); acceptance
|
||||
criteria = the 8 E2E tests + unit pins + regressions; UI
|
||||
Visualization & Structure (the four layers, stacking/no-occlusion,
|
||||
opacity-only motion, reduced-motion); Playwright Mapping Rule
|
||||
(test → `tests/e2e/test_background_no_motion.py`).
|
||||
5. **`.agent/user_stories/background-animation.md`** — add a short
|
||||
supersession note at the top (the motion design is superseded by the
|
||||
owner direction 2026-08-25 — see `background-no-motion.md`); do not
|
||||
rewrite the phase-22 history.
|
||||
6. **`.agent/reports/25_background_no_motion/`** — short report: the two
|
||||
root causes (with the phase-22 measurements as context), the design
|
||||
change, and the screenshot pairs: the task-01 `before.png`/
|
||||
`before_4s.png` (jitter + uniform pulse) plus a new `after.png`/
|
||||
`after_4s.png` pair captured the same way (the after pair must show a
|
||||
brightness change with no positional shift of the grid).
|
||||
7. **Final validation** (in order; DB up: `podman compose up -d db`):
|
||||
- `uv run pytest` (unit + integration)
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` — TOTAL ≥
|
||||
pre-change, app/ gate >90%
|
||||
- `uv run pytest tests/e2e/test_background_no_motion.py -v --no-cov`
|
||||
(the story gate, **in isolation**)
|
||||
- `uv run pytest tests/e2e/test_background_animation.py -v --no-cov`
|
||||
(in isolation, adapted)
|
||||
- `uv run pytest tests/e2e/test_dark_tech_theme.py -v --no-cov` and
|
||||
`uv run pytest tests/e2e/test_responsive_polish.py -v --no-cov`
|
||||
(in isolation, regressions)
|
||||
- `uv run ruff check . && uv run pyright`
|
||||
8. **One atomic commit** — stage **only this phase's files**; the
|
||||
unrelated dirty `TODO.md` in the worktree must NOT be staged:
|
||||
```bash
|
||||
mv .agent/phases/todo/25_background_no_motion .agent/phases/complete/25_background_no_motion
|
||||
git add -f .agent/phases/complete/25_background_no_motion \
|
||||
.agent/user_stories/background-no-motion.md \
|
||||
.agent/user_stories/background-animation.md \
|
||||
.agent/reports/25_background_no_motion \
|
||||
.agent/screenshots/25_background_no_motion
|
||||
git add frontend/assets/styles.css tests/unit tests/e2e
|
||||
git commit --no-gpg-sign -m "fix(ui): background no longer moves — static grid, three glow spots fading in and out on their own slow cycles (owner 2026-08-25)"
|
||||
```
|
||||
(Move the directory first so the committed copy lives in `complete/`;
|
||||
`.agent/` is gitignored by design, hence `git add -f`.)
|
||||
|
||||
## Testing & Quality
|
||||
- The new E2E suite is the story gate (A16): green **in isolation**;
|
||||
the adapted regression suites green in isolation; `test_responsive_
|
||||
polish.py` green in isolation.
|
||||
- `app/` coverage >90% and TOTAL ≥ pre-change (`app/` is untouched).
|
||||
- ruff + pyright clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All 8 tests in `tests/e2e/test_background_no_motion.py` pass in
|
||||
isolation.
|
||||
- [ ] `test_background_animation.py` and `test_dark_tech_theme.py`
|
||||
adapted and green in isolation; `test_responsive_polish.py` green
|
||||
in isolation.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app` TOTAL ≥
|
||||
pre-change; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Story file exists; the old story carries the supersession note;
|
||||
the report + four screenshots exist.
|
||||
- [ ] Exactly one `--no-gpg-sign` commit, staging only this phase's
|
||||
files (`git status` shows no staged `TODO.md`); the phase
|
||||
directory now lives in `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Phase 27 — Global Tuning Manager
|
||||
|
||||
**Source:** `TODO.md L3 — "Add a way to add 'global tuning' without having a chat to reply to. Also previous tunes should be editable."`
|
||||
**Story:** `.agent/user_stories/global-tuning.md`
|
||||
**Context:** Phase 15 added steering notes (owner instructions injected into every system prompt as the `<tuning>` section); phase 16 gated the whole `/api/steering` router behind `require_admin`. The current UI only lets the admin **create** a note by pressing "Tune" under a completed chat bubble, and the header "Tuning" panel lists notes newest-first with **delete-only** per note. There is no way to add a note without a chat, and no way to edit an existing one.
|
||||
|
||||
## Objective
|
||||
Give the admin a **Global Tuning** page (`/tuning.html`) where notes can be created, **edited**, listed, and deleted **without any chat conversation** — and expose it via an admin-only header button. Add a `PUT /api/steering/{note_id}` endpoint for updates. The existing chat-page "Tune" button and header panel keep working (create + delete) so nothing regresses.
|
||||
|
||||
## Dependencies
|
||||
- `15_steering_notes` (complete) — the `steering_notes` table, the `<tuning>` prompt section, the `/api/steering` router (list/create/delete), and the chat-page "Tune" button + header panel this phase augments.
|
||||
- `16_admin_auth` (complete) — `require_admin` + the signed-cookie auth; the tuning page is admin-only, following the same gate pattern as the Sources page.
|
||||
- `19_shared_header` (complete) — the shared header bar that gains the admin-only "Tuning" link.
|
||||
|
||||
## Tasks
|
||||
1. `01_steering_put_endpoint.md` — add `PUT /api/steering/{note_id}` (update a note; 404 unknown, 422 invalid) + `PATCH`-friendly schema in `app/schemas.py`
|
||||
2. `02_tuning_page_html_css.md` — create `frontend/tuning.html` (title, note list, create form, edit-in-place) + CSS
|
||||
3. `03_tuning_js_crud.md` — create `frontend/assets/tuning.js` with full CRUD (list, create, edit, cancel, delete) + live announcer
|
||||
4. `04_header_button_and_e2e.md` — add the admin-only "Tuning" header button → `/tuning.html`; E2E suite for the tuning page
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: `PUT /api/steering/{note_id}` (admin 200, anon 403, unknown 404, invalid 422) — unit + integration.
|
||||
- Coverage: **>90%** on `app/` (new endpoint + schema).
|
||||
- E2E: `tests/e2e/test_global_tuning.py` — the story gate, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin can open `/tuning.html` (via the header button) and create a note without any chat; the note appears in the list and steers future answers (verifiable via the chat page after a turn, or via the list).
|
||||
- [ ] Admin can **edit** an existing note inline (edit button → textarea pre-filled → Save → updated text); the change is reflected in the list and in the `<tuning>` prompt (integration test).
|
||||
- [ ] Delete still works; create-then-edit-then-delete round-trips cleanly.
|
||||
- [ ] Anonymous users get 403 on `/tuning.html`'s data and on `PUT /api/steering/…`; the header "Tuning" link is hidden for anonymous.
|
||||
- [ ] The chat-page "Tune" button and header panel are unchanged (create + delete still work).
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL ≥ pre-change number.
|
||||
- [ ] `uv run pytest tests/e2e/test_global_tuning.py -v --no-cov` green in isolation.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): tuning page — landmarks, labeled controls, contrast ≥4.5:1, focus-visible, ≥44px targets, centered column, no CDN.
|
||||
- [ ] `.agent/user_stories/global-tuning.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit staging only this phase's files; `.agent/phases/todo/27_global_tuning/` moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A10 untouched** — `PUT /api/steering/{note_id}` is a new stateless route under `/api`; the API stays stateless, admin-only via `require_admin` (the existing gate).
|
||||
- **A11 untouched** — vanilla HTML/CSS/JS, no CDN, no new packages, system font stack; the tuning page is a new page + new module script.
|
||||
- **No schema change / no migration** — the `steering_notes` table already stores `note`; `PUT` updates the existing `note` column (A13 untouched).
|
||||
- **A16 honoured** — one new story E2E suite + adapted regressions.
|
||||
- **A17 honoured** — one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Phase 28 — Git-Based Sources
|
||||
|
||||
**Source:** `TODO.md L5 — "We shouldn't be hard-coding Homelab and Deployments. Instead, a list of git links should be specified. import_docs should clone or pull to a dedicated repository location and then index all the specified repository code."`
|
||||
**Story:** `.agent/user_stories/git-sources.md`
|
||||
**Context:** Phase 11 / the README document the import workflow. `scripts/import_docs.py` currently defaults to the hardcoded `DEFAULT_SOURCES = [Path("~/Homelab"), Path("~/Deployments")]` (repeatable via `--source`). The importer (`app/rag/importer.py::import_sources`) already walks any list of local directories — it needs no change to the walking logic, only to receive the resolved local paths.
|
||||
|
||||
## Objective
|
||||
Replace the hardcoded `~/Homelab` + `~/Deployments` default with a **list of git repository URLs** (`BOR_GIT_SOURCES`). `import_docs` clones (first run) or pulls (subsequent runs) each repository into a dedicated local location (`BOR_SOURCES_DIR`, default `~/bor-sources`) and indexes the resulting directories. The existing `--source` flag still overrides for manual paths.
|
||||
|
||||
## Dependencies
|
||||
- `01_infrastructure` (complete) — the base app + importer.
|
||||
- `11_long_answers` and the README import workflow (complete) — the `import_sources` contract and the documented CLI.
|
||||
|
||||
## Tasks
|
||||
1. `01_env_var_and_settings.md` — add `BOR_GIT_SOURCES` (comma-separated git URLs) + `BOR_SOURCES_DIR` settings; update `.env.example` + README.
|
||||
2. `02_git_clone_pull_utility.md` — create `scripts/git_sync.py` with `clone_or_pull(url, dest)` (clone --depth 1 / pull --ff-only, auth via URL or SSH).
|
||||
3. `03_import_docs_script_refactor.md` — refactor `scripts/import_docs.py` to resolve git sources → local dirs, then pass to `import_sources`.
|
||||
4. `04_integration_test_and_docs.md` — integration test for the git flow (mocked `git`); README + `.env.example` finalised.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: `scripts/git_sync.py` `clone_or_pull` (clone when missing, pull when present, error propagation) — unit test with a mocked `subprocess`; `import_docs` resolves git URLs → local dirs (integration).
|
||||
- Coverage: **>90%** on `app/` (the importer is unchanged; the new logic is in `scripts/`, covered by its own tests).
|
||||
- E2E: none required (no `app/` or UI change) — but the smoke suite must stay green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With `BOR_GIT_SOURCES` set and `BOR_SOURCES_DIR` writable, `uv run python -m scripts.import_docs` clones (first run) or pulls (subsequent runs) each repo into `BOR_SOURCES_DIR/<name>/` and indexes them.
|
||||
- [ ] `--source <path>` still overrides to import an arbitrary local directory (unchanged behaviour).
|
||||
- [ ] Unreachable/invalid git URL → the script fails loudly (non-zero exit) naming the repo, without importing partial junk.
|
||||
- [ ] `.env.example` documents `BOR_GIT_SOURCES` + `BOR_SOURCES_DIR`; README import section rewritten.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL ≥ pre-change number.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `.agent/user_stories/git-sources.md` exists.
|
||||
- [ ] One `--no-gpg-sign` commit staging only this phase's files; `.agent/phases/todo/28_git_based_sources/` moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A13 untouched** — no DB migration; the importer's `documents`/`chunks` schema is unchanged (source/path are stored as before, keyed by the repo-relative path).
|
||||
- **A11 untouched** — no new Python packages; `git` CLI is assumed available (standard on homelab machines). The clone/pull is done via `subprocess` (stdlib).
|
||||
- **A9 untouched** — the A9 format filter / hidden-dir skip / exclusion list still apply to the cloned content (the importer's `iter_importable_files` is unchanged).
|
||||
- **No anchor revised** — this is an operator-workflow change (how sources get onto disk), not a product/anchor change. Recorded in §11 + `.env.example`, not in §2.
|
||||
- **A16 honoured** — unit + integration tests for the new logic; no story E2E required (no UI/API change).
|
||||
- **A17 honoured** — one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Phase 34 — One Navbar on Every Page
|
||||
|
||||
**Source:** `TODO.md` L3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
**Context:** The shared header (phase 19, `frontend/assets/header.js`) + the Tuning nav link (phase 29) already standardize nav + auth on chat / sources / tuning — but `document.html` still uses the separate `.doc-header` variant (back + title + actions, **no nav at all**), `login.html` misses the Tuning link, and two functional controls remain page-scoped: the Tuning steering toggle + panel (chat only, logic in `app.js`) and the Sync sources button (Sources only, logic in `sources.js`). Owner confirmation (2026-08-26): the bar must be identical on **all** pages — nav, Tuning toggle, Sync, New chat, and the auth pair all present everywhere; the locked A10 UI revision stays (admin-only controls hidden for anonymous, active for the admin on every tab).
|
||||
|
||||
## Objective
|
||||
Make the header bar **identical on all five pages** (chat, sources, document viewer, tuning, login): one shared markup block, one owner of all functional control behavior (`header.js`), the viewer's back link + title preserved in a second titlebar row, and the phase-12/19 height contract (64px desktop / 58px ≤640px) applied to the standard row on every page.
|
||||
|
||||
## Dependencies
|
||||
- `19_shared_header` (complete) — the `header.js` module, the nav/auth markup + ids, the cached-one-whoami contract, the ship-hidden/reveal-for-admin pattern.
|
||||
- `29_tuning_nav_link` (complete) — the admin-only `#nav-tuning` reveal pattern this phase completes on the remaining pages.
|
||||
- `15_steering_notes` + `27_global_tuning` (complete) — the steering toggle/panel logic being moved into the shared module; chat-page behavior must not change.
|
||||
- `32_admin_sync_button` (complete) — the sync button state machine + `GET/POST /api/sync` being moved into the shared module; Sources-page behavior (result line + error banner) must not change.
|
||||
- `13_document_back_navigation` (complete) — the `#doc-back` target-resolution behavior the viewer titlebar must preserve.
|
||||
- `16_admin_auth` (complete) — the whoami gate, the soft-gate pages, the sign-out binding.
|
||||
|
||||
## Tasks
|
||||
1. `01_steering_moves_to_module.md` — the steering toggle + panel logic moves from `app.js` into `header.js` (exported `refreshSteering()`); the chat per-bubble Tune form keeps working.
|
||||
2. `02_sync_and_chat_moves_to_module.md` — the sync state machine moves from `sources.js` into `header.js` (`bor:sync-status` event); one module-owned New chat binding; the sign-in `?next=` rewrite.
|
||||
3. `03_full_header_all_pages.md` — all five pages ship the identical header block; `#steering-panel` exists on every page; the viewer becomes standard row + titlebar row; login gains the full header.
|
||||
4. `04_viewer_titlebar_styles.md` — the two-row viewer header styles, the sync button's failed state on non-Sources pages, theme/contrast/focus preserved.
|
||||
5. `05_e2e_and_contract_update.md` — the story E2E suite `test_nav_consistency.py`; `test_header_consistency.py` + `test_shared_header.py` updated to the new viewer contract; regression pass; commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: frontend-only — no new `app/` logic; the no-CDN integration test (`tests/integration/test_api.py::test_index_html_served_locally`) must still pass (all new markup is same-origin, no new tags).
|
||||
- Coverage: **>90%** on `app/` — unchanged by this phase (no Python change).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_nav_consistency.py` — the story gate, run in isolation; plus the two contract suites updated in task 05 and the regression list below.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The same visible header controls appear on **all five pages** in the same order — brand, nav [Chat, Sources, Tuning], Tuning toggle, Sync sources (admin), New chat, exactly one of Sign in / Sign out — verified in `test_nav_consistency.py` for both the admin and the anonymous role.
|
||||
- [ ] The document viewer shows the standard bar (row 1) + back link and title (row 2); `#doc-back` target resolution (phase 13) unchanged.
|
||||
- [ ] The login page carries the full header (nav incl. Tuning, Tuning toggle, Sync, New chat, auth pair).
|
||||
- [ ] Chat page: the steering panel + per-bubble Tune + inline form behave exactly as before; Sources page: the sync button state machine + `#sync-result` line + `#sync-error-banner` behave exactly as before.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged (>90%).
|
||||
- [ ] Regressions green in isolation: `test_header_consistency.py`, `test_shared_header.py`, `test_document_back_navigation.py`, `test_document_viewer.py`, `test_steering.py`, `test_global_tuning.py`, `test_sync_button.py`, `test_tuning_nav_link.py`, `test_smoke.py`, `test_chat_rag.py`, `test_admin_auth.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean (no Python change, but run the gate).
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): landmarks / labels / contrast ≥4.5:1 / focus-visible preserved; no CDN (rule 6).
|
||||
- [ ] One `--no-gpg-sign` commit staging only this phase's files; `.agent/phases/todo/34_consistent_navbar/` moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A10 UI revision preserved** — admin-only controls (Sources / Tuning nav links, Sync button) ship hidden and are revealed only for the signed-in admin; anonymous visitors get the reduced bar, identically on every page (owner confirmation 2026-08-26 — "hidden for anon, visible for admin").
|
||||
- **A11 untouched** — vanilla HTML/CSS/JS, no CDN, no new packages.
|
||||
- **Phase 19 module contract extended, not replaced** — `header.js` keeps the cached one-whoami-per-page promise; it gains ownership of the controls' behavior, not a second whoami.
|
||||
- **Viewer bar superseded** — the phase-19 single-row viewer bar (PLAN.md §7.1 "the viewer bar = back + title + the same actions") is replaced by the two-row layout at the owner's request (this TODO). `PLAN.md` is not edited (Protocol B); this phase directory records the revision.
|
||||
- **A16 / A17 honoured** — one new story E2E suite + one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 01 — Steering toggle + panel move into header.js
|
||||
|
||||
**Phase:** `34_consistent_navbar` · **Source:** `TODO.md:3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."`
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
|
||||
## Objective
|
||||
Make `frontend/assets/header.js` the owner of the steering toggle + panel behavior (today in `frontend/assets/app.js`), so the toggle can sit in every page's header (task 03) with zero page-script duplication. The chat page's behavior — panel open/close, list, count badge, per-note delete, per-bubble Tune form — must be byte-for-byte the same from the user's perspective.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/header.js` — add the steering logic (runs at module import, like the existing sign-out binding):
|
||||
- `loadSteering()` — `GET /api/steering`; non-2xx (the anonymous 403, unreachable API) → empty list (the current chat-page anonymous state); render via `renderSteeringPanel(notes)`.
|
||||
- `renderSteeringPanel(notes)` — newest-first `<li class="steering-note">` rows: the note as `textContent` in a `span.steering-note-text` (XSS contract unchanged — never innerHTML for the note), a per-note Remove `button.steering-delete` with `aria-label="Delete tuning note: …"`; toggle `#steering-empty`'s `hidden` on `notes.length`; set the `#steering-count` badge text.
|
||||
- `deleteSteeringNote(id, btn)` — disable the row button, `DELETE /api/steering/{id}`, re-load the list, announce through `#steering-announcer` (`role="status"`).
|
||||
- The `#steering-toggle` click binding — open/close `#steering-panel`, flip `aria-expanded`, move focus into the panel on open (the chat-page a11y contract; read `app.js`'s current implementation first and mirror it exactly, including any close-on-Esc / outside-click behavior it has).
|
||||
- **Export `refreshSteering()`** (fetch + render) — task 01's `app.js` change wires the per-bubble Tune form's success path to it.
|
||||
- Update the file's header comment (it now owns the steering controls).
|
||||
2. `frontend/assets/app.js` — remove the steering **panel** section (the `#steering-toggle` / `#steering-count` / `#steering-panel` / `#steering-list` / `#steering-empty` / `#steering-announcer` refs, `loadSteering`, `renderSteeringPanel`, `deleteSteeringNote`, `announceSteering`, the toggle binding) — **keep** the per-bubble `appendTuneButton` + `openTuneForm` (a chat-specific feature): the inline form's success path calls `refreshSteering()` imported from `./header.js` instead of the removed `loadSteering()`. Keep `TUNE_ICON` and the form's fetch/error handling untouched.
|
||||
3. Update the comments that describe the panel as chat-page-owned (app.js header comment, index.html steering comments) — the panel now belongs to the shared module; index.html's markup stays for now (task 03 copies it to the other pages).
|
||||
|
||||
Notes:
|
||||
- All elements are looked up null-safe (`querySelector` + guard) — a page that (still) lacks the panel markup is a no-op, mirroring how `initSharedHeader()` already works. This keeps the app functional between tasks.
|
||||
- Do not change the steering API (`app/api/steering.py`), the panel markup in `index.html`, or the `#steering-panel` styles.
|
||||
|
||||
## Testing & Quality
|
||||
- No Python change; the no-CDN integration test is unaffected.
|
||||
- Coverage: `app/` gate unaffected (no Python change).
|
||||
- The moved logic is behavior-verified by the regression suites in task 05 (`test_steering.py`, `test_global_tuning.py`); until then `uv run pytest` (unit + integration) must stay green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `header.js` exports `refreshSteering()` and owns the toggle binding, panel render, per-note delete, count badge, and announcer.
|
||||
- [ ] `app.js` no longer contains the panel logic; the per-bubble Tune button + inline form remain and call `refreshSteering()` on save.
|
||||
- [ ] The chat page (`/`) still loads, opens, lists, and deletes steering notes exactly as before (manual smoke via the dev server or the regression suites in task 05).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 02 — Sync state machine + New chat + sign-in next move into header.js
|
||||
|
||||
**Phase:** `34_consistent_navbar` · **Source:** `TODO.md:3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."`
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
|
||||
## Objective
|
||||
Make `header.js` the owner of the Sync button state machine (today in `sources.js`), the single New chat binding (today duplicated across `app.js` / `sources.js` / `tuning.js` / `document.js`), and the sign-in `?next=` derivation — so the same markup on any page (task 03) behaves identically.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/header.js` — add the sync state machine (read `sources.js`'s sync section first and mirror its contract exactly):
|
||||
- **Boot (admin only):** `await fetchIsAdmin()` on the cached whoami — non-admins never poll (the status endpoint is admin-only). One `GET /api/sync/status`: `running` → enter the running state + start polling (the phase-32 reload-mid-sync re-attach); terminal → render the last result.
|
||||
- **Click `#sync-btn`:** `POST /api/sync` → 202 enters running; 409 attaches to the running state (one sync at a time).
|
||||
- **Poll** `GET /api/sync/status` every 2000 ms — one live timer, stopped on a terminal state. **No client-side hard timeout** (phase-32 locked decision — a sync can outlive the page; the state machine simply keeps polling).
|
||||
- **Button states (§7.4 never-stale):** idle → label "Sync"; running → `disabled` + `aria-busy="true"` + spinner class (`.sync-icon.is-spinning`) + label "Syncing…"; success → label "Synced HH:MM"; failed → error state with the sanitized error string in the button's `title` + `aria-label` (on non-Sources pages that is where the failure is visible — the Sources page's own banner is driven by the event below).
|
||||
- **On every state change** dispatch `window.dispatchEvent(new CustomEvent("bor:sync-status", { detail: <status> }))` where `detail` is the `GET /api/sync/status` object — step 2 points the Sources page's banner/result line at it.
|
||||
2. `frontend/assets/header.js` — **one** New chat binding (module scope, null-safe): if `#messages` exists (chat page) → `window.dispatchEvent(new CustomEvent("bor:new-chat"))` and let the page script act; otherwise `clearChatStorage()` + `location.href = "/"` (the existing non-chat behavior — "new chat" means go to the chat, fresh).
|
||||
3. `frontend/assets/header.js` — **sign-in `?next=` rewrite:** in `initSharedHeader()` (or the module-scope boot), set `#sign-in-link`'s `href` to `/login.html?next=<current pathname>` (default `/`) — the admin lands back on the page they signed in from.
|
||||
- ASSUMPTION: on the chat page this changes the static fallback `?next=/sources.html` to `/` at runtime — landing on the page you signed in from ("return to where you were"). The page markup keeps its current href as the no-JS fallback.
|
||||
4. `frontend/assets/sources.js` — remove the sync state machine (the `#sync-btn` click handler, the 2 s poll loop, the button-state helpers, the boot re-attach). **Keep** `#sync-result` + `#sync-error-banner` rendering, now driven by a `window.addEventListener("bor:sync-status", …)` subscription: `running` → clear the result line, hide the banner; `success` → render the last-result counts in `#sync-result` (reuse the existing formatting, "added" always shown); `failed` → show `#sync-error-banner` with the error text; `idle` → hide the banner, clear the result.
|
||||
5. `frontend/assets/app.js` — replace the direct `#new-chat-btn` click binding with `window.addEventListener("bor:new-chat", startNewChat)` (the `startNewChat` function itself is unchanged).
|
||||
6. `frontend/assets/sources.js`, `frontend/assets/tuning.js`, `frontend/assets/document.js` — remove their `#new-chat-btn` click bindings (the module owns them). Update the file-header comments (document.js: the module now owns New chat; sources.js: sync is module-owned, the banner is event-driven).
|
||||
|
||||
Notes:
|
||||
- Null-safe element lookups throughout (a page that doesn't (yet) have `#sync-btn` is a no-op — the app stays functional between tasks).
|
||||
- The module must keep exactly **one** whoami per page load (the cached promise) — the sync boot may await it but must not add a fetch.
|
||||
- Do not touch `app/api/sync.py` — the API contract is unchanged.
|
||||
|
||||
## Testing & Quality
|
||||
- No Python change; the no-CDN integration test is unaffected.
|
||||
- Coverage: `app/` gate unaffected.
|
||||
- Behavior parity is verified by the regression suites in task 05 (`test_sync_button.py`, `test_shared_header.py`, `test_chat_rag.py`); until then `uv run pytest` must stay green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `header.js` owns: the sync state machine (+ `bor:sync-status` event), the single New chat binding (`bor:new-chat` on chat, clear+navigate elsewhere), and the sign-in `next` rewrite.
|
||||
- [ ] `sources.js` no longer contains the sync state machine — `#sync-result` / `#sync-error-banner` render off the event; no `#new-chat-btn` binding remains in any page script.
|
||||
- [ ] On the Sources page the full phase-32 cycle (click → polling → success counts / failure banner, reload re-attach) still works — confirmed in task 05 via `test_sync_button.py`.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Task 03 — The identical full header on all five pages
|
||||
|
||||
**Phase:** `34_consistent_navbar` · **Source:** `TODO.md:3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."`
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
|
||||
## Objective
|
||||
Ship the **identical header block** on all five pages — brand, nav [Chat, Sources, Tuning], Tuning toggle, Sync sources, New chat, Sign in / Sign out — and the `#steering-panel` section on every page; the document viewer keeps back + title in a second titlebar row; the login page finally carries the full header.
|
||||
|
||||
## Work
|
||||
The canonical block is `index.html`'s current header **plus** the `#sync-btn` copied verbatim from `sources.html` (hidden by default, `#sync-label` + `.sync-icon` inside). Place the Sync button **after** the Tuning toggle and **before** the New chat button on every page.
|
||||
|
||||
1. `frontend/index.html` (chat) — add the `#sync-btn` block to the header (the only missing control); everything else already ships. `#steering-panel` stays where it is (after `#kb-banner` in `<main>`).
|
||||
2. `frontend/sources.html` — add the Tuning toggle block (copied from `index.html`: `#steering-toggle` + `#steering-count`) after the nav; add the `#steering-panel` section (copied from `index.html`, incl. the `#steering-announcer` paragraph) as the **first child of `<main>`**; keep the existing `#sync-btn` where it is.
|
||||
3. `frontend/tuning.html` — add the Tuning toggle block + the `#sync-btn` block to the header (same order as chat); add the `#steering-panel` section as the first child of `<main>`.
|
||||
4. `frontend/document.html` — restructure the header:
|
||||
- **Row 1** becomes the standard `.app-header` / `.header-inner` bar, byte-for-byte the same block as the other pages: brand, `<nav class="app-nav">` with `Chat` + `#nav-sources` (hidden) + `#nav-tuning` (hidden), Tuning toggle, `#sync-btn` (hidden), New chat, Sign in (`?next=/document.html` static fallback) / Sign out.
|
||||
- **Row 2** — a new `.doc-titlebar` container inside the same `<header>`, carrying the existing `#doc-back` link + `#doc-title` + `#doc-meta` (moved out of the old `.doc-header-inner` title block, markup otherwise unchanged — `renderDocument` addresses them by id, so `document.js` needs no render change).
|
||||
- ASSUMPTION: **no nav link gets `is-active` / `aria-current` on the viewer** — a document is a detail view reachable from chat or Sources (phase 13's `back` param), so no single nav target is "current". The back link carries the navigation affordance.
|
||||
- The old `.doc-header-actions` wrapper is dropped — its buttons now live in row 1's standard `.header-inner`.
|
||||
5. `frontend/login.html` — full header: the nav gains the `#nav-tuning` link (after `#nav-sources`, same hidden-by-default markup as the other pages); add the Tuning toggle + `#sync-btn` + New chat + the Sign in / Sign out pair (sign-in static fallback `?next=/login.html`); add the `#steering-panel` section as the first child of `<main>`.
|
||||
6. `frontend/assets/header.js` — comment updates only: the viewer now has a nav (its "the viewer has no nav" notes are stale); the module's "missing element is a no-op" contract still holds for any page missing an element. No behavior change — the reveal code already handles `#nav-sources` / `#nav-tuning` / `#sync-btn` / the auth pair wherever they exist.
|
||||
7. Update the stale HTML comments in the touched headers (phase-19/29 comments describing the old page-specific layouts) to reference this phase + the owner confirmation (2026-08-26).
|
||||
|
||||
Rules:
|
||||
- Keep every existing id / class / aria attribute exactly as it exists today (the E2E suites key off them); only ADD missing blocks and move the viewer's title elements.
|
||||
- Ship-hidden stays ship-hidden: `#nav-sources`, `#nav-tuning`, `#sync-btn`, and exactly one of the auth pair are `hidden` in the markup on every page — `header.js` reveals at load (one whoami, cached).
|
||||
- Preserve indentation/markup style so the five headers stay diff-identical (that identity is what task 05's E2E asserts).
|
||||
|
||||
## Testing & Quality
|
||||
- No Python change; the no-CDN integration test must still pass (same-origin markup only).
|
||||
- Coverage: `app/` gate unaffected.
|
||||
- Manual smoke before task 05: with the dev server, as admin and as anonymous, each of the five pages shows the full bar (admin) / reduced bar (anonymous) with no console errors.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All five page headers contain the same control inventory in the same order: brand, nav [Chat, Sources, Tuning], `#steering-toggle`, `#sync-btn`, `#new-chat-btn`, `#sign-in-link` + `#sign-out-btn`.
|
||||
- [ ] `#steering-panel` (+ `#steering-announcer`) exists on all five pages (chat: after `#kb-banner`; others: first child of `<main>`).
|
||||
- [ ] `document.html` = standard row + `.doc-titlebar` row with `#doc-back` / `#doc-title` / `#doc-meta`; no nav link carries `is-active` there.
|
||||
- [ ] `login.html` carries the full header incl. the `#nav-tuning` link and the auth pair.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Task 04 — Two-row viewer header styles + sync failed state
|
||||
|
||||
**Phase:** `34_consistent_navbar` · **Source:** `TODO.md:3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."`
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
|
||||
## Objective
|
||||
Style the document viewer's new two-row header (standard row + titlebar row) so row 1 is visually indistinguishable from the other pages' bars, and give the Sync button a visible failed state on pages that have no error banner (every page except Sources).
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/styles.css` — **viewer header:**
|
||||
- The viewer's `<header>` keeps the sticky app-frame behavior; **row 1** reuses the existing `.app-header` / `.header-inner` rules verbatim (64px desktop, 58px at ≤640px — the phase-12/19 pinned heights apply because row 1 *is* the standard bar).
|
||||
- New `.doc-titlebar` rules for row 2: `.container`-width inner row with `#doc-back` + the title block; its own height (title line + meta line), `border-top` separator in the existing hairline color, same surface color (`#121a2e`/`#0a0e17` family per the phase-08 palette); `#doc-title` truncates with an ellipsis + `title` attribute instead of the old pill-clipping rule.
|
||||
- The old `.doc-header` / `.doc-header-inner` / `.doc-header-actions` / title-clipping rules are removed or reduced to the two-row structure (keep class names that task 05's updated contract suites reference — check which selectors the suites use before deleting: `.doc-header` may remain as the header element's class wrapping both rows).
|
||||
- The `.steering-panel` positioning rules must work from the new placement on the non-chat pages (first child of `<main>`) — the panel is an in-flow section, so this is expected to be a no-op; verify visually and in the E2E.
|
||||
2. `frontend/assets/styles.css` — **sync failed state (non-Sources pages):** the failed `#sync-btn` gets an error treatment from the phase-08 palette (error ink `#fca5a5` on the error surface `#2d1318`, border `#f59e0b`-free — the error chip uses `#fca5a5`/`#2d1318`, ≈9.1:1) so a failed sync is visible on every page, complementing the `title`/`aria-label` error text set by `header.js`. Reuse the existing `.sync-btn` state classes/styles if phase 32 already defines a failed look; otherwise add it.
|
||||
3. `frontend/assets/styles.css` — **login page:** the full header needs no new rules (it reuses `.app-header`), but confirm the login card layout still centers correctly with the full bar (no header-height regression at ≤640px).
|
||||
4. Accessibility checks (AGENTS.md rule 5): `:focus-visible` 3px outline on the new titlebar back link (it already has the existing `.doc-back` styles — preserve); contrast ≥4.5:1 for title/meta text; `prefers-reduced-motion` still stills the sync spinner.
|
||||
|
||||
## Testing & Quality
|
||||
- CSS-only — no Python change; the no-CDN integration test is unaffected.
|
||||
- Coverage: `app/` gate unaffected.
|
||||
- Visual pass (dev server, admin + anonymous, desktop + 640px): all five pages, viewer row 1 identical to chat's bar; viewer row 2 shows back + title + meta; the sync button's failed state is visible and readable.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The viewer's row 1 renders pixel-consistent with the chat/sources bars (same height 64px / 58px, same paddings, same controls).
|
||||
- [ ] `.doc-titlebar` renders back + title (ellipsis) + meta badges on one or two tidy lines; sticky with the header.
|
||||
- [ ] A failed sync shows an error-colored `#sync-btn` on non-Sources pages with the sanitized error in `title` / `aria-label`.
|
||||
- [ ] No console layout breakage on the login page (card still centered, 58px bar at ≤640px).
|
||||
- [ ] `uv run pytest` green (no-CDN test included); `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 05 — Story E2E + contract-suite updates + regression pass
|
||||
|
||||
**Phase:** `34_consistent_navbar` · **Source:** `TODO.md:3 — "I want the navbar to be consistent between every page. I don't want buttons to pop in and out of existance. Just keep all those buttons active across all tabs."`
|
||||
**Story:** `.agent/user_stories/nav-consistency.md`
|
||||
|
||||
## Objective
|
||||
Prove the contract with the story's dedicated Playwright suite — identical visible header control inventory on all five pages for each role — update the two pre-existing contract suites that encoded the old viewer bar, and run the full regression list.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_nav_consistency.py` (NEW — the story gate, run in isolation). Fixtures: the standard E2E app + DB (see `tests/e2e/conftest.py`); a fixture document for the viewer URL (the `source=docs&path=homelab%2Fkubernetes.md` pattern from `test_shared_header.py`); admin session via `tests/e2e/auth_helpers.py`.
|
||||
- **Admin inventory (all five pages):** `/`, `/sources.html`, `/document.html?source=…&path=…`, `/tuning.html`, `/login.html` — each header contains, visible: the three nav links (Chat, `#nav-sources`, `#nav-tuning`), `#steering-toggle`, `#sync-btn`, `#new-chat-btn`, and `#sign-out-btn` visible with `#sign-in-link` hidden. Assert the same **id + class inventory and DOM order** of the header controls on every page (normalize: the current-page `is-active` nav marker and the sign-in `?next=` value legitimately differ per page).
|
||||
- **Anonymous inventory (all five pages):** nav present with Chat visible and `#nav-sources` / `#nav-tuning` hidden (locked A10 UI revision); `#sync-btn` hidden; `#sign-in-link` visible, `#sign-out-btn` hidden; `#steering-toggle` visible.
|
||||
- **Viewer specifics:** row 1 height equals the chat page's header height (64px desktop / 58px ≤640px); the titlebar row is visible with `#doc-back` + `#doc-title` (rendered document title) + `#doc-meta` badges; clicking `#doc-back` honors the `back` param (phase 13 — one positive + one rejection case).
|
||||
- **Steering works off-chat:** as admin, on `/tuning.html` — seed zero notes (truncate `steering_notes` via a `SessionLocal` like the other suites), click `#steering-toggle` → `#steering-panel` visible + `aria-expanded="true"` + empty state shown; add a note through the panel? (the panel has no add form — it lists notes; assert the toggle open/close cycle + the count badge reads 0) — keep this deterministic, no chat needed.
|
||||
- **Sync present, not triggered:** as admin on `/tuning.html` assert `#sync-btn` is visible (do NOT click it — a real sync would clone real repos; the full state machine is `test_sync_button.py`'s job).
|
||||
2. `tests/e2e/test_header_consistency.py` (UPDATE to the new contract): the viewer assertions change — `.doc-header` is now the two-row header; assert **row 1** (the standard bar) is 64px desktop / 58px ≤640px and identical to the chat/sources bars (the existing `_box_height(page, ".doc-header")` measurement must be pointed at the standard row — use the row-1 selector, e.g. `.app-header .header-inner` inside the viewer header), and assert the titlebar row is present (height > 0). The chat/sources assertions are unchanged.
|
||||
3. `tests/e2e/test_shared_header.py` (UPDATE to the new contract): the "the viewer has no nav — no Sources link in the DOM" assertions flip — the viewer now carries the same nav contract (`.app-nav` with Chat + hidden `#nav-sources` + hidden `#nav-tuning`, revealed for admin). The auth-pair + New chat assertions for the viewer stay (they move from `.doc-header-actions` to the standard bar — update the selectors).
|
||||
4. **Regression pass — each in isolation** (`uv run pytest tests/e2e/<file>.py -v --no-cov`): `test_header_consistency.py`, `test_shared_header.py`, `test_document_back_navigation.py`, `test_document_viewer.py`, `test_steering.py`, `test_global_tuning.py`, `test_sync_button.py`, `test_tuning_nav_link.py`, `test_smoke.py`, `test_chat_rag.py`, `test_admin_auth.py`. Fix fallout in the suites above where the old contract is encoded; fix app code where behavior genuinely changed.
|
||||
5. Full gate: `uv run pytest` (unit + integration), `uv run pytest --cov=app --cov-report=term-missing` (TOTAL unchanged, >90%), `uv run ruff check . && uv run pyright`.
|
||||
6. **UI Structure Check** (AGENTS.md rule 5) on the five headers + the new titlebar: landmarks (`<header>`, `<nav aria-label>`, `<main>`), labels, contrast ≥4.5:1, focus-visible, no CDN (rule 6 — the no-CDN integration test covers it).
|
||||
7. **Commit** (A17): stage only this phase's files (`frontend/**`, `tests/e2e/**`), message `feat(ui): one consistent navbar on every page (TODO.md L3)`, always `--no-gpg-sign`. Move `.agent/phases/todo/34_consistent_navbar/` to `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `tests/e2e/test_nav_consistency.py` green **in isolation** — the story gate (A16: one story, one file).
|
||||
- Unit/integration: no new `app/` logic — the existing suite (incl. the no-CDN test) stays green.
|
||||
- Coverage: **>90%** on `app/` — unchanged (no Python change).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_nav_consistency.py -v --no-cov` green in isolation.
|
||||
- [ ] `test_header_consistency.py` + `test_shared_header.py` updated and green; every suite in the task 05 regression list green in isolation.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged (>90%).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Phase 37 — Agent document tools (list + read)
|
||||
|
||||
**Source:** `TODO.md` L3–L9 — "Still having trouble referencing specific documents. The agent should be able to list the available sources as a tool and the read the ones it thinks are relevant… it will need a basic agent loop. I'm thinking it gets one opportunity to list documents and then one opportunity to add exactly one extra document to its context before being required to answer. These values should be configured by environment variables." (L3; L5–L9 is the failure example: `aws-route53.md` references `example-record-file.json` whose contents are not in context, so the model refuses to guess)
|
||||
**Story:** `.agent/user_stories/agent-document-tools.md`
|
||||
**Context:** PLAN §3/§4 (chat flow + SSE contract), §6 (locked persona prompt), §9 (per-turn log line); `app/api/chat.py` (`plan_turn` + stream), `app/rag/llm.py` (`StreamPiece`, `chat_stream`), `app/rag/prompts.py`, `app/schemas.py`, `frontend/assets/app.js` (chat feedback state machine + thinking block), `tests/e2e/mock_llm.py`.
|
||||
|
||||
## Objective
|
||||
Give the chat model two server-side tools on **grounded** turns — `list_documents` (list the indexed sources) and `read_document(source, path)` (add exactly one more indexed document, full text) — with the opportunity counts tunable by env vars (`BOR_AGENT_LIST_CALLS`, `BOR_AGENT_READ_CALLS`, default 1 each); once both budgets are spent the tools are dropped and the model must answer. The UI shows a "calling tool" state in addition to "thinking".
|
||||
|
||||
## Dependencies
|
||||
- `03_story_chat_rag` (complete) — the A7/A8/A15 pipeline this phase extends.
|
||||
- `14_chat_persistence` (complete) — the saved chat record shape this phase extends with `tools`.
|
||||
- `17_thinking_display` (complete) — the SSE `thinking` event + `StreamPiece` kinds + the UI scratchpad the `tool` event sits beside; the `BOR_STREAM_THINKING` kill-switch pattern.
|
||||
- `24_whole_document_context` (complete) — the whole-document context contract (`read_document` never truncates).
|
||||
- `31_kb_overview_prompt` (complete) — HIGH-prompt section order the `<tools>` instructions join.
|
||||
- No dependency on 34/35/36 (frontend touch surface is `app.js`/`styles.css` only).
|
||||
|
||||
## Tasks
|
||||
1. `01_probe_tool_calling.md` — extend `scripts/llm_probe.py` with a live `--tools` probe against `turbo`; record the verdict (tool-calling vs documented prompt-based fallback).
|
||||
2. `02_llm_client_tools.md` — `app/rag/llm.py`: `chat_stream(messages, tools=None)` accumulates `tool_calls` deltas into `ToolCallPiece`; `tools=None` stays byte-identical.
|
||||
3. `03_agent_loop.md` — `app/rag/agent.py`: `run_agent` loop with env-tuned budgets, DB accessors, the `<tools>` prompt section, `app/config.py` settings.
|
||||
4. `04_api_sse_tool_event.md` — `app/api/chat.py` + `app/schemas.py`: SSE `tool` events, agent on grounded turns, `done.sources`/`query_log` include the read doc, `tool_calls=N` log field, PLAN §4/§9 revision notes.
|
||||
5. `05_frontend_tool_states.md` — `app.js`/`styles.css`: "calling tool" state + `.tool-call` lines + persistence; UI Structure Check.
|
||||
6. `06_e2e_docs_commit.md` — mock-LLM tool behavior, the story E2E, README + `.env.example`, commit, move the phase dir.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `app/rag/agent.py` loop mechanics (budgets, forced answer, dedupe, unknown tool, missing doc, round cap) with a scripted fake LLM + monkeypatched DB accessors; `llm.py` tool-call delta accumulation; config defaults.
|
||||
- Integration: the agent's DB accessors against real Postgres; the `/api/chat` SSE contract with `tool` events (mock LLM); no schema change in this phase.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_agent_document_tools.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run python -m scripts.llm_probe --tools` runs against the live endpoint; the verdict is recorded in the `app/rag/agent.py` docstring + the commit message.
|
||||
- [ ] `POST /api/chat` streams `{"type":"tool","name":"list_documents"}` and `{"type":"tool","name":"read_document","argument":"<source/path>"}` frames for a tool-using model; the non-tool path (deflected, or no tool call) is byte-identical to today's SSE.
|
||||
- [ ] `BOR_AGENT_LIST_CALLS=0 BOR_AGENT_READ_CALLS=0` reproduces pre-phase behavior (no `tools` in the LLM request, no `tool` events).
|
||||
- [ ] `done.sources` + `query_log.sources` include the read document (deduped); the per-turn log line carries `tool_calls=N`.
|
||||
- [ ] The UI shows the "calling tool" label + tool lines while tools run; tool lines re-render after a reload; WCAG basics hold.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_agent_document_tools.py -v --no-cov` green in isolation; existing chat E2E suites green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5) + no CDN (rule 6).
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A7/A8 honoured** — the retrieval gate is untouched; `read_document` appends the **full** document text (no truncation, A7-revised contract); the deflection path is byte-identical.
|
||||
- **A15 extended (revision note, owner permission 2026-08-26)** — the SSE contract gains `{"type":"tool","name":…,"argument":…}`; `delta`/`done` shapes are unchanged.
|
||||
- **A5 honoured** — same aipi endpoint/models; tools are plain OpenAI `tools`/`tool_calls`. If the task-01 probe shows `turbo` lacks tool-calling, the phase falls back to the documented prompt-based structured call (task 03) — recorded in the `agent.py` docstring, never silent.
|
||||
- **No schema change** — the tools query existing tables; no migration in this phase.
|
||||
- **Budgets-as-kill-switch** — both budgets at 0 disables the tools entirely (request byte-identical to pre-phase); no separate kill-switch env var.
|
||||
- **A16/A17 honoured** — one new story E2E suite + one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 06 — Mock-LLM tool behavior + story E2E + docs + commit
|
||||
|
||||
**Phase:** `37_agent_document_tools` · **Source:** `TODO.md:3–9 — "The agent should be able to list the available sources as a tool and the read the ones it thinks are relevant"` (L5–L9 failure example: `aws-route53.md` references `example-record-file.json` for the exact JSON shape but the file's contents are not in context — "I don't want to invent it!")
|
||||
**Story:** `.agent/user_stories/agent-document-tools.md`
|
||||
|
||||
## Objective
|
||||
A deterministic mock tool-call behavior, the story's isolated Playwright suite, the docs, and the phase commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/mock_llm.py` — new marker behavior (follow the existing marker-driven conventions documented in the module docstring): user message contains `use your tools` **and** the system prompt contains the `<tools>` section →
|
||||
- Request 1 (`tools` parameter present): stream only `tool_calls` deltas — call `list_documents` (synthetic id `call_0`, no arguments), `finish_reason: "tool_calls"`, no content.
|
||||
- Request 2 (messages contain a `tool`-role result carrying the catalog): parse the **first** catalog line (`source/path — title` → split on `" — "`, then `rsplit("/", 1)` for source/path) and stream a `tool_calls` delta calling `read_document` with it (id `call_1`).
|
||||
- Request 3 (no `tools` parameter): a content answer, deterministic: `Read <source/path>. <first 80 chars of the read document's tool-result content>` — so the test can assert the read document's content reached the model and landed in the answer.
|
||||
- All other requests behave exactly as today. `E2E_REAL_LLM=1` ignores the marker (the real model does what it does).
|
||||
2. KB fixture — seed a two-document pair that reproduces the TODO failure (follow the KB-seeding pattern of `tests/e2e/test_whole_document_context.py`, phase 24): `aws-route53.md` (references `example-record-file.json` "for the exact JSON shape of reeselink.json" but does not include it) + `example-record-file.json` (the JSON shape). The marker question must be high-relevance under the mock's 0.30 E2E threshold (genuine token overlap with the fixture docs).
|
||||
3. `tests/e2e/test_agent_document_tools.py` (the story gate — one story, one file, run in isolation):
|
||||
- The marker question → the SSE contains `tool` frames (list, then read); while a tool runs the UI shows the "calling tool" label (poll the button/label text) and the bubble shows both tool lines (`Listing documents`, `Reading <source>/example-record-file.json`); the final answer quotes the read document (the mock's deterministic quote); the source chips include the read document (a `done`-sources chip linking to the viewer).
|
||||
- Reload → the persisted record re-renders the tool lines (phase 14).
|
||||
- Plain grounded question (no marker) → **no** `tool` frames, the answer renders as today (regression inside the story file).
|
||||
- Deflected question (the mock's unrelated-question path) → no `tool` frames (the grounded-only scope).
|
||||
4. Docs: `.env.example` (`BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS` — defaults 1/1, "0 disables the tool") and README (chat-behavior section: the two tools, the budgets, the SSE `tool` frame, the "calling tool" UI state).
|
||||
5. Regression pass: `uv run pytest` (unit + integration), `uv run pytest --cov=app --cov-report=term-missing` (>90%), `uv run pytest tests/e2e/test_agent_document_tools.py -v --no-cov` (in isolation), plus the chat-adjacent E2E suites green: `test_chat_rag.py`, `test_thinking_display.py`, `test_honest_deflection.py`, `test_chat_persistence.py`.
|
||||
6. Commit — one atomic `--no-gpg-sign` Conventional Commits commit for the whole phase (AGENTS.md rule 8), e.g. `feat(rag): agent document tools — list/read tools with env-tuned budgets, SSE tool events + "calling tool" UI`; include the task-01 probe verdict in the commit message; move the phase directory to `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- The gates above are this task's quality bar (A16: one story, one isolated E2E file, coverage >90%).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The marker flow is deterministic across two consecutive isolated runs.
|
||||
- [ ] Step-5 suites all green; coverage >90%.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Phase 38 — Local directory sources
|
||||
|
||||
**Source:** `TODO.md` L11 — "Also need a way to import from existing directory if it's not a git repo"
|
||||
**Story:** `.agent/user_stories/local-directory-sources.md`
|
||||
**Context:** `35_git_sources_admin` (todo — runs first: the `git_sources` table, the admin API, the sources page, `effective_git_sources()`), `32_admin_sync_button` (the in-process sync pipeline), `28_git_based_sources` (the `import_docs` resolution order + `clone_or_pull`), `16_admin_auth` (the `require_admin` pattern).
|
||||
|
||||
## Objective
|
||||
Make an **existing local directory** (not a git repo) a first-class source: the admin adds it on the sources page (kind `local` + path), and the Sync button and `import_docs` import it alongside the git checkouts — with fail-loud validation at add-time and at sync-time.
|
||||
|
||||
## Dependencies
|
||||
- `35_git_sources_admin` (todo — runs before this phase) — the table/API/page this phase extends.
|
||||
- `32_admin_sync_button` (complete) — the in-process sync pipeline this phase extends.
|
||||
- `28_git_based_sources` (complete) — `import_docs` source-resolution order + `scripts/git_sync.clone_or_pull`.
|
||||
- `16_admin_auth` (complete) — the `require_admin` router pattern.
|
||||
|
||||
## Tasks
|
||||
1. `01_kind_column.md` — migration 0007: `kind` + `path` columns on `git_sources` (reversible) + model update.
|
||||
2. `02_api_local_kind.md` — the admin API accepts/returns the local kind (path validation → 422, per-kind 409); git contract unchanged.
|
||||
3. `03_sync_import_local.md` — the sync pipeline + `import_docs` resolve DB git + local rows together (local = direct walk; missing dir fails loudly before importing).
|
||||
4. `04_admin_page_local.md` — the page: a second add form for local directories + kind badges on list rows + updated sync hint.
|
||||
5. `05_e2e_docs_commit.md` — the story E2E (add / validate / sync-import / prune-remove), README, commit, move the phase dir.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the kind/path validation + the combined source resolution (git + local, origin logging, both-empty fail-loud).
|
||||
- Integration: migration 0007 up/down; the API local-kind contract (403/201/409/422); the sync pipeline with a temp local dir (the KB actually updated); the existing `test_git_sources_api.py` / `test_sync_api.py` / `test_import_docs_git.py` suites stay green through the indirection.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_local_directory_sources.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Migration 0007 applied; `git_sources` has `kind` (default `git`, check `git`|`local`) + `path` (unique, nullable); existing rows read as `kind='git'`.
|
||||
- [ ] `POST /api/git-sources` with `kind=local` + an existing directory → 201; a missing/relative path → 422 naming the path; a duplicate path → 409; anonymous → 403 on all routes (the phase-35 contract extended).
|
||||
- [ ] `POST /api/sync` with mixed git + local rows imports both in one run (prune over the union); a missing local dir → `failed` status with the path named (sanitized, phase-32 convention).
|
||||
- [ ] `import_docs` (no `--source`) resolves DB git + local rows; `--source` still wins; the env fallback stays git-only; both-empty fails loudly ("no sources configured").
|
||||
- [ ] The page: the Local add form + the Git/Local badges + the updated hint; anonymous still gets the sign-in gate.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_local_directory_sources.py -v --no-cov` green in isolation; the task-05 regression list green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5) + no CDN (rule 6).
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **The phase-35 table is extended, not duplicated (owner permission 2026-08-26)** — one `git_sources` table with a `kind` discriminator, one admin page, one Sync button; no `local_sources` table, no second page.
|
||||
- **A13 honoured** — the new columns land via a reversible Alembic migration.
|
||||
- **Phase-32 scope boundary kept** — adding a local dir does NOT import immediately; the Sync button performs the import (the page hint says so); removal prunes on the next sync (`prune=True` over the union).
|
||||
- **Local sources are DB-registered only** — no env var for local paths (the DB is the registry; `BOR_GIT_SOURCES` remains the git-only fallback while the table is empty).
|
||||
- **A10 honoured** — the API stays behind the phase-16 `require_admin` pattern; no new session state.
|
||||
- **A16/A17 honoured** — one new story E2E suite + one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 05 — Story E2E + docs + commit
|
||||
|
||||
**Phase:** `38_local_directory_sources` · **Source:** `TODO.md:11 — "Also need a way to import from existing directory if it's not a git repo"`
|
||||
**Story:** `.agent/user_stories/local-directory-sources.md`
|
||||
|
||||
## Objective
|
||||
The story's isolated Playwright suite (add → validate → sync-import → prune-remove), the README update, and the phase commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_local_directory_sources.py` (the story gate; the DB prerequisite per the conftest):
|
||||
- Fixture: a host temp dir (e.g. `tmp_path` — the app server runs on the same host, so the path is visible to it) containing one fixture `.md` with distinctive tokens.
|
||||
- As admin (the `auth_helpers.py` form-login): open the sources page → **Local directory** form → add the temp dir → the row appears with the Local badge; add a missing path (`/nonexistent/bor-e2e`) → the inline error names it, no row added; add the same temp dir again → the 409 duplicate error.
|
||||
- Sync: click the Sync button (the phase-32 pattern) → poll `/api/sync/status` until success → the fixture doc appears in `GET /api/docs`; delete the fixture file from the temp dir, sync again → the doc is pruned (union prune); then Remove the row on the page → the row disappears.
|
||||
- Anonymous: the page soft-gates and the API 403s (the phase-35 assertions, regression).
|
||||
- Keep the file self-contained (one story, one file, isolated run — A16).
|
||||
2. README: the "Sources" section — the two kinds (git = clone/pull mirror; local = direct import of an existing directory), add-time validation, union pruning; note that the DB is the local-source registry (no env var for local paths).
|
||||
3. `.env.example` — no new variable; extend phase 35's `BOR_GIT_SOURCES` note if needed ("env fallback is git-only — local directories are registered on the admin page").
|
||||
4. Regression pass: `uv run pytest` + the coverage gate (>90%) + the isolated story E2E + `tests/e2e/test_git_sources_admin.py` (phase 35's suite — its page assertions must survive the new form; if a selector collided, scope the test to the git form and note it in the commit message) + `tests/e2e/test_sync_button.py`.
|
||||
5. Commit — one atomic `--no-gpg-sign` Conventional Commits commit for the whole phase (AGENTS.md rule 8), e.g. `feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges`; move the phase directory to `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- The gates above are this task's quality bar (A16: one story, one isolated E2E file, coverage >90%).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The story E2E is green in isolation, deterministic across two consecutive runs.
|
||||
- [ ] The step-4 regression list green; coverage >90%.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Phase 40 — Tuning toggle anonymous flash
|
||||
|
||||
**Source:** `TODO.md` L3 — "Loading the page briefly shows the 'Tuning' button in the header even when the user isn't authenticated. Only show that if the user is authenticated."
|
||||
**Story:** `.agent/user_stories/tuning-toggle-flash.md`
|
||||
**Context:** Phase 15/34 shared header (`frontend/assets/header.js` owns `#steering-toggle` / `#steering-panel` on all six pages; anonymous → `remove()` post-whoami). The admin-only **nav links** already ship `hidden` (phase-19 contract) — the flashing control is the **steering toggle button labeled "Tuning"**, which ships visible in all six pages and is removed only after `/api/whoami` resolves.
|
||||
|
||||
## Objective
|
||||
Kill the anonymous flash: the tuning toggle ships `hidden` in every page's markup and is revealed only when whoami says admin (the exact ship-hidden / reveal-for-admin contract the nav links use), so an anonymous user never sees the "Tuning" button — not for a single frame.
|
||||
|
||||
## Dependencies
|
||||
- `39_configurable_brand` (complete; last existing phase) — current header state: full shared bar on all six pages.
|
||||
- `19_shared_header` / `16_admin_auth` / `34_consistent_navbar` (complete) — the `fetchIsAdmin()` gate, the ship-hidden nav contract, and the module-owned steering controls this task modifies.
|
||||
|
||||
## Tasks
|
||||
1. `01_toggle_ships_hidden.md` — add `hidden` to `#steering-toggle` in all six pages and reveal-for-admin in `header.js`; pin at source level.
|
||||
2. `02_flash_e2e_and_regression.md` — story E2E suite (never-visible-for-anonymous, admin reveal, nav-contract regression) + regression pass + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: new `tests/unit/test_steering_toggle_visibility.py` — `hidden` present on `#steering-toggle` in all six HTML pages; `header.js` unhides for admin (line before `refreshSteering()`) and the anonymous `remove()` path is intact; any existing source-pin test asserting the exact old markup is updated (check `tests/unit/test_shared_header.py`, `test_steering.py`).
|
||||
- Coverage: frontend-only — the `app/` >90% gate is unaffected (must stay unchanged).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_tuning_toggle_flash.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Anonymous load of every page: the toggle is never visible (MutationObserver records zero visible frames) and is absent from the DOM after load.
|
||||
- [ ] Admin load: toggle visible, panel opens, count badge correct — admin behavior unchanged.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged.
|
||||
- [ ] `uv run pytest tests/e2e/test_tuning_toggle_flash.py -v --no-cov` green in isolation.
|
||||
- [ ] Regression E2E suites green in isolation: `test_shared_header.py`, `test_global_tuning.py`, `test_steering.py`, `test_tuning_nav_link.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): no new focus targets; landmarks/contrast unchanged; no CDN.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved `.agent/phases/todo/` → `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A10 untouched** — no API change; the fix is pure UI visibility off the existing `/api/whoami` gate.
|
||||
- **A11 untouched** — no new assets, no CDN.
|
||||
- **Phase-16 contract preserved** — anonymous still gets "absent, not hidden" (remove-from-DOM); this phase only removes the pre-whoami flash window.
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Task 02 — Flash E2E + regression + commit
|
||||
|
||||
**Phase:** `40_tuning_toggle_flash` · **Source:** `TODO.md:3` — "Loading the page briefly shows the 'Tuning' button in the header even when the user isn't authenticated. Only show that if the user is authenticated."
|
||||
**Story:** `.agent/user_stories/tuning-toggle-flash.md`
|
||||
|
||||
## Objective
|
||||
Prove the flash is gone at the browser level (never visible, not even for a frame) and that the shared-header contract is intact; commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_tuning_toggle_flash.py` (new) — mock-only suite (DB up), per the story's Playwright Mapping Rule:
|
||||
- a helper `install_visibility_observer(page)`: `page.add_init_script` a MutationObserver on `document.documentElement` that appends to `window.__tuningVisibleFrames` every time `#steering-toggle` is added/attribute-changed and is both in the DOM **and** not `[hidden]` (check `el.offsetParent !== null` or `!el.hidden`);
|
||||
- `test_anonymous_never_sees_toggle` — load `/` anonymously, wait for network idle + header settle (whoami resolved), assert `window.__tuningVisibleFrames` is empty and `#steering-toggle` is absent from the DOM;
|
||||
- `test_anonymous_other_pages_never_flash` — same on `/sources.html`, `/tuning.html`, `/login.html`;
|
||||
- `test_admin_toggle_revealed_and_working` — `login()` (e2e.auth_helpers), reload `/`, toggle visible + clickable (opens `#steering-panel`, `aria-expanded="true"`), count badge matches the list;
|
||||
- `test_nav_contract_regression` — anonymous: `#nav-sources` / `#nav-git-sources` / `#nav-tuning` stay hidden; admin: revealed.
|
||||
2. Regression pass (isolation runs, per A16): `test_shared_header.py`, `test_global_tuning.py`, `test_steering.py`, `test_tuning_nav_link.py`, `test_smoke.py` — all green; fix only true regressions.
|
||||
3. `uv run pytest` (unit+integration) green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `fix(header): ship the tuning toggle hidden — no anonymous flash`, staging this phase's changed files; move `.agent/phases/todo/40_tuning_toggle_flash/` → `.agent/phases/complete/` (force-add per AGENTS.md rule 8 if the history tracks the tree).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_tuning_toggle_flash.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`).
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The story E2E file passes in isolation; the four regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Phase 41 — Sync fails fast + modal when a model is down
|
||||
|
||||
**Source:** `TODO.md` L4 — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
|
||||
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
|
||||
**Context:** `app/api/sync.py::_run_sync` (phase 32/35/38) runs source resolution → git clones → `import_sources` (embeds) → overview (lite) — with a dead LLM endpoint the run discovers it only mid-import, after slow clones. The sync state machine is module-owned by `frontend/assets/header.js` (`applySyncFailure` → button title/aria + `.is-error` + `bor:sync-status` event; the Sources page renders `#sync-error-banner`). No dialog component exists yet.
|
||||
|
||||
## Objective
|
||||
When `embed` or `lite` is unreachable, the sync fails **before any expensive work** with a message naming the model, and the failure is readable in a **modal dialog** on every page that carries `#sync-btn`.
|
||||
|
||||
## Dependencies
|
||||
- `40_tuning_toggle_flash` (todo) — current shared-header state (sequential; no code overlap, but both touch `header.js` — keep this phase's changes confined to the sync section).
|
||||
- `32_admin_sync_button` / `35_git_sources_admin` / `38_local_directory_sources` (complete) — the pipeline, the status contract, and the module-owned button lifecycle this phase extends.
|
||||
|
||||
## Tasks
|
||||
1. `01_model_probe_fail_fast.md` — `check_models()` probe in `app/rag/llm.py`, called first in `_run_sync`; unit + integration tests.
|
||||
2. `02_sync_error_modal.md` — `header.js` modal (built in JS, all pages) + CSS; source pins.
|
||||
3. `03_model_down_e2e_and_commit.md` — dedicated E2E suite (dead-LLM module app) + phase-32 regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: probe success/failure paths with a fake LLM client (embed-down, lite-down, both up); the sync task's fail-fast ordering (probe before source resolution — assert no clone call happens).
|
||||
- Integration: `POST /api/sync` with a stubbed failing client → `GET /api/sync/status` reaches `failed` with the model-naming error; healthy path regression.
|
||||
- Coverage: **>90%** on `app/` including the new probe code.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_sync_model_down.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With a dead LLM endpoint: sync fails within seconds, **before** any clone, error names the unavailable model; the modal shows it; button settles retry-ready.
|
||||
- [ ] Modal contract: `role="alertdialog"`, `aria-modal`, text via `textContent`, close via button / `Esc` / backdrop, focus in-and-out.
|
||||
- [ ] Healthy sync pipeline (clone → import → overview) unchanged — phase-32 suite green in isolation.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_sync_model_down.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A12 untouched** — still in-process, no queue, no new service; the probe is two cheap model calls.
|
||||
- **A10 untouched** — no new endpoint; `/api/sync` + `/api/sync/status` keep their shapes (a model failure is just another `failed` state).
|
||||
- **Phase-32 contract kept** — 2 s poll, 202/409, no client timeout, `bor:sync-status` event, button title/aria affordance, Sources banner (the modal is additive).
|
||||
- **Owner-locked (2026-08-27, roadmap A4):** probe runs **before** git clones; the modal is the primary failure surface on every page.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 01 — Model probe: fail fast before any clone
|
||||
|
||||
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
|
||||
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
|
||||
|
||||
## Objective
|
||||
The sync run verifies both models it needs (`embed` + the summary `lite`) **first** — before source resolution, before any `clone_or_pull` — and fails the run with a clear, model-naming error when either is unreachable.
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py` — add `class ModelUnavailableError(LLMError)` and:
|
||||
```python
|
||||
async def check_models(llm: LLMClient) -> None:
|
||||
"""Verify the models a sync needs (embed + summary) before any
|
||||
expensive work; raise ModelUnavailableError naming the model."""
|
||||
```
|
||||
- `await llm.embed_one("sync model check")` — wrap `EmbeddingError` (and any other exception) in `ModelUnavailableError`: message names the **embedding model** (use `llm.settings.llm_embed_model`, e.g. "The embedding model ('embed') is not available — check the model endpoint and retry.")
|
||||
- `await llm.chat([{"role": "user", "content": "ping"}])` (defaults to `llm_summary_model`) — wrap `LLMError`/other in `ModelUnavailableError` naming the **summary model** (`llm.settings.llm_summary_model`, e.g. "The summary model ('lite') is not available — check the model endpoint and retry.").
|
||||
- Docstring notes the probe is deliberately tiny (one short embedding + one 1-token-scale completion) and that the sync sanitizer downstream still masks any embedded credentials.
|
||||
2. `app/api/sync.py` — in `_run_sync()`, construct `llm = LLMClient()` **before** the DB/source block and call `await check_models(llm)` as the **first** pipeline step (before `effective_sources`, before the clone loop). Update the module docstring's pipeline list (the probe is step 1: "verify `embed` + summary model availability — fail fast before any clone") and renumber. `ModelUnavailableError` falls into the existing `except Exception` → `failed` state with the sanitized error (no special-casing needed — verify the message survives `_sanitize_error` unchanged).
|
||||
3. `tests/unit/test_sync_model_probe.py` (new) — with a fake LLM client (duck-typed `embed_one`/`chat`, see `tests/fakes.py` `FakeEmbedder` for the shape):
|
||||
- both up → `check_models` returns, both methods called;
|
||||
- embed raises → `ModelUnavailableError` mentioning the embed model name, `chat` never called;
|
||||
- chat raises → `ModelUnavailableError` mentioning the summary model name;
|
||||
- message content assertions (model name present, "not available" wording).
|
||||
4. `tests/integration/test_sync_api.py` — extend:
|
||||
- **fail-fast:** monkeypatch `app.api.sync.LLMClient` (or `check_models`) so the probe raises `ModelUnavailableError`; also monkeypatch `clone_or_pull` to *assert it is never called*; `POST /api/sync` → poll `GET /api/sync/status` until terminal → `state == "failed"`, `error` names the model;
|
||||
- **ordering:** a spy on `effective_sources` shows the probe ran before it;
|
||||
- **healthy regression:** the existing success/failure tests stay green (they stub the LLM — the stub must now satisfy the probe: `FakeEmbedder` already implements `chat`; if the existing stub lacks `embed_one`, add it — `FakeEmbedder.embed` exists, so subclass or delegate).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration as above; the probe must be covered (success + both failure modes) to keep `app/` **>90%**.
|
||||
- `uv run pytest tests/unit/test_sync_model_probe.py tests/integration/test_sync_api.py -v` green; full suite green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `check_models` exists, is called first in `_run_sync`, and names the failing model in `ModelUnavailableError`.
|
||||
- [ ] A dead-model sync fails **before** any clone (spy-asserted) with a sanitized, model-naming error in the `failed` state.
|
||||
- [ ] Healthy pipeline behavior unchanged (existing sync integration tests green).
|
||||
@@ -0,0 +1,29 @@
|
||||
# Task 02 — Sync error modal (module-owned, every page)
|
||||
|
||||
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
|
||||
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
|
||||
|
||||
## Objective
|
||||
A readable, accessible modal dialog for sync failures, built by the shared header module (which owns the sync state machine), so every page carrying `#sync-btn` gets it with zero page-markup changes.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/header.js` — in the sync section, add:
|
||||
- `showSyncModal(error)`: lazily create the dialog **once** and append to `document.body` (module-level `let syncModal = null`):
|
||||
- backdrop `<div class="sync-modal-backdrop">`;
|
||||
- panel `<div class="sync-modal" role="alertdialog" aria-modal="true" aria-labelledby="sync-modal-title" aria-describedby="sync-modal-error">` with an `<h2 id="sync-modal-title">Sync failed</h2>`, a `<p id="sync-modal-error">` whose text is set via **`textContent`** (the sanitized error — XSS-safe, never `innerHTML`), and a `<button type="button" class="sync-modal-close" aria-label="Close error dialog">` (×);
|
||||
- opening: add a `.is-open` class (or remove `hidden`), move focus to the close button, remember `document.activeElement` (expected `#sync-btn`);
|
||||
- closing: reverse (focus returns to the remembered element — `#sync-btn` when present), `Esc` keydown on `document` while open, backdrop click (click on the backdrop element itself, not the panel), and the close button all call the same close function; a second failure while open **updates the error text in place** (no stacking).
|
||||
- call `showSyncModal(status.error)` from `applySyncFailure(status)` **after** the existing button-title/aria/`.is-error` + `emitSyncStatus` lines (those stay byte-identical — the Sources page's `#sync-error-banner` keeps rendering off the event).
|
||||
- null-safe: everything guards on `syncBtn`/`document.body`; pages without `#sync-btn` never create the modal (the function is only reachable from the sync state machine).
|
||||
- Update the module docstring's sync bullet: the failed state now also opens the module-owned error modal (2026-08-27, `TODO.md` L4).
|
||||
2. `frontend/assets/styles.css` — `.sync-modal-backdrop` (fixed, full-viewport, `rgba` dim over the page, `z-index` above the header) + `.sync-modal` (centered panel, max-width ≈28rem, the dark-theme **error palette** from PLAN §7.2: panel on the error-surface `#2d1318` family, text `#fca5a5`-class ink, 1px error border; title in ink, error text ink-soft-on-error-surface ≥4.5:1); open/close via `.is-open` (visibility/opacity, no motion under `prefers-reduced-motion`); the close button keeps the global `:focus-visible` 3px outline; 44px touch floor.
|
||||
3. `tests/unit/test_sync_button.py` — add source pins (house style): `header.js` contains `role="alertdialog"`, the `textContent` assignment of the modal error, the `Esc` close binding, the backdrop-click close, focus return to `#sync-btn`, and the `showSyncModal` call inside `applySyncFailure` (after `emitSyncStatus`); `styles.css` carries the `.sync-modal` rules + the reduced-motion stilling. Update any pin that asserts the exact `applySyncFailure` body.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; full suite green (no `app/` change — coverage TOTAL unchanged).
|
||||
- Coverage: **>90%** on `app/` (unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `applySyncFailure` opens the modal with the sanitized error; button title/aria + `bor:sync-status` event behavior byte-identical.
|
||||
- [ ] Modal: `role="alertdialog"`, `aria-modal`, labeled, `textContent`-rendered error, close via button/`Esc`/backdrop, focus in-and-out to `#sync-btn`.
|
||||
- [ ] No page HTML changed (the modal is JS-built); no CDN (A11).
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — Model-down E2E + regressions + commit
|
||||
|
||||
**Phase:** `41_sync_fail_fast_models` · **Source:** `TODO.md:4` — "If the embedding or lite model is not accessible the sync button should fail fast and there should be a modal error popup explaining that the model isn't available."
|
||||
**Story:** `.agent/user_stories/sync-model-fail-fast.md`
|
||||
|
||||
## Objective
|
||||
Prove the whole story in the browser against a **dead model endpoint** — fast failure, readable modal, dismissal, unchanged secondary surfaces, and an untouched healthy pipeline — then commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_sync_model_down.py` (new) — mock-only, DB up, git on PATH. Follow the `test_sync_button.py` module-app pattern, but boot **two** module-scoped apps on distinct ports (import `APP_PORT`, `ADMIN_PASSWORD`, `SESSION_SECRET`, `_wait_http` from `e2e.conftest`; use e.g. `APP_PORT + 41` for the dead-model app so the isolated run never clashes with a session app):
|
||||
- **dead-model app** env: `BOR_LLM_BASE_URL=http://127.0.0.1:9/v1` (closed port — instant connection refused), `BOR_GIT_SOURCES=file://<the test_sync_button fixture repo pattern>` (a local `file://` fixture repo, built the same way `test_sync_button.py` does — a (regressed, non-fail-fast) run would therefore spend real time cloning before failing), its own `BOR_SOURCES_DIR` under `tmp_path`;
|
||||
- `test_model_down_fails_fast_with_modal` — admin login, click `#sync-btn`; expect (budget ≤ ~10 s, contrast with the 60 s healthy budget) the button settling retry-ready **and** the modal visible: `role="alertdialog"`, title "Sync failed", error text naming the model ("embedding model" / the model id), `aria-modal="true"`;
|
||||
- `test_modal_dismissal` — one fresh failure, then close via the × button (focus returns to `#sync-btn`); a fresh failure, close via `Esc`; a fresh failure, close via backdrop click;
|
||||
- `test_sync_error_surfaces_unaffected` — after a failure the button keeps `title` + `.is-error`; on `/sources.html` (same dead-model app) the `#sync-error-banner` renders the error off `bor:sync-status`;
|
||||
- `test_healthy_sync_still_succeeds` — a **healthy** module app (same port scheme, `BOR_LLM_BASE_URL` = the session mock like `test_sync_button.py`) runs the full pipeline to "Synced HH:MM" (counts + idempotency as in phase 32) — proves the probe didn't break the happy path.
|
||||
- Module fixture teardown: terminate both apps (the conftest pattern).
|
||||
2. Regression pass (isolation runs): `tests/e2e/test_sync_button.py` (phase 32 — must stay green unmodified), `test_git_sources_admin.py`, `test_local_directory_sources.py`.
|
||||
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(sync): fail fast with a modal when a model is unavailable`, staging this phase's files; move `.agent/phases/todo/41_sync_fail_fast_models/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_sync_model_down.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (the probe code is fully covered by task 01's tests).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The model-down suite passes in isolation: fast fail before clones, modal with model-naming error, all three dismissal paths, secondary surfaces intact, healthy run unaffected.
|
||||
- [ ] Phase-32/35/38 regression suites green in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Phase 42 — No reply autoscroll
|
||||
|
||||
**Source:** `TODO.md` L5 — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
|
||||
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
|
||||
**Context:** Phase 18 ("follow-the-bottom", owner choice 2026-08-23) added `NEAR_BOTTOM_PX = 200` / `isNearBottom()` / `scrollReveal(wrap, behavior, force)` in `frontend/assets/app.js`: the page auto-scrolls on every `thinking` / `tool` / `delta` frame while the user is within 200px of the bottom. The owner now finds that fighting their own scroll. The gate and the per-frame scrolls are **removed**; scrolling happens only on explicit user intent (submit, restore landing).
|
||||
|
||||
## Objective
|
||||
The chat page never auto-scrolls during a turn. The viewport moves only when the user submits (their message is revealed) or when a persisted conversation is restored (one-shot landing) — both user-initiated.
|
||||
|
||||
## Dependencies
|
||||
- `41_sync_fail_fast_models` (todo) — sequential execution only (no code overlap).
|
||||
- `18_follow_bottom_scroll` (complete) — the code being removed; `14_chat_persistence` (complete) — the restore landing that must survive; `17_thinking_display` / `11_long_answers` (complete) — the thinking window-pin and long-answer behavior this phase must not break.
|
||||
|
||||
## Tasks
|
||||
1. `01_remove_autofollow.md` — strip the phase-18 gate + per-frame scrolls from `app.js`; rewrite the unit pin for the new contract.
|
||||
2. `02_no_autoscroll_e2e_and_commit.md` — replace the phase-18 E2E with the inverse-contract suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_frontend_scroll.py` **rewritten** — pins the new contract: no `NEAR_BOTTOM_PX` / `isNearBottom` in `app.js`; the scroll helper scrolls unconditionally (smooth / reduced-motion-aware); the user-submit path scrolls; the thinking/tool/delta handlers contain **no** page-scroll call; the restore landing keeps its one-shot forced scroll.
|
||||
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_no_reply_autoscroll.py`, run in isolation. `tests/e2e/test_follow_bottom_scroll.py` is **deleted** (behavior intentionally removed by owner direction 2026-08-27).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] During thinking / tool / answer streaming, `window.scrollY` is stable (±1px) while the viewport is scrolled up.
|
||||
- [ ] Submit still reveals the user's message; reload still lands one-shot on the latest message.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
|
||||
- [ ] `uv run pytest tests/e2e/test_no_reply_autoscroll.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_thinking_display.py`, `test_chat_persistence.py`, `test_long_answers.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner direction (2026-08-27, roadmap A1)** revises the phase-18 owner choice (2026-08-23): follow-the-bottom auto-follow is removed; submit-reveal + restore-landing are kept. Recorded in the story file and the `app.js` docstring (PLAN.md §7.4's Scroll row is a PLAN-side revision to be noted by the owner — this phase does not edit PLAN.md).
|
||||
- **A15 unchanged** — the SSE contract is untouched; this is pure client-side behavior.
|
||||
- **The thinking window's internal pin** (`textEl.scrollTop`, phase 17) is untouched here — phase 43 reworks it separately.
|
||||
- **A16/A17 honoured** — one story E2E suite (replacing the removed one), one atomic commit.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 01 — Remove the auto-follow gate and per-frame scrolls
|
||||
|
||||
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md:5` — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
|
||||
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
|
||||
|
||||
## Objective
|
||||
`app.js` scrolls only on explicit user intent: sending a message and the phase-14 restore landing. No scroll happens anywhere in the streaming path.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js` —
|
||||
- **Delete** `export const NEAR_BOTTOM_PX = 200` and `function isNearBottom()`.
|
||||
- **Simplify** `scrollReveal(wrap, behavior = SCROLL, force = false)` → an unconditional `wrap.scrollIntoView({ behavior })` (keep the `SCROLL` constant: smooth, `auto` under `prefers-reduced-motion`; keep the "Calm, don't remove" comment). Rename the gate comment block: the phase-18 "follow-the-bottom scroll contract" paragraph is replaced by the new contract — *"No reply autoscroll (owner direction 2026-08-27, `TODO.md` L5): the page never auto-scrolls while a turn streams. The only scroll call sites are the user submit (reveal my message) and the phase-14 restore landing (one-shot, load-time)."*
|
||||
- **`addMessage(who, html, scrollBehavior = SCROLL, force = false)`** → change the signature to `addMessage(who, html, scroll = false)`: the internal `scrollReveal(wrap, scrollBehavior, force)` becomes `if (scroll) scrollReveal(wrap)`. Update the call sites (line numbers are pre-change anchors):
|
||||
- the **user submit** call (`addMessage("user", renderMarkdown(text))`, ~line 896) → `addMessage("user", renderMarkdown(text), true)` (my message must be revealed — the owner-kept behavior);
|
||||
- the **phase-14 restore** calls (~lines 772/775: `addMessage("user", …, "auto", true)` / `addMessage("brain", …, "auto", true)`) → keep the one-shot forced scroll under the new signature (e.g. `addMessage("user", renderMarkdown(m.text), true)` — the "auto" (non-smooth) behavior for the landing is preserved by passing it through if the new signature keeps a behavior param, otherwise the default `SCROLL` is acceptable and must be noted in the docstring);
|
||||
- the brain first-bubble creations in the SSE handlers (`if (!wrap) wrap = addMessage("brain", "")`, ~lines 955/978/995, plus the `"…"` fallback ~1004 and the error fallback ~1046) → `scroll: false` (default) — a streaming turn never scrolls the page;
|
||||
- `addTyping()` (~line 356): the `scrollReveal(wrap)` after `messagesEl.appendChild(wrap)` is **removed** (a typing bubble must not yank the page).
|
||||
- **SSE handlers** — remove the page-scroll calls: in the `thinking` frame drop the `scrollReveal(wrap); // page follows only while pinned (phase 18)` line **but keep** `textEl.scrollTop = textEl.scrollHeight;` (the thinking *window* pin — phase 17, reworked in phase 43); in the `tool` frame drop its `scrollReveal(wrap);`; in the `delta` frame drop its `scrollReveal(wrap);`.
|
||||
- Update the file-top docstring's scroll paragraph (lines ~73–81: "Scroll (phase 18, owner choice…)") to the new contract.
|
||||
2. `tests/unit/test_frontend_scroll.py` — **rewrite** for the new contract (keep the house style — source pins over `app.js`):
|
||||
- `NEAR_BOTTOM_PX` / `isNearBottom` are **absent** from `app.js`;
|
||||
- the scroll helper scrolls unconditionally (no `force`-or-near-bottom condition in its body);
|
||||
- the user-submit `addMessage` call passes the scroll intent; the brain-bubble creation does not;
|
||||
- the `thinking` / `tool` / `delta` handler bodies contain no `scrollReveal` call (the thinking handler's `textEl.scrollTop` pin is still present);
|
||||
- the restore landing still performs its one-shot scroll (pin the marker comment / call);
|
||||
- the `SCROLL` reduced-motion handling is intact.
|
||||
- Delete the now-obsolete phase-18 test functions (the band constant, the gate logic) — do not leave dead pins.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the rewritten pin file + the full suite green (no `app/` change — coverage TOTAL unchanged).
|
||||
- Coverage: **>90%** on `app/` (unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] No page scroll happens in the streaming path (grep-verifiable + unit-pinned); submit and restore landing still scroll.
|
||||
- [ ] `uv run pytest` green; the thinking window-pin and all message rendering are byte-identical elsewhere.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Task 02 — No-autoscroll E2E (replaces phase 18) + regressions + commit
|
||||
|
||||
**Phase:** `42_no_reply_autoscroll` · **Source:** `TODO.md:5` — "Get rid of the chat reply autoscroll, it's breaking things like making it impossible for the user to scroll while a reply generates."
|
||||
**Story:** `.agent/user_stories/no-reply-autoscroll.md`
|
||||
|
||||
## Objective
|
||||
Prove the inverse of the phase-18 contract in the browser: no streaming autoscroll, submit-reveal and restore-landing intact — then delete the obsolete phase-18 suite and commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_no_reply_autoscroll.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule:
|
||||
- `test_no_autoscroll_during_long_answer` — `LONG_ANSWER_TRIGGER` question (the mock's ~8 s long answer); once the answer starts streaming, `window.evaluate` a scroll up ~2× the answer's height; sample `window.scrollY` across ≥10 frames (and after `done`): stable within 1px;
|
||||
- `test_no_autoscroll_during_thinking` — `THINKING_TRIGGER` question; scroll up during the ~4.5 s thinking stream; viewport stable across chunks (no per-chunk page follow);
|
||||
- `test_submit_reveals_user_message` — in a populated conversation scrolled to the very top, send a question; after send the user's message is in view (its bounding box within the viewport);
|
||||
- `test_restore_landing_one_shot` — settle a conversation (phase-14 persistence), reload; the page lands on the latest message and stays (no further movement while idle);
|
||||
- `test_answer_content_intact` — the long answer completes with sources; a thinking turn persists + restores (collapsed block, phase 17).
|
||||
2. **Delete** `tests/e2e/test_follow_bottom_scroll.py` (its behavior is intentionally removed — owner direction 2026-08-27; the unit pin was rewritten in task 01).
|
||||
3. Regression pass (isolation runs): `test_chat_rag.py`, `test_thinking_display.py`, `test_chat_persistence.py`, `test_long_answers.py`, `test_smoke.py` — all green; fix only true regressions.
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `fix(chat): stop autoscrolling while a reply streams (owner direction)`, staging this phase's files (including the deleted E2E); move `.agent/phases/todo/42_no_reply_autoscroll/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_no_reply_autoscroll.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The new suite passes in isolation; the phase-18 suite is gone; the five regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Phase 43 — Thinking scroll back (user scroll + generate-time autoscroll)
|
||||
|
||||
**Source:** `TODO.md` L7 — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
|
||||
**Story:** `.agent/user_stories/thinking-scroll-back.md`
|
||||
**Context:** Phase 17 streams reasoning into the collapsible `.thinking` block with a per-chunk bottom-pin (`textEl.scrollTop = textEl.scrollHeight` in the `thinking` SSE handler). Phase 21 (owner choice 2026-08-24) made `.thinking-text` a no-scroll live tail: `overflow-y: hidden` (the JS pin is the sole scroller). The owner now reverses phase 21: the window is user-scrollable again, and the pin becomes **gated** — follow the tail only while the user is pinned near the window's bottom. This is the window-level successor of the phase-18 pattern (the page-level one is removed in phase 42, which runs first and touches the same `thinking` handler line — order matters).
|
||||
|
||||
## Objective
|
||||
The Thinking block follows its live tail while reasoning is generating **and** the user is at the bottom; a scrolled-up user is never yanked down, and returning to the bottom resumes following.
|
||||
|
||||
## Dependencies
|
||||
- `42_no_reply_autoscroll` (todo) — must run **first**: it strips the page-level scroll from the same `thinking` handler; this phase then reworks the window pin in the cleaned-up handler.
|
||||
- `17_thinking_display` (complete) — the block, the pin, the auto-collapse on first delta.
|
||||
- `21_thinking_no_scroll` (complete) — the `overflow-y: hidden` + 320px window being reversed (the 320px clip is kept).
|
||||
|
||||
## Tasks
|
||||
1. `01_window_user_scrollable.md` — CSS: `overflow-y: auto` back, comment replaced (owner direction 2026-08-27).
|
||||
2. `02_gated_bottom_pin.md` — `app.js`: `THINKING_NEAR_BOTTOM_PX = 32` + gated pin; unit pin rewritten (phase-21 file replaced).
|
||||
3. `03_thinking_scroll_e2e_and_commit.md` — replace the phase-21 E2E with the new-contract suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_thinking_no_scroll.py` **deleted**, replaced by `tests/unit/test_thinking_scroll.py` — pins: `overflow-y: auto` + `max-height: 320px` in the `.thinking-text` rule; the 2026-08-27 owner-direction comment; `export const THINKING_NEAR_BOTTOM_PX = 32`; the pin is gated on `isThinkingNearBottom(textEl)` (no unconditional pin).
|
||||
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_thinking_scroll.py`, run in isolation. `tests/e2e/test_thinking_no_scroll.py` is **deleted** (behavior intentionally reversed).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Wheel/drag/keyboard move `.thinking-text` (frozen-tail state); computed `overflow-y: auto`, `max-height: 320px`.
|
||||
- [ ] While pinned at the window bottom: each chunk re-pins to the tail (±1px). Scrolled up: no re-pin across chunks. Return to bottom: following resumes.
|
||||
- [ ] Auto-collapse on first delta, reduced-motion stillness, answer-bubble scroll (phase 11), restored-collapsed block (phase 17) all unchanged.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_scroll.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_thinking_display.py`, `test_chat_persistence.py`, `test_no_reply_autoscroll.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner direction (2026-08-27, roadmap A2)** reverses the phase-21 owner choice (2026-08-24): the window is user-scrollable again; autoscroll only while pinned near the bottom (32px band). The 320px clip is kept (owner-confirmed).
|
||||
- **A15 unchanged** — SSE contract untouched; pure client-side.
|
||||
- **A16/A17 honoured** — one story E2E suite (replacing the removed one), one atomic commit.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 01 — Window user-scrollable again (CSS + unit pin swap)
|
||||
|
||||
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
|
||||
**Story:** `.agent/user_stories/thinking-scroll-back.md`
|
||||
|
||||
## Objective
|
||||
Restore user scrolling on the Thinking window — `overflow-y: auto`, 320px clip kept, comment updated — and swap the phase-21 unit pins for the new contract so the suite stays green.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/styles.css` — in the phase-17/21 thinking section, the `details.thinking .thinking-text` rule:
|
||||
- `overflow-y: hidden;` → `overflow-y: auto;`
|
||||
- replace the phase-21 comment (*"no user scroll back (owner choice 2026-08-24): the window is a live tail only — the phase-17 JS bottom-pin … is the sole scroller"*) with: *"user-scrollable window (owner direction 2026-08-27, `TODO.md` L7): autoscroll follows the live tail only while the user is pinned near the window's bottom — the phase-17 pin, gated in app.js (task 02: `THINKING_NEAR_BOTTOM_PX`); scrolling up pauses the follow, returning to the bottom resumes it."*
|
||||
- `max-height: 320px` and **every other declaration in the rule stay byte-identical**; the tightened `p`/`ul` margins rule and the reduced-motion chevron block are untouched.
|
||||
2. **Delete** `tests/unit/test_thinking_no_scroll.py` (its pins assert the reversed behavior) and create `tests/unit/test_thinking_scroll.py` (house style — source pins, same slicing helpers as the deleted file) with, for now, the CSS contract only:
|
||||
- the `.thinking-text` rule body contains `overflow-y: auto`, `max-height: 320px`, and the 2026-08-27 owner-direction comment (assert `"owner direction 2026-08-27"` and `"TODO.md L7"`);
|
||||
- `overflow-y: hidden` / `overflow-y: scroll` are absent from that rule body;
|
||||
- the phase-17 bottom-pin marker (`textEl.scrollTop = textEl.scrollHeight`) is still present in `app.js` (it becomes gated in task 02 — the pin's existence is asserted now so task 02's diff stays minimal and reviewable).
|
||||
- The JS-gate pins (`THINKING_NEAR_BOTTOM_PX`, `isThinkingNearBottom`, gated call) are added in task 02 — do not assert them yet.
|
||||
3. Check `tests/e2e/test_thinking_no_scroll.py` still passes at this checkpoint: it asserts computed `overflow-y: hidden` — **it will fail** (the behavior is intentionally changed). Per the gate, the phase's E2E replacement is task 03; to keep the per-task gate green, **delete** that E2E file in this task as well (its behavior is reversed; task 03 lands the replacement suite). Note the deletion in the final commit message of task 03.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_thinking_scroll.py` green; full `uv run pytest` green (the deleted E2E file does not run under the unit/integration gate, but the full pytest run must not collect it either — it is gone from the tree).
|
||||
- Coverage: **>90%** on `app/` (unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `overflow-y: auto` + 320px clip + new comment in the CSS rule; all other declarations byte-identical.
|
||||
- [ ] Old unit + old E2E phase-21 files deleted; new unit file pins the CSS contract and the surviving pin marker.
|
||||
- [ ] `uv run pytest` green at this checkpoint.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Task 02 — Gated bottom pin (follow while pinned)
|
||||
|
||||
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
|
||||
**Story:** `.agent/user_stories/thinking-scroll-back.md`
|
||||
|
||||
## Objective
|
||||
The phase-17 per-chunk pin becomes a **gate**: the window follows the live tail only while the user is near its bottom; a scrolled-up user is never re-pinned; returning to the bottom re-arms the pin automatically.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js` —
|
||||
- add (near the existing `SCROLL` constant, with the phase-18 comment block already removed by phase 42):
|
||||
```js
|
||||
/* Thinking-window follow-the-tail contract (owner direction
|
||||
* 2026-08-27, `TODO.md` L7): the scratchpad autoscrolls to its live
|
||||
* tail only while the user is pinned near the window's bottom —
|
||||
* the 32px band is the "window bottom in view" threshold. Scrolling
|
||||
* up pauses the follow; returning to the bottom resumes it (the
|
||||
* check runs on every chunk). Exported so the band is unit-pinned
|
||||
* (same pattern as TURN_TIMEOUT_MS). */
|
||||
export const THINKING_NEAR_BOTTOM_PX = 32;
|
||||
|
||||
function isThinkingNearBottom(textEl) {
|
||||
return (
|
||||
textEl.scrollHeight - textEl.scrollTop - textEl.clientHeight <=
|
||||
THINKING_NEAR_BOTTOM_PX
|
||||
);
|
||||
}
|
||||
```
|
||||
- in the `thinking` SSE handler, replace the phase-17 block:
|
||||
```js
|
||||
if (block.open) {
|
||||
textEl.scrollTop = textEl.scrollHeight; // pin the stream to the bottom
|
||||
}
|
||||
```
|
||||
(phase 42 already removed the `scrollReveal(wrap)` line there) with the gated pin:
|
||||
```js
|
||||
if (block.open && isThinkingNearBottom(textEl)) {
|
||||
// Follow the live tail only while the user is pinned to the window
|
||||
// bottom (owner direction 2026-08-27); a scrolled-up reader is
|
||||
// never re-pinned — returning to the bottom re-arms the pin.
|
||||
textEl.scrollTop = textEl.scrollHeight;
|
||||
}
|
||||
```
|
||||
- everything else in the handler (acc, sawThinking, clearTurnTimeout, ensureThinkingBlock, `textEl.innerHTML = renderMarkdown(thinkingAcc)`) stays byte-identical.
|
||||
2. `tests/unit/test_thinking_scroll.py` — extend (from task 01) with the JS pins:
|
||||
- `app.js` exports `const THINKING_NEAR_BOTTOM_PX = 32`;
|
||||
- `isThinkingNearBottom` computes `scrollHeight - scrollTop - clientHeight <= THINKING_NEAR_BOTTOM_PX`;
|
||||
- the thinking handler's pin is gated — the pin line is preceded by `isThinkingNearBottom(textEl)` in the same `if` (assert the combined condition; assert there is **no** unconditional `if (block.open) { textEl.scrollTop = ... }` left);
|
||||
- `block.open` is still part of the gate (closed blocks never pin);
|
||||
- the restore path renders collapsed blocks (phase 17) — keep the surviving assertion from task 01.
|
||||
3. `uv run pytest` green at this checkpoint (E2E not run by the unit gate; the replacement suite lands in task 03).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the extended pin file; full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The pin fires only when the block is open **and** the window is within 32px of its bottom; scrolled-up users are never re-pinned; the gate re-arms on return (by construction — the check runs per chunk).
|
||||
- [ ] `uv run pytest` green at this checkpoint.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — Thinking-scroll E2E (replaces phase 21) + regressions + commit
|
||||
|
||||
**Phase:** `43_thinking_scroll_back` · **Source:** `TODO.md:7` — "Add scrolling back to the thinking block, but have it autoscroll while thinking content is generating."
|
||||
**Story:** `.agent/user_stories/thinking-scroll-back.md`
|
||||
|
||||
## Objective
|
||||
Prove the full contract in the browser — user scroll restored, follow-while-pinned, pause-on-scroll-up, resume-on-return, CSS contract, and the phase-11/17 regressions — then commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_thinking_scroll.py` (new) — mock-only, DB up. Reuse the phase-21 determinism machinery (`mock_llm.compose_thinking` is already ~2 700 chars ≈ 4.5 s of paced frames, overflowing the 320px window ~2×; the phase-20 hesitation trigger gives a deterministic 4 s frozen-tail state with the block open). Per the story's Playwright Mapping Rule:
|
||||
- `test_thinking_window_user_scrollable` — frozen tail: focus `.thinking-text`, wheel up / `Home` / mouse-drag up → `scrollTop` moves and earlier content is visible;
|
||||
- `test_thinking_window_follows_while_pinned` — live stream: at the window bottom, after the 2nd-to-last and the last chunk the window is pinned to the tail (±1px); the last chunk's text renders inside the visible rectangle;
|
||||
- `test_thinking_window_stops_on_scroll_up` — mid-stream: scroll up ~half the window; over the next ≥5 chunks `scrollTop` stable (±1px);
|
||||
- `test_thinking_window_resumes_on_return` — from the paused state, set `scrollTop` to the bottom; on the next chunk the window is re-pinned to the tail (±1px);
|
||||
- `test_thinking_window_css_contract` — computed `overflow-y: auto`, `max-height: 320px`, `scrollHeight > clientHeight` (real clip);
|
||||
- `test_answer_bubble_still_scrollable` (phase 11) — long answer: page scrolls, bubble overflow untouched;
|
||||
- `test_restored_collapsed_thinking_unaffected` (phase 17) — settled thinking turn reloads collapsed with full text.
|
||||
2. Regression pass (isolation runs): `test_thinking_display.py`, `test_chat_persistence.py`, `test_no_reply_autoscroll.py` (phase 42 — the cleaned `thinking` handler must not have lost the phase-42 contract), `test_smoke.py`.
|
||||
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(chat): thinking window scrolls again, follows the tail only while pinned`, staging this phase's files **including the two deleted phase-21 test files** (`tests/unit/test_thinking_no_scroll.py`, `tests/e2e/test_thinking_no_scroll.py`) and the new unit + E2E files; move `.agent/phases/todo/43_thinking_scroll_back/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_thinking_scroll.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The new suite passes in isolation (all seven tests); the four regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit covering both deleted and both new test files; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Phase 44 — Markdown tables (chat, viewer, thinking)
|
||||
|
||||
**Source:** `TODO.md` L6 — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
|
||||
**Story:** `.agent/user_stories/markdown-tables.md`
|
||||
**Context:** `frontend/assets/markdown.js` is the shared escape-first renderer (no libs, A11): fence protection → escape → inline transforms (`code`, `**bold**`, `*em*`, h1–h3, lists) → paragraph pass → fence restore. It has **no table support** — GFM pipe tables render as one raw `|`-littered paragraph. The renderer serves the chat answer, the document viewer/modal, and the thinking block, so one change covers all three.
|
||||
|
||||
## Objective
|
||||
GFM pipe tables render as semantic, styled, XSS-safe `<table>` elements everywhere the shared renderer runs, with a horizontal-overflow guard for wide tables.
|
||||
|
||||
## Dependencies
|
||||
- `43_thinking_scroll_back` (todo) — sequential only (the thinking block also renders markdown; no shared-file conflict beyond the renderer itself).
|
||||
- `08_story_dark_tech_theme` (complete) — the palette tokens `.md-table` must use.
|
||||
- `26_document_modal_viewer` / `10_story_document_viewer` (complete) — the second renderer consumer (viewer/modal).
|
||||
|
||||
## Tasks
|
||||
1. `01_table_renderer_and_styles.md` — table pass in `markdown.js` + `.md-table` CSS.
|
||||
2. `02_mock_table_trigger.md` — deterministic table answer (incl. a wide table) in `mock_llm.py`.
|
||||
3. `03_tables_e2e_and_commit.md` — unit pins + story E2E suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: new `tests/unit/test_markdown_tables.py` — source pins in the house style (regex over `markdown.js` / `styles.css`): the table-protection pass exists and runs **after** the fence pass and **before** the escape pass; cells are escaped + inline-transformed; output carries `class="md-table"`, `<thead>`, `th scope="col"`, and the `.md-table-wrap` wrapper; `styles.css` has the wrapper overflow rule + table borders + reduced-motion-relevant rules. (Behavior is browser-proven by the E2E; unit pins catch silent regressions without a browser — the established frontend pattern.)
|
||||
- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_markdown_tables.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A pipe table in a chat answer renders `<div class="md-table-wrap"><table class="md-table">` with `<thead>`/`<tbody>`, `<th scope="col">` headers, correct cell texts; no raw `|---|` in the bubble.
|
||||
- [ ] A wide table scrolls inside its wrapper; the 46rem column does not overflow the page.
|
||||
- [ ] XSS-safe (escaped cells), fences win over tables, lone pipes stay text.
|
||||
- [ ] The document viewer/modal renders the same table for a fixture document containing one.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL unchanged.
|
||||
- [ ] `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A11 untouched** — still the local ~90-line renderer, no library, no CDN.
|
||||
- **Owner-locked (2026-08-27, roadmap A3):** scope = GFM pipe tables (header + separator + body); links/blockquotes/hr out of scope; alignment colons parsed but rendered left; wide tables get the `overflow-x: auto` wrapper.
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Task 01 — Table pass in the shared renderer + styles
|
||||
|
||||
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
|
||||
**Story:** `.agent/user_stories/markdown-tables.md`
|
||||
|
||||
## Objective
|
||||
`renderMarkdown` turns GFM pipe-table blocks into semantic tables (XSS-safe, inline markdown in cells), wrapped in a horizontal-overflow container, styled in the dark-tech palette.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/markdown.js` — in `renderMarkdown(md)`, between step 1 (fence protection) and step 2 (escape + inline transforms), add a **table protection pass** using the same placeholder mechanism as the fences:
|
||||
- **Detection** (line-oriented over the fence-protected text): a **table block** starts at a line containing `|` whose **next** line is a **separator** — the separator line consists of ≥1 pipe-separated cells, each matching `^\s*:?-+:?\s*$` (allowing a leading/trailing pipe and inter-cell whitespace). The block then extends over every following line that still contains `|` (the body; zero body rows is a valid table — header only). A maximal such block is one table. Anything else (a single `|` in prose, a separator with no `|`-header line above it, a 1-line "table") is left untouched.
|
||||
- **Extraction:** for each table run, split each line on `|`, drop the leading/trailing empty entries produced by leading/trailing pipes, `trim()` each cell.
|
||||
- **Cell rendering:** each cell goes through the same inline pipeline as the rest of the text — `escapeHtml(cell)` first (XSS-safe, invariant of the renderer), then the inline transforms (`` `code` ``, `**bold**`, `*em*` — the exact same `.replace` chain step 2 uses; factor the inline chain into a small local helper if it makes the cell path cleaner, keeping the whole-text path byte-identical in output).
|
||||
- **Assembly:**
|
||||
```html
|
||||
<div class="md-table-wrap">
|
||||
<table class="md-table">
|
||||
<thead><tr><th scope="col">h1</th>…</tr></thead>
|
||||
<tbody><tr><td>…</td>…</tr>…</tbody>
|
||||
</table>
|
||||
</div>
|
||||
```
|
||||
Rows with fewer cells than the header are padded with empty `<td>`; rows with more are truncated to the header width (defensive — the mock and real answers are well-formed). Alignment colons in the separator are **parsed but ignored** (all cells left — owner decision).
|
||||
- **Placeholders:** reuse the `\u0000CODEn\u0000` array pattern — e.g. push the table HTML into a second array and emit `\u0000TABLEn\u0000`, restored alongside the code blocks in step 4 (update the restore step accordingly; tables inside the protected span are already final HTML — they must not re-enter the paragraph pass, which the placeholder guarantees).
|
||||
- Update the file's header comment (the ~60-line no-CDN renderer now also does tables — 2026-08-27, `TODO.md` L6).
|
||||
2. `frontend/assets/styles.css` — near the markdown/content styling (the chat bubble content rules):
|
||||
- `.md-table-wrap { overflow-x: auto; }` — the wrapper is the scroller;
|
||||
- `.md-table { border-collapse: collapse; width: 100%; font-size: 0.9rem; }`;
|
||||
- `.md-table th, .md-table td { border: 1px solid var(--line); padding: 0.4rem 0.6rem; text-align: left; vertical-align: top; }`;
|
||||
- `.md-table thead th { background: <surface-darker token>; color: var(--ink); }` — pick the existing token that keeps ≥4.5:1 (PLAN §7.2: ink `#e8ebf4` on surface `#121a2e` is 14.5:1 — use the plain surface family, not brand);
|
||||
- ensure the rule set is inside or consistent with the reduced-motion constraints (no animation involved — nothing to still).
|
||||
3. Sanity: run an existing markdown-consuming E2E (e.g. `test_chat_rag.py`) to confirm byte-identical output for non-table content (the inline-chain factor, if done, must not change any existing rendering).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_markdown_tables.py` (new) source pins per the phase overview (pass ordering, escape-first for cells, output markers, CSS rules). Full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `renderMarkdown` handles the shapes in the story's acceptance criteria 1–4 (table, XSS cell, fence-wins, non-tables stay text) — verifiable via the unit pins now and the E2E in task 03.
|
||||
- [ ] No existing rendering changes for non-table markdown (regression suite from step 3 green).
|
||||
@@ -0,0 +1,40 @@
|
||||
# Task 02 — Deterministic table answer in the mock
|
||||
|
||||
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
|
||||
**Story:** `.agent/user_stories/markdown-tables.md`
|
||||
|
||||
## Objective
|
||||
The E2E mock serves a byte-stable table answer (plus a deliberately wide table and an XSS cell) on demand, following the existing trigger convention.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/mock_llm.py` —
|
||||
- add `TABLE_TRIGGER = "show me a table"` (same case-insensitive-substring convention as `LONG_ANSWER_TRIGGER` / `THINKING_TRIGGER` / `TOOLS_TRIGGER`);
|
||||
- in `compose_answer(body)`, **before** the default tail-echo branch (and before `DEFLECT_MODE` — a deflection prompt never carries the marker, same reasoning as `SUMMARY_MODE`): when the trigger is in the lowercased user message, return the fixed table answer:
|
||||
```
|
||||
Here's the shape, in a table:
|
||||
|
||||
| Service | Port | Host |
|
||||
|---|---|---|
|
||||
| Caddy | 80 | homelab-gw |
|
||||
| GitLab | 8929 | homelab-git |
|
||||
| ntfy | 2087 | homelab-ntfy |
|
||||
|
||||
<img src=x onerror=alert(1)>
|
||||
|
||||
And the wide one:
|
||||
|
||||
| A very long column header to force overflow | Second column with some padding text | Third column | Fourth | Fifth |
|
||||
|---|---|---|---|---|
|
||||
| value-one | value-two | value-three | value-four | value-five |
|
||||
```
|
||||
(The `<img onerror>` line is the XSS assertion's payload — it must survive the mock byte-for-byte so the E2E can prove the renderer neutralizes it; the wide table guarantees `scrollWidth > clientWidth` inside the 46rem column.)
|
||||
- keep the answer a plain grounded response (no `DEFLECT_MODE` interplay): the trigger question is asked against an on-topic fixture so the honesty gate is HIGH in the E2E (the suite asserts non-deflection as part of the table test).
|
||||
2. Update the module docstring's marker list (the file documents every trigger — add the table row).
|
||||
3. `uv run pytest tests/e2e/mock_llm.py-related unit tests` — run `uv run pytest tests/unit -k "mock" tests/integration -x` (or the mock's existing test file, if any — check `tests/` for mock-specific tests) to prove the new branch breaks no existing flow; the full suite is green (the new branch only fires on the marker).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: full suite green; the new branch is covered by the E2E (task 03) — if a mock-level unit test file exists, add the table case there so the branch is unit-covered too.
|
||||
- Coverage: **>90%** on `app/` (mock lives in `tests/` — the gate is unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `TABLE_TRIGGER` returns the fixed table answer (byte-stable), including the XSS line and the wide table; no existing mock behavior changes for marker-less requests.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — Tables E2E + regressions + commit
|
||||
|
||||
**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
|
||||
**Story:** `.agent/user_stories/markdown-tables.md`
|
||||
|
||||
## Objective
|
||||
Prove the table contract in the browser — chat, overflow, XSS, viewer, and the two "not a table" regressions — then commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_markdown_tables.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule:
|
||||
- `test_chat_table_renders` — ask an on-topic question containing `TABLE_TRIGGER` (pick a fixture topic that retrieves HIGH — reuse a question pattern from `test_chat_rag.py`); the brain bubble contains `<div class="md-table-wrap"><table class="md-table">`, a `<thead>` with three `<th scope="col">` (Service/Port/Host), the body cell texts ("Caddy", "8929", …), and **no** `|---|` separator text in the bubble;
|
||||
- `test_wide_table_scrolls` — in the same answer, the wide table's wrapper has `scrollWidth > clientWidth`; horizontal scrolling (wheel/`scrollLeft`) moves it; the page itself has no horizontal overflow (`document.documentElement.scrollWidth <= clientWidth`);
|
||||
- `test_table_xss_safe` — the `<img src=x onerror=…>` line renders as visible text (no `<img>` element inside the bubble; `onerror` can never fire — assert `page.evaluate` found zero injected img nodes and the tag text is present);
|
||||
- `test_viewer_table_renders` — add a fixture document (extend `tests/fixtures/docs/homelab/` with a small `.md` file containing a pipe table — e.g. `tables.md` with a 3×3 table; re-import per the `test_document_documents.py`/`test_import_documents.py` fixture pattern), open it from the Sources table (admin) in the modal; the modal content renders `<table class="md-table">`;
|
||||
- `test_fence_not_a_table` — a question/fixture whose content puts `|`-heavy lines inside a ``` fence (existing fixtures have fenced blocks — pick/extend one) renders `<pre><code>` with no `<table>`;
|
||||
- `test_plain_pipe_stays_text` — an off-trigger grounded answer containing a single `|` in prose (assert via an existing deterministic answer or a minimal new fixture) renders as text, no `<table>`.
|
||||
- Assert non-deflection (`.is-deflected` absent) in the table tests — the honesty gate interplay is part of the contract.
|
||||
2. Regression pass (isolation runs): `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py` (the renderer is shared — summaries render through it too), `test_smoke.py`.
|
||||
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(chat): render markdown tables in answers, viewer, and thinking`, staging this phase's files; move `.agent/phases/todo/44_markdown_tables/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The story E2E suite passes in isolation (all six tests); the four regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Phase 45 — Agent makes as many tool calls as it wants
|
||||
|
||||
**Source:** `TODO.md` L8 — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
|
||||
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
|
||||
**Context:** Phase 37 shipped the grounded-turn agent loop (`app/rag/agent.py::run_agent`) with per-turn budgets — `agent_list_calls` / `agent_read_calls` (default 1 each, `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS`), "budgets-as-kill-switch" locked decision. The exhaustion refusals (`LIST_EXHAUSTED` / `READ_EXHAUSTED`) are where correct multi-document answers die. Owner direction (2026-08-27): remove both budgets; the loop keeps one guard — a configurable **round cap** that also doubles as the no-tools kill switch (`0`).
|
||||
|
||||
## Objective
|
||||
`list_documents` / `read_document` can be called as many times as the model needs (re-lists included), bounded only by `BOR_AGENT_MAX_ROUNDS` (default 10; `0` = no tools, byte-identical to the pre-phase-37 path).
|
||||
|
||||
## Dependencies
|
||||
- `44_markdown_tables` (todo) — sequential only (no shared files: this phase is `app/` + tests + mock).
|
||||
- `37_agent_document_tools` (complete) — the loop, the `tool` SSE event, the UI tool lines, the per-turn `tool_calls=N` log field, and the phase-37 locked decision being revised.
|
||||
- `31_kb_overview_prompt` (complete) — the `lite` one-shot path is untouched by this phase.
|
||||
|
||||
## Tasks
|
||||
1. `01_config_round_cap.md` — the server core, atomically: `agent_max_rounds` replaces the budgets in `app/config.py` + `app/rag/agent.py`, unit + integration rewrites, `.env.example` (one task so the per-task gate stays green).
|
||||
2. `02_mock_multi_read_flow.md` — the E2E mock's deterministic multi-read (list → read #1 → read #2 → answer) flow.
|
||||
3. `03_unlimited_tools_e2e_and_commit.md` — story E2E + phase-37 regression + PLAN.md revision note + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_agent.py` rewritten around the round cap (always-calling mock LLM: N tool rounds then a forced `tools=None` final answer; `max_rounds=0` → exactly one request with `tools=None`; rejected-call spam — unknown tool / already-in-context — is bounded by the cap, not by budgets; re-lists execute and count in `tool_calls`); `tests/unit/test_config.py` (default 10, `BOR_AGENT_MAX_ROUNDS` override, `0`, the budget env vars are gone).
|
||||
- Integration: `tests/integration/test_chat_api.py` — the `agent_list_calls=0, agent_read_calls=0` fixtures become `agent_max_rounds=0`; the tool SSE event shape and the `done.sources` extension assertions stay.
|
||||
- Coverage: **>90%** on `app/` — `agent.py` + `config.py` fully covered.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_agent_unlimited_tools.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS` are gone (config, `.env.example`, agent, tests); no exhaustion refusal remains.
|
||||
- [ ] A multi-read turn (list + 2 reads) streams three tool lines, answers non-deflected, and `done.sources` lists the retrieved doc(s) + both reads deduped.
|
||||
- [ ] `agent_max_rounds=0` → single `tools=None` request (kill switch); at the cap the loop forces a final no-tools answer (log warning kept).
|
||||
- [ ] `tool` SSE event shape and `tool_calls=N` per-turn log field unchanged.
|
||||
- [ ] `.agent/PLAN.md` carries the phase-37 revision note (owner permission 2026-08-27, `TODO.md` L8) — the only PLAN edit in this phase.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_agent_unlimited_tools.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_agent_document_tools.py`, `test_chat_rag.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked revision (2026-08-27, roadmap R2):** the phase-37 "budgets-as-kill-switch" decision is **revised** — both per-tool budgets removed; `BOR_AGENT_MAX_ROUNDS` (default 10) is the only loop guard and the kill switch (`0`). Recorded as a PLAN.md revision note (the established owner-permission pattern, like the A10/A7/A9/A15 notes) — a recorded revision, not a silent deviation (AGENTS.md rule 3).
|
||||
- **A15 extension unchanged** — the `tool` SSE event shape, the `done` shape, and the per-turn log line (`tool_calls=N`) are untouched; the revision note amends the phase-37 note's budget wording only.
|
||||
- **Rejections kept:** `Unknown tool.`, `MISSING_READ_ARGS`, `Already in your context.` (non-budget rejections; the cap bounds their pathological repetition).
|
||||
- **A17 honoured** — one atomic commit.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Task 03 — Unlimited-tools E2E + PLAN revision note + commit
|
||||
|
||||
**Phase:** `45_agent_unlimited_tools` · **Source:** `TODO.md:8` — "Allow the LLM to make as many tool calls as it wants, remove the restrictions, they're causing problems getting correct answers"
|
||||
**Story:** `.agent/user_stories/agent-unlimited-tools.md`
|
||||
|
||||
## Objective
|
||||
Prove the multi-tool turn end to end, record the phase-37 decision revision in PLAN.md, run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_agent_unlimited_tools.py` (new) — mock-only, DB up (the grounded-turn prerequisite: the fixture KB imported, per the `test_agent_document_tools.py` fixture pattern):
|
||||
- `test_multi_read_turn` — a grounded question carrying `TOOLS_TRIGGER` + `MULTI_READ_TRIGGER`: the turn streams **three** tool lines (`.tool-call` rows: one `list_documents` — "is listing documents" — and two `read_document` — "is reading <source/path>") in order, then a final non-deflected answer containing the mock's "I read <path1> and <path2>." line;
|
||||
- `test_done_sources_include_reads` — the source chips under the answer list the retrieval doc(s) **plus both** read documents, deduped (the phase-37 `done.sources` extension contract, now with 2 reads);
|
||||
- `test_relist_allowed` — the listing tool ran without a "No listing budget left" refusal: assert no refusal text anywhere in the bubble/tool lines (the old refusal strings must be gone — grep the app for them is task 01's job; here assert the UI never shows one);
|
||||
- `test_single_tool_flow_regression` (phase 37) — the original 3-step flow (marker without the multi-read trigger) still answers after exactly one read with its single tool pair (this may be a targeted re-assertion; the full suite `test_agent_document_tools.py` runs in the regression pass).
|
||||
2. `.agent/PLAN.md` — **the only PLAN edit in this phase** (owner-locked revision, roadmap R2): in the §4 SSE-revision block, after the phase-37 revision note, add a new note in the established style:
|
||||
> **SSE revision (phase 45, owner permission 2026-08-27):** the phase-37
|
||||
> per-turn tool budgets are **removed** (owner: "allow the LLM to make
|
||||
> as many tool calls as it wants — `TODO.md` L8): `BOR_AGENT_LIST_CALLS`
|
||||
> / `BOR_AGENT_READ_CALLS` no longer exist; `BOR_AGENT_MAX_ROUNDS`
|
||||
> (default 10) caps the tool rounds and `0` disables the tools
|
||||
> entirely (the pre-phase-37 path). The `tool` event shape and the
|
||||
> `done` shape are unchanged — a recorded revision of the phase-37
|
||||
> note's budget wording, not a silent deviation.
|
||||
Also update the phase-37 note's budget clause if it reads as current
|
||||
truth ("budgeted by `BOR_AGENT_LIST_CALLS` / `BOR_AGENT_READ_CALLS`")
|
||||
by appending "(removed in phase 45 — see the revision note below)".
|
||||
Touch **nothing else** in PLAN.md (Protocol B: no roadmap-table edit for appended phases).
|
||||
3. Regression pass (isolation runs): `test_agent_document_tools.py` (phase 37 — must pass **unmodified**), `test_chat_rag.py`, `test_smoke.py`.
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(rag): unbounded agent tool calls behind a round cap (owner revision)`, staging this phase's files **including the force-added `.agent/PLAN.md`** (AGENTS.md rule 8: `git add -f .agent/PLAN.md`) and the phase dir move `.agent/phases/todo/45_agent_unlimited_tools/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_agent_unlimited_tools.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (task 01's rewritten tests carry it).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The multi-read E2E suite passes in isolation; the phase-37 suite passes unmodified in isolation.
|
||||
- [ ] PLAN.md carries the phase-45 revision note (owner permission 2026-08-27) and nothing else changed.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Phase 47 — Import quadlet + jinja files
|
||||
|
||||
**Source:** `TODO.md` L10–L11 — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
|
||||
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
|
||||
**Context:** A9 (LOCKED, revised 2026-08-21): default import formats `md, markdown, txt, yaml, yml, json, py`; `app/config.py::_ALLOWED_IMPORT_EXTENSIONS` bounds `BOR_IMPORT_EXTENSIONS` (narrow-only); `app/rag/chunker.py::_FORMAT_CHUNKERS` maps suffix → chunker (unknown suffix → `chunk_text` fallback). The owner-locked revision (2026-08-27, roadmap R1): ten new formats join the allowed **and** default set — the full Podman quadlet family (`container, network, volume, image, pod, kube, swap, os, endpoint`) plus `j2` — chunked as plain text.
|
||||
|
||||
## Objective
|
||||
Quadlet unit files and Jinja templates are indexed like any other A9 format: allowed + default in config, dispatched to plain-text chunking, and provable end to end (import → catalog → viewer → retrieval).
|
||||
|
||||
## Dependencies
|
||||
- `46_mobile_hamburger_nav` (todo) — sequential only (no shared files: this phase is `app/` + `scripts/` + tests + fixtures + docs).
|
||||
- `38_local_directory_sources` / `28_git_based_sources` (complete) — the import path the new formats ride (`import_sources` / sync).
|
||||
- `02_story_import_documents` (complete) — the A9 format machinery (walk, title, delta, prune) the new formats inherit unchanged.
|
||||
|
||||
## Tasks
|
||||
1. `01_config_formats.md` — allowed + default extension sets, `.env.example`, config unit tests.
|
||||
2. `02_chunker_dispatch_fixtures.md` — chunker dispatch for the ten suffixes + the fixture files.
|
||||
3. `03_importer_integration.md` — importer walk/delta/prune parity + integration test.
|
||||
4. `04_quadlet_e2e_and_docs_commit.md` — story E2E + README + PLAN A9 revision note + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `test_config.py` (allowed set contains all ten; the default CSV carries them after the original seven; the env validator accepts the new names and still rejects unknown ones; `import_extension_set` dotted form); `test_chunker.py` (dispatch for **every** new suffix → `chunk_text` semantics: a quadlet TOML fixture and a jinja fixture chunk under `HARD_MAX_CHARS`, paragraph packing behaves); `test_importer.py` (a directory walk with the new files indexes them; hidden dirs + exclusions still filter).
|
||||
- Integration: import over a temp tree with quadlet+j2 files → `documents` + `chunks` rows, delta re-import idempotent.
|
||||
- Coverage: **>90%** on `app/` — config/chunker/importer changes fully covered.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_quadlet_jinja_import.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A default-extensions import indexes `.container` / `.network` / `.volume` / `.image` / `.pod` / `.kube` / `.swap` / `.os` / `.endpoint` / `.j2` files (fixture-proven); no env configuration needed.
|
||||
- [ ] The Sources table lists them; the viewer shows a `.container` file's TOML content with the stem as title.
|
||||
- [ ] A question containing a `.j2` sentinel is non-deflected with the `.j2` doc as a source chip (A8 FTS-OR honesty gate).
|
||||
- [ ] `BOR_IMPORT_EXTENSIONS` still rejects truly unknown extensions (validator intact).
|
||||
- [ ] README + `.env.example` + PLAN.md A9 revision note record the extended set.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_quadlet_jinja_import.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_import_documents.py`, `test_sync_button.py`, `test_git_sources_admin.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked revision (2026-08-27, roadmap R1):** A9's format set is **revised** — ten formats added to the allowed + default set (the full quadlet family + `j2`); plain-text chunking (no TOML/Jinja-aware splitters); recorded as a PLAN.md A9 revision note with owner permission — a recorded revision, not a silent deviation (AGENTS.md rule 3).
|
||||
- **A9 invariants kept:** hidden (dot) directories still skipped; the exclusion list unchanged; narrow-only `BOR_IMPORT_EXTENSIONS` validator; sha256 delta / prune unchanged; `HARD_MAX_CHARS` (1200) honored by the `chunk_text` dispatch.
|
||||
- **A17 honoured** — one atomic commit.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 04 — Quadlet/jinja E2E + docs (README, PLAN A9 revision) + commit
|
||||
|
||||
**Phase:** `47_quadlet_jinja_import` · **Source:** `TODO.md:10–11` — "Add '.container', '.network', '.volume' and other quadlet files to the list of allowed/parsed files" / "Add '.j2' jinja files to the list of allowed/parsed files"
|
||||
**Story:** `.agent/user_stories/quadlet-jinja-import.md`
|
||||
|
||||
## Objective
|
||||
Prove the story end to end (import → catalog → Sources table → viewer → FTS retrieval), record the A9 revision in the docs, run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_quadlet_jinja_import.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule (the import happens out-of-band against the session mock, exactly like `test_import_documents.py`: truncate the KB, `import_sources([FIXTURES], LLMClient(settings))` in a thread — reuse that file's helpers/pattern; the task-02 fixtures are already in the tree):
|
||||
- `test_quadlet_and_jinja_indexed` — after the module import, `GET /api/docs` lists `quadlet/compose.container`, `quadlet/lan.network`, `quadlet/cache.volume`, `templates/deploy.j2`, each with a non-zero chunk count and the stem as title;
|
||||
- `test_sources_table_shows_them` — admin: `/sources.html` renders rows for the four files (path links present, `.doc-link`);
|
||||
- `test_container_content_viewable` — open `compose.container` from the Sources table (modal, phase 26): the content area contains the `[Container]` section text and the `RESE-QUADLET-SENTINEL-77aa` sentinel; the title is the stem (`compose`);
|
||||
- `test_jinja_retrievable_not_deflected` — ask a question containing `RESE-JINJA-SENTINEL-33dd` (the A8 gate: an FTS hit among the candidates keeps it honest-positive — LOW requires **zero** FTS hits): the brain bubble is **not** `.is-deflected` and a source chip names `templates/deploy.j2` (the mock's answer shape is deterministic; the assertion is on the gate + the chips, not the prose).
|
||||
- module fixture: truncate `documents`/`chunks`/`query_log` per test module (the house E2E pattern) and re-import — note: this file's re-import changes the KB for the session; it is run in **isolation** (A16), so no cross-suite interference.
|
||||
2. **Docs:**
|
||||
- `README.md` — wherever the import format list is documented (the "Import & update" section mirrors PLAN §11 / A9), extend it with the ten new formats (the 2026-08-27 A9 revision, plain-text chunking);
|
||||
- `.agent/PLAN.md` — **the only PLAN edit in this phase** (owner-locked revision R1): in the §2 anchors table, the A9 row's decision text gains the extension — append to the A9 row (keep the original wording, mark the revision in the row's notes/status or in a revision note under the table, the established style): "**A9 revision (phase 47, owner permission 2026-08-27):** the format set extends with the Podman quadlet family (`container, network, volume, image, pod, kube, swap, os, endpoint`) and `j2` (Jinja templates) — plain-text chunking (`chunk_text`), owner: `TODO.md` L10–L11. The narrow-only `BOR_IMPORT_EXTENSIONS` rule and the hidden-dir/exclusion invariants are unchanged." Update PLAN §5's chunking-policy format line and §11's workflow line to list the extended set (same note style). Touch **nothing else** in PLAN.md.
|
||||
3. Regression pass (isolation runs): `test_import_documents.py` (task 02's count-constant update must hold — the tree now has four more files), `test_sync_button.py`, `test_git_sources_admin.py`.
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(import): index quadlet unit files and jinja templates (A9 revision)`, staging this phase's files **including the force-added `.agent/PLAN.md`** (AGENTS.md rule 8) and the phase dir move `.agent/phases/todo/47_quadlet_jinja_import/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_quadlet_jinja_import.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (tasks 01–03 carry it).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The story E2E suite passes in isolation (all four tests); the three regression suites pass in isolation.
|
||||
- [ ] README + `.env.example` (task 01) + PLAN.md (A9 row + §5 + §11) record the extended set; no other PLAN change.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Phase 48 — Stop / Cancel an In-Flight Answer
|
||||
|
||||
**Source:** `TODO.md` L3 — "Need a way to stop or cancel generation of text in the chat"
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-29)
|
||||
**Context:** A chat turn is `POST /api/chat` → an SSE stream (`thinking` → `tool` → `delta` → `done`/`error`) produced by `app/api/chat.py`'s `stream()` over `app.rag.agent.run_agent` (grounded turns) or `LLMClient.chat_stream` (deflected turns — `app/rag/llm.py`, an `AsyncOpenAI` streaming request). The frontend consumes it in `frontend/assets/app.js` (`handleSend` → `fetch` → `readSSE`) with the never-stale state machine (`setUiState`: idle → thinking → streaming → done|error → idle, PLAN §7.4) and the phase-14 localStorage conversation (`bor.chat.v1`; optional per-record fields like `thinking`/`tools`/`stopped` are the no-version-bump convention).
|
||||
|
||||
## Objective
|
||||
The user can stop an in-flight answer at any time: while a turn is live the Send button becomes a **Stop** button; stopping keeps the partial answer on screen and in the persisted conversation (marked as stopped), settles the UI to idle with no error banner, and makes the server tear down the model's HTTP stream promptly so the local model stops generating.
|
||||
|
||||
## Dependencies
|
||||
- `14_chat_persistence` (complete) — the `bor.chat.v1` conversation records + save points; the stopped partial persists through the same helpers (new optional `stopped` field).
|
||||
- `17_thinking_display` + `20_sources_midstream_bug` (complete) — the thinking/tool frames + the "thinking-only turns persist nothing brain-side" convention the pre-token stop path follows.
|
||||
- `37_agent_document_tools` (complete) — the agent loop grounded turns run in; its per-round streams must tear down on abandon too.
|
||||
|
||||
## Tasks
|
||||
1. `01_llm_stream_teardown.md` — deterministic model-stream teardown on client disconnect + the cancelled-turn log line (no `query_log` row).
|
||||
2. `02_stop_button.md` — the Send↔Stop one-button morph, `AbortController` abort, partial keep + `stopped` persistence + restore note; CSS.
|
||||
3. `03_e2e_stop_generation.md` — the story Playwright suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_llm_stream_teardown.py` (the openai stream is `aclose`d on full consumption AND on mid-iteration abandon), `tests/unit/test_chat_cancel.py` (an SSE turn torn down mid-stream: fake LLM stream closed, cancel log line written, no `query_log` row, `error`/`done` paths unchanged) — both follow the fake/override patterns of `tests/unit/test_chat_gate.py` + `tests/fakes.py`.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
- Frontend source pins per the house pattern (`tests/unit/test_frontend_feedback.py` style): in-flight enabled + "Stop" label, `AbortController` signal, `stopped` record key, no-error stop path.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_stop_generation.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] While a turn is in flight the button reads **Stop** (rose treatment, ≥44px, focus-visible); clicking it (or pressing Enter) stops the turn.
|
||||
- [ ] A mid-stream stop keeps the partial text, shows a "Stopped" note, no error banner; the `bor.chat.v1` record carries `stopped: true`; a reload restores it with the note.
|
||||
- [ ] A pre-token stop leaves the question in the conversation, no brain bubble, no error banner.
|
||||
- [ ] The model's stream is closed promptly on abandon (unit-proven); a cancelled turn logs `cancelled=true` and writes no `query_log` row; completed/error turns log and record exactly as before.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_stop_generation.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_persistence.py`, `test_loading_feedback.py`, `test_chat_rag.py` (the loading-feedback suite's in-flight button assertions are revised in place to the new contract — see task 02).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A10 untouched** — `/api/chat` stays stateless; stopping is a client disconnect, no new endpoint, no server session state.
|
||||
- **Owner-locked (2026-08-29, roadmap confirmation):** (1) one-button morph — Send becomes Stop while in flight; click *or* Enter stops; (2) the partial answer is kept, persisted with the optional `stopped` marker, no sources, no error banner; a pre-token stop persists nothing brain-side (phase-20 convention); (3) the server closes the model stream on disconnect, logs `cancelled=true`, and skips `query_log` for cancelled turns.
|
||||
- **Revised contract (owner-locked 2026-08-29):** the in-flight button is the Stop control — enabled, labeled "Stop", spinner hidden (the typing dots / Thinking block / tool lines remain the in-flight feedback per the phase-06/17 contract); `tests/e2e/test_loading_feedback.py` assertions that pinned the old disabled-"Thinking…"-button + visible-spinner state are updated in place to the new contract (its 120s-guard + live-region pins stay).
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): stop an in-flight answer — Send becomes Stop, the partial is kept and persisted, the model stream is torn down"
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
# Task 01 — Deterministic model-stream teardown on client disconnect
|
||||
|
||||
**Phase:** `48_stop_generation` · **Source:** `TODO.md:3` — "Need a way to stop or cancel generation of text in the chat"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
When the SSE consumer goes away (client disconnect / user stop), the app closes the aipi HTTP stream promptly — the local model stops generating — and the turn settles observably: one cancelled-turn log line, no `query_log` row, no half-emitted `done`.
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py` — `LLMClient.chat_stream`: make the openai SDK stream's lifetime explicit. Invariants the restructure must keep:
|
||||
- a failure of the `create()` call itself wraps exactly as today (generic `except → LLMError` with the `chat stream from … failed` message);
|
||||
- a failure inside the `async for` wraps exactly as today (`LLMError` re-raised, everything else wrapped);
|
||||
- whenever `create()` succeeded, `await stream.aclose()` is awaited on **every** subsequent exit — normal exhaustion (a quiet no-op on an already-closed SDK stream, so the completed path stays byte-identical), exception paths, and GeneratorExit (consumer abandon — awaiting in the `finally` is safe because it does not yield).
|
||||
- Update the docstring: the stream is closed on consumer abandon (stop/cancel — 2026-08-29, `TODO.md` L3).
|
||||
2. `app/rag/agent.py` — `run_agent`: teardown must be deterministic, not GC-dependent. Per round, bind the stream to a variable (`stream = llm.chat_stream(...)`) and wrap the `async for piece in stream:` in `try/finally await stream.aclose()` — the same pattern for the round-cap's final `tools=None` call. The loop logic, the `holder` counters, and the `while True` structure are unchanged; a GeneratorExit raised into `yield piece` propagates only after the in-flight model stream is closed.
|
||||
3. `app/api/chat.py` — `stream()`: add one terminal flag, `settled = False`, set `True` at **every** terminal exit (immediately before the `done` event yields; every `return` that follows an `error` event; the embed-failure and retrieval-failure returns; the 503 pre-stream path never enters `stream()` so it needs no flag). Add a `finally` block to `stream()`: `if not settled: logger.warning("chat: turn cancelled question=%r total_ms=%d", request.message, int((time.monotonic() - started) * 1000))` — the §9-style per-turn cancel line (owner-locked: cancelled turns log `cancelled=true` and skip `query_log` — the step-4 `QueryLog` write sits after the stream loop and is simply never reached when the generator is closed). The `finally` must not yield.
|
||||
- Starlette closes the response-body generator (`aclose`) when the client disconnects; with tasks 1–2 in place the close chain is SSE generator → `run_agent`/`chat_stream` → aipi httpx response.
|
||||
4. `tests/unit/test_llm_stream_teardown.py` (new) — follow the fake patterns in `tests/fakes.py`:
|
||||
- a fake `AsyncOpenAI`-shaped client whose `chat.completions.create(stream=True)` returns a fake async stream that records `aclose()` calls and yields N pieces with small `asyncio.sleep`s between them;
|
||||
- (a) full consumption → `aclose` called exactly once;
|
||||
- (b) abandon after the first piece (`await gen.aclose()` on the `chat_stream` generator) → the fake stream's `aclose` was awaited before the generator's close completed.
|
||||
5. `tests/unit/test_chat_cancel.py` (new) — follow the fixture approach of `tests/unit/test_chat_gate.py` (fake LLM + DB):
|
||||
- drive `POST /api/chat` with a slow fake stream, read a few SSE frames, then drop the client mid-stream (close the client/connection the way the test harness permits — the SSE generator must receive `aclose`);
|
||||
- assert: the fake LLM stream was closed, the log capture contains the cancel line ("turn cancelled"), and **no** `query_log` row exists for the question;
|
||||
- regression pins: a completed turn still writes `query_log` + emits the `done` frame; the mid-stream `LLMError` path still emits the structured `error` frame and is **not** logged as cancelled (it settles).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (the new branches in llm/agent/chat covered by the new tests).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Abandoning `chat_stream` mid-iteration closes the openai stream (unit-proven with the recording fake).
|
||||
- [ ] `run_agent` closes the in-flight round's model stream when its consumer stops (unit-proven).
|
||||
- [ ] A cancelled turn: cancel log line, no `query_log` row, no `done`/`error` frame after the disconnect.
|
||||
- [ ] Completed/error turns behave byte-identically to before (existing `test_chat_gate.py` + friends green).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Task 02 — The Stop button (Send↔Stop morph) + partial keep
|
||||
|
||||
**Phase:** `48_stop_generation` · **Source:** `TODO.md:3` — "Need a way to stop or cancel generation of text in the chat"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
One-button control on the chat page: while a turn is in flight `#send-btn` becomes **Stop** (click or Enter aborts the turn); the partial answer is kept on screen and persisted (with the `stopped` marker), the UI settles to idle with a live-region confirmation and no error banner.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js`:
|
||||
- **Abort plumbing:** at the start of a turn create `const ac = new AbortController()` and store it in module scope (`let turnAbort = null`, reset per turn, cleared in the `finally`); pass `signal: ac.signal` to the `fetch("/api/chat", …)` call. The 120s guard keeps `cancelStream(res)` as the backstop and additionally calls `turnAbort?.abort()` — same outcome, one owner.
|
||||
- **`stopTurn()`:** new module function — a no-op unless `uiState` is thinking/streaming; sets a module-scope `stoppedByUser = true` (reset at turn start, next to `aborted`), then `turnAbort.abort()`. The abort makes the in-flight `fetch`/`await readSSE(...)` throw (`AbortError`) into `handleSend`'s `catch`, where the stop finalizes:
|
||||
- in `catch (err)`: if `stoppedByUser` (or `err?.name === "AbortError"`) → **stop path**: no `showErrorBanner`. When `wrap` exists and `acc` is non-empty: `closeThinkingBlock(wrap)`, `appendTuneButton(wrap)` (parity with the restore path — admin-only, anonymous gets nothing), `appendStoppedNote(wrap)`, then `rememberBrainTurn(acc, { thinking: thinkingAcc || undefined, tools: toolAcc.length ? toolAcc : undefined, stopped: true })` (owner-locked optional `stopped` marker — phase-14 convention, `STORAGE_VERSION` stays 1). When there is no answer text yet (pre-token / thinking-only stop): persist **nothing** brain-side (phase-20 convention — the question is already saved on send). Set `sendStatus.textContent = "Answer stopped."` before the `finally` settles `idle` (the existing `finally`'s `setUiState(idle)` + focus-back remains the single settle path — the stop path must not double-settle, and it never scrolls: no `scrollReveal` on stop, phase-42 contract).
|
||||
- **One-button morph in `setUiState`:** in-flight states (thinking/streaming) keep `#send-btn` **enabled** (it is the Stop button now — owner-locked), `sendLabel.textContent = "Stop"`, and `sendBtn.classList.toggle("is-stop", inFlight)`. The spinner is hidden while in flight (`sendBtn.querySelector(".spinner").hidden = inFlight` — the label alone reads "Stop"; the CSS treatment carries the state). Idle/error: existing behavior (label "Send", class removed, spinner hidden as today).
|
||||
- **Tool frames:** the phase-37 `tool` branch no longer writes `sendLabel.textContent = "Calling tool…"` — the button stays "Stop" while in flight (owner-locked); the "calling tool" status stays exactly where it is today in `#send-status` + the typing indicator's `aria-label`.
|
||||
- **Submit-while-in-flight:** `handleSend`'s first guard becomes `if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) { stopTurn(); return; }` (the button is enabled while in flight, so both the click and Enter-to-submit land here; the `!text` guard for idle stays).
|
||||
- **Restore:** `renderStoredMessage` — when `m.stopped` is true, call the same `appendStoppedNote(wrap)` helper. New small helper: creates/reuses the `.msg-meta` row exactly like `appendTuneButton` does (including the `role=list` → button/`span` `role=listitem` rule) and appends a `.stopped-note` span — an inline stop-glyph SVG (a small filled square, `aria-hidden="true"`) + the text "Stopped" (the text carries the accessible meaning).
|
||||
- **Header comment:** update the file's top comment — the loading-feedback machine gains a user-stop terminal (stop → idle, no banner) and the button's dual role (2026-08-29, `TODO.md` L3).
|
||||
2. `frontend/assets/styles.css`:
|
||||
- `.send-btn.is-stop` — the stop treatment in the phase-08 dark-tech palette: a rose-family background derived from the brand rose `#f43f5e` but darkened so the near-white label keeps ≥4.5:1 (check the contrast — e.g. a `#be123c`-range token with `#fff` label), the same border-radius/height/focus-visible ring as `.send-btn`, a darker hover step, hit target ≥44px unchanged.
|
||||
- `.stopped-note` — the meta-row note: muted ink-soft color (≥4.5:1 on the bubble surface), 10–12px glyph baseline-aligned with the Tune button, non-interactive (no hover/focus).
|
||||
3. `frontend/index.html` — no markup change (`#send-btn` already carries spinner + `#send-label`); update the composer comment block to document the Send↔Stop morph + the stopped-note meta row.
|
||||
4. **Revise the old contract in place** — `tests/e2e/test_loading_feedback.py`: its pins of the in-flight button (disabled, "Thinking…" label, visible spinner) are updated to the new contract (enabled "Stop" button, `is-stop` class, spinner hidden); its 120s-guard, live-region, and typing-dots pins stay intact. The unit pins in `tests/unit/test_frontend_feedback.py` that assert the same old button state are updated in place the same way (they live with the `SEND_STATUS`/`setUiState` pins — keep the file's structure).
|
||||
5. Frontend source pins (house pattern, `tests/unit/test_frontend_feedback.py` style — extend that file): the in-flight `is-stop` toggle + enabled button + "Stop" label; the `AbortController` + `signal` in the fetch; the `stoppedByUser` stop branch (no `showErrorBanner`); the `stopped: true` key in the persisted record; the `appendStoppedNote` restore path; the tool branch no longer writing `sendLabel`.
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): one-button morph — click or Enter while in flight stops the turn.
|
||||
- ASSUMPTION (owner-locked 2026-08-29): the partial answer is kept, persisted with the optional `stopped` marker (no sources/suggestions), "Stopped" note, no error banner; a pre-token stop persists nothing brain-side; the live stopped bubble gets the Tune button for admin (parity with the restore path).
|
||||
- ASSUMPTION (owner-locked 2026-08-29): the server side of a stop is the task-01 teardown — this task never calls a cancel endpoint (there is none — A10).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: source pins as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged by this frontend task — the phase's app coverage came from task 01).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] In flight: button enabled, label "Stop", `is-stop` class; idle: "Send" (state machine otherwise unchanged — the revised `test_loading_feedback.py` green).
|
||||
- [ ] Click/Enter while in flight aborts; the partial is kept + persisted (`stopped: true`); no error banner; the live region confirms "Answer stopped.".
|
||||
- [ ] A reload restores the stopped answer with the Stopped note (unit-pinned now, E2E in task 03).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Task 03 — Stop E2E + regressions + commit
|
||||
|
||||
**Phase:** `48_stop_generation` · **Source:** `TODO.md:3` — "Need a way to stop or cancel generation of text in the chat"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove the stop contract in the browser (mid-stream stop, pre-token stop, persistence of the stopped turn), run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_stop_generation.py` (new) — mock-only, DB up (conftest `page` fixture; no admin login needed — chat is public):
|
||||
- `test_stop_mid_stream` — ask an on-topic question carrying `LONG_ANSWER_TRIGGER` ("write a long answer …" — reuse the on-topic phrasing `test_long_answers.py` uses so the honesty gate is HIGH); wait for the brain bubble to exist and its text to grow past a few words (first deltas); assert in flight: `#send-btn` enabled, label "Stop", class `is-stop`; click it; assert: label back to "Send" and `is-stop` gone; **no** error banner (`#kb-banner` without `.is-error`/`role="alert"`); the bubble keeps the partial text and shows a `.stopped-note` "Stopped"; the text is stable (re-read after ~1.5 s — no growth) and shorter than the mock's full long answer (`LONG-ANSWER-END` absent from the bubble); `bor.chat.v1`'s last brain record has `stopped === true` and a `text` lacking `LONG-ANSWER-END`.
|
||||
- `test_stop_pre_token` — ask an on-topic question containing "pretend to think slowly" (the mock's 3s warm-up); while the button reads "Stop" (thinking state) click it; assert: no error banner, **no** brain bubble in `#messages`, `bor.chat.v1`'s last record is the user's question (`who: "user"`), button "Send" + enabled, input focused.
|
||||
- `test_stopped_turn_survives_reload` — the `test_stop_mid_stream` flow, then `page.reload()`; the stopped answer restores with the `.stopped-note`, the conversation order is intact (user, then the stopped brain bubble), the button is idle "Send".
|
||||
- Determinism notes: the mock streams 12 chars / 0.02 s, so the long answer takes ~8 s — a comfortable stop window; wait for observable states, no fixed sleeps beyond those.
|
||||
2. Regression pass (isolation runs): `test_chat_persistence.py` (save/restore + New chat — the button and state machine changed), `test_loading_feedback.py` (the revised contract — 120s guard + live regions + typing dots), `test_chat_rag.py` (a normal turn still streams to `done` with sources + Tune).
|
||||
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section — staging this phase's files; move `.agent/phases/todo/48_stop_generation/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_stop_generation.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All three story tests pass in isolation; the three regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Phase 49 — Retry the Last Answer (Redo)
|
||||
|
||||
**Source:** `TODO.md` L4 — "Need a retry button to retry the last answer, like a redo button"
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-29)
|
||||
**Context:** `frontend/assets/app.js` — `handleSend` owns the whole turn (user-bubble append + conversation push → fetch/SSE → save points), `appendTuneButton` is the house pattern for a per-bubble meta-row action (reuses the `.msg-meta` row, the `role=list` → `role=listitem` rule, the admin gate), `conversation`/`rememberBrainTurn` hold the persisted records, and phase 48's Stop button means the last brain bubble can be a **stopped partial** — the natural retry candidate.
|
||||
|
||||
## Objective
|
||||
A **Retry** button on the last brain answer re-asks the preceding question in place — the old answer is removed (DOM + persisted record), the fresh answer streams into its place, and the conversation never duplicates the question.
|
||||
|
||||
## Dependencies
|
||||
- `48_stop_generation` (todo) — the button/state-machine work it builds on (the meta-row button family, the in-flight guard) and the stopped partials it makes retryable.
|
||||
- `14_chat_persistence` (complete) — the conversation records the retry edits in place.
|
||||
|
||||
## Tasks
|
||||
1. `01_retry_button.md` — the `runTurn` extraction + the Retry button + redo-in-place semantics + CSS.
|
||||
2. `02_e2e_retry_answer.md` — the story Playwright suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Frontend source pins (house pattern): the `runTurn(text, {reask})` extraction (reask skips the user append/push), last-bubble-only Retry management, the redo-in-place record edit, the in-flight no-op.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_retry_answer.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The last brain bubble (completed, deflected, empty-fallback, or stopped partial — admin and anonymous alike) carries a Retry button; earlier brain bubbles do not.
|
||||
- [ ] Clicking Retry removes the old answer (DOM + `bor.chat.v1` record), re-sends the preceding question without duplicating it, and streams the fresh answer into its place; the new answer carries the Retry button (it is the new last).
|
||||
- [ ] While a turn is in flight, Retry does nothing (no double turn).
|
||||
- [ ] `uv run pytest` green; coverage TOTAL unchanged (>90%).
|
||||
- [ ] `uv run pytest tests/e2e/test_retry_answer.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_chat_persistence.py`, `test_stop_generation.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked (2026-08-29, roadmap confirmation):** redo-in-place (the old answer is replaced, the question is not duplicated); only the **last** brain bubble is retryable; Retry is available to **all** visitors (chat is public — unlike Tune, which is admin-only); inert while a turn is in flight.
|
||||
- **A10 untouched** — retry is a fresh `POST /api/chat` (the API stays stateless).
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): retry the last answer — redo-in-place Retry button on the latest brain bubble"
|
||||
```
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 01 — The Retry button (redo in place)
|
||||
|
||||
**Phase:** `49_retry_answer` · **Source:** `TODO.md:4` — "Need a retry button to retry the last answer, like a redo button"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
`handleSend` is split so a question can be re-run without re-adding it, and a Retry button on the last brain bubble performs that redo: old answer gone (DOM + storage), fresh answer streaming into its place.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js`:
|
||||
- **Extract `runTurn(text, { reask = false })`** from `handleSend`: everything from the `setUiState(thinking)` / `armTurnTimeout` point through the `finally` settle moves into `runTurn`; the `reask` flag skips (a) the `addMessage("user", …)` append and (b) the `conversation.push({ who: "user", … })` + `saveConversation()` (the question is already in both). `handleSend` keeps the form-level pre-work — the `!text` guard, the **in-flight guard that calls `stopTurn()`** (phase 48), `clearErrorBanner()`, the input clear + autoGrow — then appends + persists the user message and calls `runTurn(text, { reask })`. The turn-local resets (`acc`, `thinkingAcc`, `sawThinking`, `sawDone`, `toolAcc`, `stoppedByUser`, `turnAbort`) stay turn-scoped exactly as phase 48 left them. **No behavior drift for the normal send path**: a plain send must produce byte-identical DOM/SSE/persistence behavior to today (the regression suite is the proof).
|
||||
- **`appendRetryButton(wrap)`** — the house `appendTuneButton` pattern for the meta row (reuse the `.msg-meta` row; `role=list` → `role=listitem` where the row is a list; one per bubble; an inline redo-glyph SVG + "Retry" text — the text is the accessible name). **Not** admin-gated (owner-locked: all visitors). Click handler `retryLastTurn(wrap)`:
|
||||
- in-flight guard: if `uiState` is thinking/streaming → no-op (owner-locked).
|
||||
- find the record: the last brain record in `conversation` (and that `wrap` is the rendered wrap of that record — the button only ever sits on the last bubble, but the guard keeps a stale click harmless).
|
||||
- **redo in place:** pop the brain record from `conversation`; `saveConversation()` immediately (a crash between the pop and the fresh `done` must never resurrect the replaced answer — what the user saw, the removed answer, is what is stored; the question remains); remove `wrap` from the DOM; locate the preceding **user** record's `text` (the record immediately before the popped one — invariant: every brain record follows its user record); call `runTurn(text, { reask: true })`. No `sendStatus` banner; no scroll (phase-42 contract — no auto-scroll; the fresh bubble lands where the old one was).
|
||||
- **Last-bubble-only management:** a helper `markLastRetryable()` — removes any existing `.retry-btn` from every rendered `.msg-meta`, then appends the Retry button to the last brain bubble (only when it has a preceding user record — always true in practice). Call sites: on `done` (after `appendTuneButton`), on the empty-answer fallback path, on the **stop finalize** path (phase 48 — the stopped partial is the prime retry candidate), and once at the end of `restoreConversation()` (after all records are rendered). `startNewChat`'s list reset removes everything anyway — no change there.
|
||||
- **Header comment:** note the Retry contract (2026-08-29, `TODO.md` L4).
|
||||
2. `frontend/assets/styles.css` — `.retry-btn`: the exact visual family of `.tune-btn` (same size/spacing/focus-visible/hover; ≥44px comfortable via the meta-row padding, as Tune has) with the redo glyph in the phase-08 palette (ink-soft → ink on hover), so the two meta actions read as a pair.
|
||||
3. `frontend/index.html` — no markup change (the button is JS-injected like Tune); update the messages-section comment to mention the meta-row actions (Tune — admin; Retry — everyone).
|
||||
4. Frontend source pins (house pattern, extend `tests/unit/test_frontend_feedback.py` or a sibling): the `runTurn` signature + the `reask` skips (no user append/push when reask); `appendRetryButton`'s no-admin-gate + one-per-bubble; `markLastRetryable`'s remove-then-append; the pop → save → rerun order in `retryLastTurn`; the in-flight no-op.
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): redo-in-place — the old answer is replaced (DOM + persisted record), the question is re-sent without duplication; only the last brain bubble carries the button; available to all visitors; inert while a turn is in flight.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: source pins as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The plain-send path is byte-identical to pre-task behavior (regression: `test_chat_rag.py` E2E green in isolation).
|
||||
- [ ] Retry on a completed / deflected / stopped answer redoes in place (unit-pinned now, E2E in task 02).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Task 02 — Retry E2E + regressions + commit
|
||||
|
||||
**Phase:** `49_retry_answer` · **Source:** `TODO.md:4` — "Need a retry button to retry the last answer, like a redo button"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove the redo contract in the browser, run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_retry_answer.py` (new) — mock-only, DB up (conftest `page`; no login — chat is public):
|
||||
- `test_retry_redoes_in_place` — ask an on-topic question (a `test_chat_rag.py`-style grounded question); wait for `done` (the source chips / Tune row appear — the turn is complete); ask a second on-topic question; assert: the **last** brain bubble carries a Retry button and the **first** brain bubble does not; click Retry on the last; assert: the old answer's text is gone from `#messages`; the retried question appears exactly **once** in the rendered list; a fresh answer is streaming (button "Stop"), then settles to the new answer; `bor.chat.v1` shape is `[u1, b1, u2, b2']` — the retried question occurs once and the last brain record is the fresh answer; the new bubble carries the Retry button.
|
||||
- `test_retry_on_stopped_partial` — the phase-48 stop flow (a long-answer question, mid-stream stop → `.stopped-note`); the stopped bubble carries Retry; click it; the turn re-runs to a fresh `done` answer replacing the partial; the stored partial is gone (the last brain record has no `stopped` flag and carries the full answer).
|
||||
- `test_retry_deflected` — an off-topic question that deflects (a `test_honest_deflection.py`-style question); the deflected bubble (with its Maybe-try chips) carries Retry; clicking it re-asks (still deflected is fine — assert the redo mechanics: old bubble gone, fresh deflected bubble in its place, the chips re-rendered).
|
||||
- `test_retry_inert_while_in_flight` — seed one completed turn (so a last bubble exists), then ask a "pretend to think slowly …" question; while in flight (button "Stop"), click the Retry button on the previous last bubble; assert nothing happens: the in-flight turn still completes to its own `done`, and after settle there is exactly one user record for the in-flight question (no second turn started, no bubble duplication).
|
||||
2. Regression pass (isolation runs): `test_chat_rag.py`, `test_chat_persistence.py` (the record shape is asserted there), `test_stop_generation.py` (phase 48 — the stop finalize path now also calls `markLastRetryable`).
|
||||
3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
|
||||
4. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section — staging this phase's files; move `.agent/phases/todo/49_retry_answer/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_retry_answer.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/` (unchanged).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All four story tests pass in isolation; the three regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Phase 50 — Save & View Chat History
|
||||
|
||||
**Source:** `TODO.md` L5 — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-29)
|
||||
**Context:** A10 (revised, phase 16) keeps the public **chat** API stateless, and phase 14 locked conversation persistence to browser-local `localStorage` (`bor.chat.v1`). This phase records an **owner-locked extension** (AGENTS.md rule 3, owner permission 2026-08-29): conversations the owner explicitly **Saves** are stored in a new Postgres `saved_chats` table — `/api/chat` itself stays stateless, and nothing is stored about a conversation that was not saved. The admin gate is `app/core/auth.require_admin` (phase 16, the `steering` router's pattern). The localStorage record shape (`{who, text, sources?, deflected?, suggestions?, thinking?, tools?, stopped?}`) is the stored `messages` payload, so a saved chat restores pixel-identical through the existing `renderStoredMessage` path. The new page follows the phase-34 shared-header contract (every page carries the identical nav block; admin-only links ship hidden and `header.js` reveals them) and AGENTS.md rule 5 (a **full-width table** — no skinny wasted-space list).
|
||||
|
||||
## Objective
|
||||
The owner can Save the current conversation, see every saved chat on a new **History** page (full-width table), click one to return to the chat with that conversation loaded, and delete a saved chat.
|
||||
|
||||
## Dependencies
|
||||
- `48_stop_generation` / `49_retry_answer` (todo, sequential) — no shared-file conflicts beyond `app.js`; ordering keeps the chat UI stable while the save/load plumbing lands.
|
||||
- `14_chat_persistence` (complete) — the record shape + the `renderStoredMessage` restore a saved chat reuses.
|
||||
- `16_admin_auth` + `34_consistent_navbar` (complete) — the `require_admin` gate + the one-bar nav contract.
|
||||
- **Owner permission (2026-08-29):** the A10 extension — a new `saved_chats` table for explicitly saved conversations (see Context).
|
||||
|
||||
## Tasks
|
||||
1. `01_saved_chat_model.md` — the `SavedChat` model + migration `0008_saved_chats` (+ migration test).
|
||||
2. `02_chats_api.md` — admin-only CRUD under `/api/chats` + schemas + integration tests.
|
||||
3. `03_save_chat_ui.md` — the chat page: Save button, `?chat=<id>` load, upsert semantics, live-region feedback, absent for anonymous.
|
||||
4. `04_history_page.md` — `history.html` + `history.js` (full-width table, Open + two-step Delete), the `#nav-history` admin-only nav link on every page, the cache-busting page registration, CSS.
|
||||
5. `05_e2e_chat_history.md` — the story Playwright suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: `tests/integration/test_chats_api.py` (the CRUD contract: 403 anonymous, create auto-title, list order, get, put replacement, delete 404/204, message-shape validation).
|
||||
- Integration: `tests/integration/test_migration_0008.py` (the house migration-test pattern from `test_migration_0007.py`).
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_chat_history.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin: Save on the chat page stores the conversation (auto-title = first question, 120-char cap); re-Save on the same conversation updates the same row; New chat unlinks.
|
||||
- [ ] `/history.html` (admin) lists saved chats in a **full-width** table (Title, Messages, Updated, Actions); a row's title opens `/?chat=<id>` and the chat renders the stored conversation (sources, thinking, stopped notes, deflection chips — pixel-identical to the local restore); a subsequent Save updates that row.
|
||||
- [ ] Delete removes the row (inline two-step confirm, no `window.confirm`); an unknown id 404s; `GET /api/chats` + the served `/history.html` carry the cache-busting contract (no-cache + `?v=` rewrite).
|
||||
- [ ] Anonymous: no Save button, no History nav link, `/api/chats*` → 403, `/history.html` shows the gated state without fetching `/api/chats`.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_persistence.py`, `test_nav_consistency.py`, `test_shared_header.py`, `test_admin_auth.py`, `test_cache_busting.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked extension (2026-08-29, recorded per AGENTS.md rule 3):** `saved_chats` in Postgres stores **only** conversations the owner explicitly saves; `/api/chat` stays stateless; phase 14's local persistence is unchanged (the localStorage session keeps working exactly as before — saving is an additional, explicit action).
|
||||
- **Owner-locked (2026-08-29):** Save/History is **admin-only** (no account system — anonymous rows would be unfindable); absent-not-hidden for anonymous (phase 16); auto-title, no rename UI in v1 (the schema still accepts an optional `title`); re-Save = upsert of the same row; `?chat=<id>` replaces the local conversation and links it; inline two-step delete confirm (no `window.confirm`).
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ alembic/versions/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): save and view chat history — admin-only saved_chats, History page, open-a-chat return"
|
||||
```
|
||||
@@ -0,0 +1,26 @@
|
||||
# Task 01 — The `SavedChat` model + migration
|
||||
|
||||
**Phase:** `50_chat_history` · **Source:** `TODO.md:5` — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
A `saved_chats` table — one row per explicitly saved conversation, the messages stored as the exact localStorage record shape — with migration `0008`.
|
||||
|
||||
## Work
|
||||
1. `app/models.py` — a new `SavedChat` model (the module docstring's data-model list gains the entry, in the house style of the phase-15/30/35 entries):
|
||||
- `id: Mapped[uuid.UUID]` — PK, `default=uuid.uuid4`
|
||||
- `title: Mapped[str]` — `String(500)` — set by the API (auto-title; the column is plain so a future rename needs no migration)
|
||||
- `messages: Mapped[list]` — `postgresql.JSONB` — `list[dict]` in the `bor.chat.v1` record shape (`{who: "user"|"brain", text, sources?, deflected?, suggestions?, thinking?, tools?, stopped?}` — raw text, never HTML, phase 14); the API always supplies a list, so no default is needed
|
||||
- `created_at` / `updated_at: Mapped[datetime]` — `DateTime(timezone=True)`, `server_default=func.now()`; `updated_at` additionally carries `onupdate=func.now()`
|
||||
- No share-related columns (phase 51 adds `share_token` in `0009` — this migration stays minimal).
|
||||
2. `alembic/versions/0008_saved_chats.py` — `revision = "0008"`, `down_revision = "0007"` (verify the head first with `uv run alembic heads`): `op.create_table("saved_chats", …)` (UUID via `sqlalchemy.dialects.postgresql.UUID(as_uuid=True)`, JSONB via `sqlalchemy.dialects.postgresql.JSONB`) and the down `op.drop_table`.
|
||||
3. `tests/integration/test_migration_0008.py` (new) — the house pattern from `tests/integration/test_migration_0007.py` (same DB/fixtures it uses; assert the table + columns exist at head, the `downgrade`/`upgrade` round-trip it exercises, and — where that pattern allows — the `updated_at` bump on row update).
|
||||
4. Apply the migration to the dev/e2e DB: `uv run alembic upgrade head` (the E2E conftest's default `BOR_DATABASE_URL` is the same Postgres the dev server uses — it is already up).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the migration test above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (model-only — no new logic).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run alembic upgrade head` applies cleanly (and the migration test's downgrade/upgrade round-trip passes).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 02 — Admin-only `/api/chats` CRUD
|
||||
|
||||
**Phase:** `50_chat_history` · **Source:** `TODO.md:5` — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
The saved-chat CRUD contract under `/api/chats`, admin-gated router-wide (the `steering` pattern), with a strict-enough message schema that a corrupted or HTML-shaped payload cannot poison a restored conversation.
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py` — new Pydantic models (the house style of the steering/git-source schemas):
|
||||
- `ToolCall` — `name: str`, `argument: str | None` (the `tools` record shape, phase 37).
|
||||
- `ChatMessage` — `who: Literal["user", "brain"]`, `text: str` (`min_length=1`), optional: `sources: list[SourceRef] | None`, `deflected: bool | None`, `suggestions: list[str] | None`, `thinking: str | None`, `tools: list[ToolCall] | None`, `stopped: bool | None` — `model_config = ConfigDict(extra="forbid")` so unknown keys (e.g. an HTML-shaped payload) are rejected at the boundary.
|
||||
- `SavedChatCreate` — `title: str | None` (`max_length=500`), `messages: list[ChatMessage]` (`min_length=1`).
|
||||
- `SavedChatUpdate` — `title: str | None`, `messages: list[ChatMessage]` (`min_length=1`).
|
||||
- `SavedChatOut` — `id: uuid.UUID`, `title: str`, `created_at`, `updated_at`, `message_count: int`, `messages: list[ChatMessage]`.
|
||||
- `SavedChatRow` — `id`, `title`, `updated_at`, `message_count` (the list page's row shape — no payloads in the list).
|
||||
- `SavedChatList` — `chats: list[SavedChatRow]`.
|
||||
2. `app/api/chats.py` (new) — `router = APIRouter(prefix="/chats", tags=["chats"], dependencies=[Depends(require_admin)])` (the phase-16 pattern; the module docstring documents the gate the way `steering.py` does, and records the A10 extension + owner permission 2026-08-29 — the "recorded revision, not a silent deviation" house style):
|
||||
- `GET ""` → `SavedChatList` — rows ordered `updated_at desc, id desc` (latest activity first).
|
||||
- `POST ""` (201) → `SavedChatOut` — the auto-title when `title` is absent/blank: the **first user message**'s text, whitespace-collapsed, truncated to 120 chars (owner-locked convention); a conversation with no user message (defensive — the UI cannot produce one) falls back to `"Chat <id-hex8>"`.
|
||||
- `GET "/{chat_id}"` → `SavedChatOut` — 404 `{"detail": "unknown chat"}` on an unknown id.
|
||||
- `PUT "/{chat_id}"` → `SavedChatOut` — full `messages` replacement; `title` replaced only when supplied (an absent `title` keeps the current one); 404 on unknown. (`updated_at` bumps via the model's `onupdate` — verify the ORM flush triggers it; if not, set `row.updated_at` explicitly in the route.)
|
||||
- `DELETE "/{chat_id}"` → 204 — 404 on unknown.
|
||||
- `db: Session = Depends(get_db)` throughout (the `noqa: B008` house style).
|
||||
3. `app/main.py` — `app.include_router(chats_router, prefix="/api")` with the other API routers (before the static mount — the block's existing order).
|
||||
4. `tests/integration/test_chats_api.py` (new) — the house pattern from `tests/integration/test_steering_api.py` (TestClient + DB fixtures as that file does it):
|
||||
- anonymous: every route 403 (list/create/get/put/delete);
|
||||
- admin (signed in via the same auth-helper pattern that file uses): create (auto-title from the first user message + the 120-char truncation; an explicit title honored; an empty `messages` list → 422; a `who: "alien"` → 422; an extra key on a message → 422), list order (a second, newer chat first), get (full payload round-trip — a brain record carrying `sources`/`thinking`/`tools`/`stopped` survives byte-identical), put (replacement + title-keep + title-set + the `updated_at` bump), delete (204 then get 404; delete unknown 404).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (the new module fully covered — 404/422/403 branches included).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The contract holds end-to-end: 403 anonymous, 201 create, 200 list/get/put, 204 delete, 404 unknown, 422 malformed.
|
||||
- [ ] A `bor.chat.v1`-shaped payload round-trips losslessly (the restore path is pixel-identical by construction).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 03 — Save button + `?chat=<id>` load on the chat page
|
||||
|
||||
**Phase:** `50_chat_history` · **Source:** `TODO.md:5` — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
The chat page gains the Save action (admin-only) and can boot into a saved chat through `/?chat=<id>` — opening a chat from History returns the owner to that exact conversation, with subsequent Saves updating the same row.
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` — beside `#new-chat-btn` (the same `.chat-shell` block, the same ghost-pill family), `#save-chat-btn`: an inline save-glyph SVG + the label "Save" (`aria-label="Save chat"`); **ships `hidden`** (the ship-hidden / reveal-for-admin contract — `app.js` reveals it only when `isAdmin`; `hidden` is display:none, so anonymous visitors see no trace); a comment block documents the upsert + `?chat=<id>` contract (2026-08-29, `TODO.md` L5).
|
||||
2. `frontend/assets/app.js`:
|
||||
- **`currentChatId`** (module scope, `string | null`): set from `?chat=<id>` at boot, set to the created row's id on a fresh Save, cleared by `startNewChat`.
|
||||
- **Boot-load** (inside the existing boot IIFE, after `fetchIsAdmin()`): when the URL's `?chat=` value is a valid uuid **and** `isAdmin`: `GET /api/chats/<id>` → on 200: `conversation = data.messages` (the records already match the local shape), render through the existing `renderStoredMessage` loop (sources / thinking / tools / stopped / deflection — pixel-identical to the local restore), `currentChatId = id`, then `saveConversation()` (the local session now mirrors the opened chat, so a plain refresh returns to it the phase-14 way) — and **skip** the localStorage restore for this load. On 404/network failure: `showErrorBanner("That saved chat isn't available — it may have been deleted.")` and fall through to the normal local restore. An invalid/absent param, or anonymous: the normal local restore runs (no fetch — the gate would 403).
|
||||
- **`saveCurrentChat()`** — the `#save-chat-btn` handler: a no-op with the live-region line "Nothing to save yet." when `conversation` is empty. Otherwise: if `currentChatId` is set → `PUT /api/chats/<id>` with `{ messages: conversation }`; else `POST /api/chats` with `{ messages: conversation }` (the server auto-titles) → `currentChatId = created.id`. On 200/201: `sendStatus.textContent = "Conversation saved."` (the live region — the never-stale contract; status text only, no banner). On 404 from the PUT: unlink (`currentChatId = null`), retry as a create, and announce the outcome — the owner is never left with an unsaved conversation because of a stale link. On 403/5xx/network: `showErrorBanner` with an actionable line.
|
||||
- **`startNewChat`** — add `currentChatId = null` to the existing reset (a new conversation is unlinked until saved again).
|
||||
- **Reveal on boot:** next to `applyAuthState()`: `saveBtn.hidden = !isAdmin` (phase 16 absent-not-hidden for anonymous).
|
||||
- **Header comment:** the save/load contract (2026-08-29, `TODO.md` L5).
|
||||
3. `frontend/assets/styles.css` — `.save-chat-btn`: the exact visual family of `.new-chat-btn` (ghost pill, ≥44px, focus-visible, hover like `.nav-link`), so the two chat-shell actions read as a pair.
|
||||
4. Frontend source pins (house pattern, extend `tests/unit/test_frontend_feedback.py` or a sibling): the `currentChatId` lifecycle (set on create/open, cleared on New chat, cleared on the 404-PUT fallback); the upsert branch (PUT when linked, POST when not, the 404→recreate fallback); the boot-load precedence (valid `?chat=` + admin replaces the local restore and mirrors it to storage; anonymous/invalid/404 → local restore); the `hidden` reveal gate.
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): Save/History is admin-only; absent-not-hidden for anonymous; re-Save updates the same row; `?chat=<id>` replaces the local conversation and links it; auto-title, no rename UI in v1.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: source pins as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only task).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin sees the Save button; anonymous doesn't (display-none — no trace in the layout).
|
||||
- [ ] Save → a row exists (integration-proven in task 02's API + E2E in task 05); re-Save updates it; New chat unlinks.
|
||||
- [ ] `/?chat=<id>` restores the saved conversation pixel-identically; a bad/deleted id degrades to the local restore with a banner (E2E in task 05).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 04 — The History page + nav link + cache-busting registration
|
||||
|
||||
**Phase:** `50_chat_history` · **Source:** `TODO.md:5` — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
A new `/history.html` page (admin data) listing saved chats in a **full-width table** (AGENTS.md rule 5 — no skinny list), with Open (return to the chat) and Delete (inline two-step confirm) per row, the `#nav-history` admin-only nav link on every page (the phase-34 one-bar contract), and the page registered in the cache-busting middleware.
|
||||
|
||||
## Work
|
||||
1. `frontend/history.html` (new) — the standard page scaffold exactly as `sources.html`/`tuning.html` do it: the head (charset, viewport, `assets/styles.css`), the skip link, the **identical** `.app-header` block copied from the other pages (brand, the `#nav-toggle` hamburger, the nav with the existing admin-only hidden links **plus** the new `<a href="/history.html" class="nav-link is-active" id="nav-history" hidden aria-current="page">History</a>` — placed after the Tuning link, the same ship-hidden contract, with a phase comment), the auth pair (the same Sign in/out markup with `?next=/history.html`), `<main id="main">` carrying the `#steering-panel` section (the shared panel ships on every page — phase 34) + `#steering-announcer`, then the page content: an `<h1>` "Saved chats" + sub-line, the **full-width table** skeleton (`<table class="history-table">` with a `<thead>`: Title | Messages | Updated | Actions (visually-hidden header text)), an empty-state row ("No saved chats yet — finish a conversation and press **Save** in the chat."), a `role="status"` live region for action feedback, and the app-footer (the version span, like the other pages). Scripts: `<script src="assets/brand.js"></script>` (classic, first) + `<script type="module" src="/assets/history.js"></script>`. **No-CDN rule** (AGENTS.md rule 6): every asset local.
|
||||
2. `frontend/assets/history.js` (new) — a module in the `sources.js`/`tuning.js` house style:
|
||||
- boot: `await initSharedHeader()` (header.js — whoami + nav reveal + the steering panel), then `fetchIsAdmin()`; **anonymous**: render the page's gated state following the `/sources.html` soft-gate pattern (the table area shows the gated/empty message; **no data fetch** — `/api/chats` is 403 for anonymous and must never be called; pin this in the E2E via the request log).
|
||||
- admin: `GET /api/chats` → render the rows: **Title** as an `<a href="/?chat=<id>">` (Open is the title link — the "return to that history with a click" requirement), **Messages** (`message_count`), **Updated** (`updated_at` as a locale date+time, `title` attribute with the full ISO), **Actions**: **Delete** only (phase 51 adds the share column). Delete is an **inline two-step** (owner-locked: no `window.confirm` anywhere in the file): the first click turns the button into a small "Delete? [Yes] [No]" confirm pair (keyboard-accessible, focus moves to Yes); Yes → `DELETE /api/chats/<id>` → the row is removed + the live region `"Deleted "<title>"."`; a No or a failed request keeps the row (+ an error line on failure). A 0-row fetch shows the empty-state row.
|
||||
- The table is **full-width**: `width: 100%` inside the standard `.container` (AGENTS.md rule 5 — no fixed skinny width).
|
||||
3. **The nav link on every page** — add the identical `#nav-history` hidden link (after `#nav-tuning`, same markup + phase comment) to the nav block of `index.html`, `sources.html`, `git-sources.html`, `tuning.html`, `document.html`, `login.html` (the phase-34 one-bar contract — all pages), and in `frontend/assets/header.js` reveal it for admin exactly like `#nav-tuning` (the same `const navHistory = document.querySelector("#nav-history"); if (navHistory) navHistory.hidden = !admin;` block + comment). `history.html` itself carries the link with `is-active` + `aria-current="page"` (step 1).
|
||||
4. `app/core/caching.py` — add `"/history.html"` to the `HTML_PAGES` tuple (with a phase-50 comment) so the new page gets the no-cache + `?v=` asset-rewrite contract like the others; update the docstring's page count wording if it names the five. `tests/unit/test_caching.py` — a pin that `HTML_PAGES` includes `/history.html` (extend the existing pins' style).
|
||||
5. `frontend/assets/styles.css` — `.history-table`: the full-width table in the phase-08 dark-tech palette (the sources-table family — borders via the `--line` token, the header row on the surface-darker token, row hover, ≥4.5:1 ink colors, `th scope="col"` headers); the title link styled as an accent link (focus-visible); `.history-confirm` inline pair (Yes on the error-rose treatment, No ghost); the empty-state row (muted centered message); the ≤640px responsive behavior (actions wrap; the table keeps full width — the phase-07 responsive contract).
|
||||
6. Frontend source pins (house pattern): the anonymous no-fetch gate; the two-step confirm (and a repo-wide `window.confirm` absence pin for `history.js`); the `/?chat=<id>` href shape; `id="nav-history"` present in **all seven** page files (a pin counting the occurrences across `frontend/*.html` = 7).
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): the table columns are Title (the open link) / Messages / Updated / Actions; Delete is inline two-step; the nav link is admin-only, placed after Tuning.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: source pins as above + the `caching.py` pin; full suite green.
|
||||
- Coverage: **>90%** on `app/` (the `HTML_PAGES` change is trivially covered by the pin).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `/history.html` renders the shared header + the full-width table (admin: the rows; anonymous: the gated state, and **no** `/api/chats` request on the wire).
|
||||
- [ ] The title link returns to `/?chat=<id>`; Delete's two-step removes the row (the API 404s afterwards).
|
||||
- [ ] `#nav-history` is present (hidden) on all seven pages and revealed for admin only; `GET /history.html` carries `Cache-Control: no-cache` with `?v=`-tagged asset refs.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 05 — History E2E + regressions + commit
|
||||
|
||||
**Phase:** `50_chat_history` · **Source:** `TODO.md:5` — "Need a way to save and view chat history in a new page, then return to that history with a click"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove the save → list → open → return → delete loop in the browser (admin + anonymous views), run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_chat_history.py` (new) — mock-only, DB up; admin login via `tests/e2e/auth_helpers.py` (the `ADMIN_PASSWORD` pattern):
|
||||
- `test_save_and_see_history` — admin: on `/`, ask an on-topic question (a `test_chat_rag.py`-style phrasing) and wait for `done`; assert `#save-chat-btn` is visible; click it → the live region reads "Conversation saved."; navigate to `/history.html` → the table has a row for this chat: the auto-title (the question's whitespace-collapsed, ≤120-char text) and message count 2; **and** the API agrees (`httpx GET /api/chats` with the admin session cookie — the row exists with the right title).
|
||||
- `test_open_chat_returns_to_history` — from the History row click the title → the URL is `/?chat=<uuid>`; the chat renders the saved conversation (the user question bubble + the brain answer with its source chips — the same answer text the History session saw); ask a **new** question and it streams fine (the conversation continues); press Save again → `GET /api/chats` (admin cookie) shows the **same single** row (the upsert) with the message count grown to 4.
|
||||
- `test_new_chat_unlinks` — after the previous flow (or a fresh open): press New chat, then Save → the list now has **two** rows (a fresh create, not an update of the opened one).
|
||||
- `test_delete_two_step` — History: click Delete on a row → the inline confirm pair appears (Playwright would hang on a real `window.confirm` — its absence is itself pinned); No → the row stays; Delete again, Yes → the row is gone; `GET /api/chats/<id>` (admin cookie) → 404; and `/?chat=<that id>` now shows the error banner + the local restore (the deleted-chat degradation).
|
||||
- `test_anonymous_cannot` — a fresh context (no login): on `/` — `#save-chat-btn` not visible and `#nav-history` not visible; a direct `GET /history.html` — the page loads, the table shows the gated/empty state, and **no** `/api/chats` request was made (assert via `page.on("request")`); `httpx GET /api/chats` without the cookie → 403.
|
||||
- DB isolation: saved-chat rows persist in the shared e2e DB across suites — each test uses a **distinctive question text** (so its auto-title is unique), never asserts on absolute row counts, and deletes the rows it creates in a `finally` (admin cookie).
|
||||
2. `tests/e2e/test_cache_busting.py` — add `/history.html` to the pages that suite walks (the no-cache + `?v=` contract applies to the new page — the minimal diff to its page list).
|
||||
3. Regression pass (isolation runs): `test_chat_persistence.py` (the boot path gained the `?chat=` branch), `test_nav_consistency.py` + `test_shared_header.py` (a seventh nav link — extend their link enumeration if they assert the exact nav set), `test_admin_auth.py` (the whoami gate unchanged), `test_cache_busting.py` (after the page-list addition).
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section — staging this phase's files; move `.agent/phases/todo/50_chat_history/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All five story tests pass in isolation; the regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Phase 51 — Share a Chat by Link (Anonymous View)
|
||||
|
||||
**Source:** `TODO.md` L6 — "Need a way to share a chat with a link so others can see it anonymously."
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-29)
|
||||
**Context:** Builds on phase 50's `saved_chats` rows: sharing is a token on a saved chat. The public surface is a **new anonymous page** `/shared/<token>` (a real route — the static mount cannot serve a dynamic path) rendering a read-only copy of the conversation through the same record shape (thinking block, tool lines, source chips, stopped note) — no composer, no controls. The documents API is admin-only (phase 16), so a guest's source chips are plain text (owner-locked). The cache-busting middleware treats known HTML paths (the `HTML_PAGES` tuple in `app/core/caching.py`) as revalidate + `?v=`-rewrite pages; the shared page joins that contract by path prefix.
|
||||
|
||||
## Objective
|
||||
The owner can turn a saved chat into a public link (`/shared/<token>`); anyone with the link sees the conversation read-only, anonymously; unsharing revokes it.
|
||||
|
||||
## Dependencies
|
||||
- `50_chat_history` (todo) — the `saved_chats` row, the Save flow, and the History table the share actions extend.
|
||||
|
||||
## Tasks
|
||||
1. `01_share_token.md` — migration `0009_saved_chat_share_token` + the share/unshare/public-read API + the `/shared/<token>` page route + the middleware prefix.
|
||||
2. `02_share_ui.md` — the Share button (chat page, save-then-share in one action) + the History table's share column (create/copy/unshare).
|
||||
3. `03_shared_page.md` — `shared.html` + `shared.js`: the anonymous read-only rendering.
|
||||
4. `04_e2e_share_chat.md` — the story Playwright suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: share/unshare/public-read contract (token shape, idempotent share, unshare revokes, a wrong token 404s, no admin needed to read, `updated_at` untouched by share/unshare) + `tests/integration/test_migration_0009.py`.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_share_chat.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin: the Share button on the chat page (saved or unsaved conversation) and the History row both produce/copy the `/shared/<token>` link; an unsaved conversation is saved + shared in one action.
|
||||
- [ ] A fresh anonymous context opening `/shared/<token>` sees the full conversation read-only (thinking collapsed, tool lines, the stopped note where present, source chips as plain text) with **no** composer, Save, Share, Tune, or Retry anywhere; a wrong/revoked token shows the "invalid or revoked" state.
|
||||
- [ ] Unshare revokes: the same URL shows the invalid state afterwards; the served shared page carries the cache-busting contract (no-cache + `?v=` rewrite).
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_share_chat.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_history.py`, `test_chat_persistence.py`, `test_smoke.py`, `test_cache_busting.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked (2026-08-29, roadmap confirmation):** share links are public by design (128-bit `uuid4` token; unshare revokes); the shared page renders the full conversation read-only (thinking collapsed) with **zero** interactive controls; source chips are plain text (guests cannot open documents — the documents API is admin-only); clipboard copy with an inline-link fallback (a homelab http origin may not be a secure context).
|
||||
- **The A10 extension (phase 50, owner permission 2026-08-29) unchanged** — sharing reuses the already-stored row; no new storage beyond the token column.
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ alembic/versions/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): share a chat by link — anonymous read-only /shared/<token> page, share/unshare"
|
||||
```
|
||||
@@ -0,0 +1,27 @@
|
||||
# Task 01 — The share token + public read API + page route
|
||||
|
||||
**Phase:** `51_share_chat` · **Source:** `TODO.md:6` — "Need a way to share a chat with a link so others can see it anonymously."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
A saved chat can be shared (a token), unshared (revoked), and read publicly by token; the `/shared/<token>` URL serves the shared page (a real route ahead of the static mount) with the cache-busting contract.
|
||||
|
||||
## Work
|
||||
1. `alembic/versions/0009_saved_chat_share_token.py` — `revision = "0009"`, `down_revision = "0008"` (verify with `uv run alembic heads`): `op.add_column("saved_chats", sa.Column("share_token", postgresql.UUID(as_uuid=True), nullable=True))` + `op.create_index("ix_saved_chats_share_token", "saved_chats", ["share_token"], unique=True)` (a unique index on a nullable column — Postgres treats NULLs as distinct, the `git_sources.path` house precedent, phase 38); the down reverses both. `app/models.py` — the `SavedChat.share_token: Mapped[uuid.UUID | None]` column (nullable unique, `index=True`… expressed as `unique=True, nullable=True` on the `mapped_column` to match the migration) + the docstring line (the phase-38 `path` column's comment style). Apply: `uv run alembic upgrade head`.
|
||||
2. `app/schemas.py` — `SharedChatOut` — `title: str`, `messages: list[ChatMessage]` (the **public** read shape: no id, no timestamps, no token — a shared chat is a content snapshot, not a handle).
|
||||
3. `app/api/chats.py`:
|
||||
- `POST "/{chat_id}/share"` (admin router) → `{"chat_id": …, "share_url": "/shared/<token>"}` — 200, **idempotent**: an existing token is returned unchanged; a new token is `uuid.uuid4()`, persisted, and `updated_at` is **not** bumped (sharing is not a content edit — write the token with a Core `session.execute(update(SavedChat).where(...).values(share_token=…))`, which skips the ORM `onupdate`; pin this in the tests); 404 on unknown chat.
|
||||
- `POST "/{chat_id}/unshare"` (admin router) → `{"chat_id": …, "shared": false}` — the token set NULL (the same Core-update pattern), idempotent (an unshared chat unshares cleanly); 404 on unknown.
|
||||
- `GET "/shared/{token}"` (public — **no** admin dependency; put it on a second module-level `public_router = APIRouter(tags=["chats"])` in the same file, registered in `main.py` with `prefix="/api"`, so the JSON endpoint is `GET /api/shared/<token>`) → `SharedChatOut`; 404 `{"detail": "unknown or revoked share link"}` for a wrong or revoked token (one message — no enumeration between the two cases).
|
||||
4. **The page route** — `app/main.py`: `GET /shared/{token}` (a small router in `app/api/chats.py` or an inline route, registered **without** a prefix and **before** the static mount — the API-routes-first convention; `/shared/<uuid>` is not a static file, so without this route the mount would 404 it) → `FileResponse(static_dir / "shared.html")` (the page lands in task 03 — guard the missing file with an explicit check returning the same 404 JSON as the API, so a stale deploy never 500s).
|
||||
5. `app/core/caching.py` — extend the middleware's known-page dispatch: a path starting with `"/shared/"` gets the same treatment as `HTML_PAGES` (no-cache + `?v=` asset-rewrite on the `text/html` body — the `FileResponse` body is drained by the existing `_read_body` path). Comment it (phase 51: the dynamic share page). `tests/unit/test_caching.py` — pins: a `/shared/<uuid>` path is treated as a known HTML page (no-cache + rewrite), an `/api/shared/<uuid>` path passes through untouched, and an unknown path is untouched.
|
||||
6. `tests/integration/test_chats_api.py` — extend the house file: share (200 + `share_url` matching `/^\/shared\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/`; a second call returns the **same** token), unshare (token NULL; the public read 404s; unshare on an unshared chat → 200 idempotent), the public read (a fresh anonymous client: 200 with title + messages round-tripping and **no** `id`/`created_at`/`updated_at`/`share_token` keys in the body; a wrong token 404s with the "unknown or revoked" detail; a revoked token 404s with the same detail), the `updated_at`-unchanged pin for share/unshare, and 404s for share/unshare on unknown ids.
|
||||
7. `tests/integration/test_migration_0009.py` (new) — the house pattern: upgrade/downgrade round-trip; the unique index exists; the NULLs-distinct behavior (two rows may both carry NULL; two identical non-NULL tokens are rejected).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: as above; full suite green.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `POST /api/chats/<id>/share` is idempotent and leaves `updated_at` alone; `unshare` revokes; `GET /api/shared/<token>` is public + 404-safe; `GET /shared/<token>` serves the page route (404-JSON guard when the file is missing).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 02 — Share button (chat) + the History share column
|
||||
|
||||
**Phase:** `51_share_chat` · **Source:** `TODO.md:6` — "Need a way to share a chat with a link so others can see it anonymously."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Two share entry points for the admin: the **Share** button on the chat page (an unsaved conversation is saved + shared in one action) and a Share column on the History table (create link / copy / unshare) — both copy the link with a clipboard + inline-link fallback.
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` — beside `#save-chat-btn`, `#share-chat-btn` (the same ghost-pill family: an inline link/share SVG + the label "Share", `aria-label="Share chat"`, ships `hidden` — revealed for admin exactly like Save); a comment documenting the save-then-share contract (2026-08-29, `TODO.md` L6).
|
||||
2. `app/schemas.py` + `app/api/chats.py` (the small server side of this task): `SavedChatCreate` gains `share: bool = False`; the create route, when `share` is true, sets `share_token = uuid.uuid4()` in the same commit and the response carries `share_url`. `SavedChatOut` and `SavedChatRow` each gain `share_url: str | None` (`None` → absent from the JSON) — the list endpoint populates it, so the History column renders from `GET /api/chats` without a second fetch.
|
||||
3. `frontend/assets/app.js`:
|
||||
- **`shareCurrentChat()`** — the `#share-chat-btn` handler (the same empty-conversation no-op guard as Save): if `currentChatId` is set → `POST /api/chats/<id>/share`; else → `POST /api/chats` with `{ messages: conversation, share: true }` → `currentChatId = created.id` (one action saves **and** shares — owner-locked). On success: copy the absolute URL of `share_url` — `navigator.clipboard.writeText(...)` in a try; on success the live region reads "Share link copied."; on failure (a non-secure http origin rejects the clipboard) render the **inline fallback**: a transient link field (an `<a>` styled as a select-on-focus field, carrying the full URL) near the status line + the live region "Share link ready — copy it from the field." (owner-locked fallback). On 403/5xx/network: `showErrorBanner` with an actionable line.
|
||||
- **Reveal on boot:** `#share-chat-btn` joins the same admin-reveal block as `#save-chat-btn`.
|
||||
- **Header comment:** the share contract (2026-08-29, `TODO.md` L6).
|
||||
4. `frontend/history.html` + `frontend/assets/history.js` — the **Share column** between Updated and the Actions/Delete cell (the `th` + the per-row cell; the table stays full-width):
|
||||
- row already shared (`share_url` present): a **Copy** button (the same clipboard + fallback helper — the per-page duplication house style: `history.js` keeps its own ~10-line copy of the helper rather than a new shared module) and an **Unshare** button (inline two-step, the phase-50 Delete-confirm pattern: "Unshare? [Yes] [No]" → `POST /api/chats/<id>/unshare` → the cell re-renders to the unshared state + the live region).
|
||||
- row unshared: a **Create link** button → `POST /api/chats/<id>/share` → the cell re-renders to the shared state (Copy + Unshare) and the link is offered for copying (same fallback pattern).
|
||||
5. `frontend/assets/styles.css` — `.share-chat-btn` (the pill family), the `.history-share` cell buttons (the Tune/Retry-family ghost buttons, ≥44px comfortable, focus-visible), `.share-link-fallback` (the inline link field — input-like look, select-on-focus), reusing the phase-50 `.history-confirm` styles for the unshare two-step.
|
||||
6. `tests/integration/test_chats_api.py` — extend: create-with-share (the response carries `share_url` matching the token shape and the row is immediately publicly readable), create-without-share (no `share_url`), the list rows carry `share_url` when shared and absent otherwise, `GET /{chat_id}` carries it too.
|
||||
7. Frontend source pins (house pattern): the `shareCurrentChat` branches (linked → POST share; unlinked → create-with-share + `currentChatId` set); the clipboard try → fallback element on rejection; the History cell's three states (unshared / shared / confirming-unshare) + the two-step unshare; `#share-chat-btn` ships hidden and reveals only for admin.
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): the Share button saves + shares an unsaved conversation in one action; the clipboard copy has the inline-link fallback; unshare is inline two-step.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (the create-with-share branch covered).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin: the Share button works on a saved (linked) and an unsaved conversation; the History column creates / copies / unshares per row.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 03 — The shared page (anonymous read-only)
|
||||
|
||||
**Phase:** `51_share_chat` · **Source:** `TODO.md:6` — "Need a way to share a chat with a link so others can see it anonymously."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
`/shared/<token>` renders the shared conversation for **anyone** — read-only, zero controls, nothing the share shouldn't expose — through the same record shape the chat uses.
|
||||
|
||||
## Work
|
||||
1. `frontend/shared.html` (new) — the page scaffold as `history.html` does it: the head (charset, viewport, `assets/styles.css`), the skip link, the **identical** shared header (the nav with all the admin-only links hidden — a guest never sees them — and the auth pair with `?next=/` on the Sign-in link: a guest signing in from a shared page returns to the app root; note this in a comment), then `<main id="main">` carrying the `#steering-panel` section + `#steering-announcer` (the shared panel ships on every page — phase 34) and the page content:
|
||||
- an `<h1 id="shared-title">` (JS-filled with the shared chat's title; static fallback text "Shared conversation");
|
||||
- a `.shared-note` line — "Shared via Brain of Reese — read-only." (the brand resolves through the `window.BOR_BRAND` convention like the other pages);
|
||||
- the messages section — `<section class="messages" id="messages" aria-label="Shared conversation">` using the **same** `.msg`/`.bubble`/`.thinking`/`.tool-calls` structure as the chat page, so the existing CSS applies unchanged (the shell maps to the 46rem centered chat column — reuse the `.chat-shell` class or a `.shared-shell` that maps to the same width rule, per the PLAN §7 column contract);
|
||||
- the invalid-state block `#shared-invalid` (hidden by default): "This share link is invalid or was revoked."
|
||||
- the app-footer (version span, like the other pages).
|
||||
- Scripts: `<script src="assets/brand.js"></script>` (classic, first) + `<script src="assets/markdown.js"></script>` (the classic renderer, as `index.html` loads it) + `<script type="module" src="/assets/shared.js"></script>`. **No** `document-modal.js`, **no** composer, **no** Save/Share/Retry/Tune markup at all (owner-locked: zero controls). No-CDN rule holds (local assets only).
|
||||
2. `frontend/assets/shared.js` (new) — a module:
|
||||
- read the token from `location.pathname` (the last path segment of `/shared/<token>`; a malformed/missing token → show `#shared-invalid` immediately, **no fetch**).
|
||||
- `await initSharedHeader()` (the header works for guests — whoami anonymous, the admin links stay hidden), then `GET /api/shared/<token>`:
|
||||
- 200 → set the `h1` to the title; render every message through a local `renderSharedMessage(m)` reusing the chat's record shape: user → the `.msg.user` bubble; brain → the `.msg.brain` bubble with the optional thinking block (**collapsed** — the phase-17 restore convention), the tool lines, the `is-deflected` class, the stopped note (the ~8-line `appendStoppedNote` markup duplicated locally — the per-page duplication house style), the deflection's "Maybe try" chips as **plain `<span class="suggestion-chip">`** text (not buttons — a guest tapping a chip has nowhere to go; owner-locked zero controls), and the source chips as **plain text `<span>`** (owner-locked: guests cannot open documents — the documents API is admin-only; no `href`, no modal wiring).
|
||||
- 404/other → show `#shared-invalid` (the title keeps its fallback), no data rendered, no error banner.
|
||||
- markdown through the global `renderMarkdown` (escape-first — the stored payloads are raw text, so the renderer's XSS safety applies unchanged).
|
||||
3. `frontend/assets/styles.css` — `.shared-note` (the muted meta line under the h1); the static-chip treatment scoped to the shared page (e.g. `.shared-shell .suggestion-chip { pointer-events: none; cursor: default; }` — or a distinct `.chip-static` class if cleaner; the interactive chips' styles elsewhere stay untouched); the invalid-state styling (a centered muted block); the shared shell's 46rem column mapping; the ≤640px responsive behavior (the phase-07 contract).
|
||||
4. Frontend source pins (house pattern): the token parse (a malformed path → no fetch, the invalid state shows); the 404 → invalid state (no data render); **zero interactive controls** (pin: `renderSharedMessage` never calls `renderChips`/`appendTuneButton`/`appendRetryButton`, and the rendered shared messages contain no `<button`/`<form` — i.e. chips are spans, source chips carry no `href`); `shared.html` does not reference `document-modal.js` or a composer.
|
||||
|
||||
- ASSUMPTION (owner-locked 2026-08-29): the shared page shows the full conversation (thinking collapsed) read-only; zero interactive controls; the "Maybe try" chips are plain text; the source chips are plain text (no document access for guests).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: source pins as above; full suite green.
|
||||
- Coverage: **>90%** on `app/` (unchanged — frontend-only task).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /shared/<token>` (the task-01 route) serves the page; a fresh anonymous browser renders the conversation read-only.
|
||||
- [ ] A wrong/revoked token shows the invalid state; no control exists anywhere on the page.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Task 04 — Share E2E + regressions + commit
|
||||
|
||||
**Phase:** `51_share_chat` · **Source:** `TODO.md:6` — "Need a way to share a chat with a link so others can see it anonymously."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove the share → anonymous view → revoke loop in the browser, run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_share_chat.py` (new) — mock-only, DB up; admin via `tests/e2e/auth_helpers.py`; the anonymous view in a **fresh context** (`browser.new_context()` — a separate session, no cookies):
|
||||
- `test_share_from_chat_page` — admin: ask an on-topic question → `done`; (the conversation is unsaved) click `#share-chat-btn` → assert the live region reads "Share link copied." when `navigator.clipboard` is available in the context, else the fallback link field is present with the `/shared/<uuid>` URL (branch the assertion on `page.evaluate(() => !!navigator.clipboard)`); the API agrees: `GET /api/chats` (admin cookie) has the new row with a non-null `share_url` matching `/^\/shared\/[0-9a-f-]{36}$/`.
|
||||
- `test_anonymous_shared_view` — open the `share_url` in the fresh anonymous context (use a "think out loud …" on-topic question so the thinking block exists): the title = the auto-title; the user question bubble + the brain answer are present (the same deterministic answer text the admin session saw); `details.thinking` exists and is **not** open (collapsed); the source chips are plain text (zero `a.source-chip` in the shared view); **no** `#composer`, no `#save-chat-btn`/`#share-chat-btn`, no `.tune-btn`, no `.retry-btn`, and the "Maybe try" chips (if deflected) are spans, not buttons; the nav admin-only links are hidden (a guest).
|
||||
- `test_share_from_history_and_unshare` — admin, History: a row without a link → the "Create link" button → click → the cell shows Copy + Unshare (or the fallback field); open the link in the fresh anonymous context → it renders; back in admin: Unshare → the two-step confirm → Yes → the cell returns to "Create link"; the anonymous view of the same URL now shows the invalid state ("invalid or was revoked"); `GET /api/chats/<id>` (admin cookie) → `share_url` null.
|
||||
- `test_bad_token_invalid_state` — a fresh context opens `/shared/00000000-0000-4000-8000-000000000000` → the invalid state, no crash, the guest header renders.
|
||||
- DB isolation: the same distinctive-question / cleanup-in-`finally` discipline as `test_chat_history.py`.
|
||||
2. `tests/e2e/test_cache_busting.py` — add the shared page to the walk: create + share a chat via the admin API for the test, then assert `GET /shared/<token>` carries `Cache-Control: no-cache` and the served HTML's asset refs are `?v=`-tagged (the task-01 prefix extension) — the minimal addition to the suite's page coverage.
|
||||
3. Regression pass (isolation runs): `test_chat_history.py` (the schemas + the History table changed), `test_chat_persistence.py` (the chat page's boot + buttons), `test_smoke.py`, `test_cache_busting.py` (after the shared-page addition).
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section — staging this phase's files; move `.agent/phases/todo/51_share_chat/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_share_chat.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All four story tests pass in isolation; the four regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Phase 53 — Invalidate Saved Chats on Sources Sync
|
||||
|
||||
**Source:** `TODO.md` L4 — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data"
|
||||
**Story:** n/a (TODO-derived — owner instruction 2026-08-30: convert without confirmation)
|
||||
**Context:** Phase 50 stores explicitly saved conversations in `saved_chats` (JSONB `bor.chat.v1` records; admin-only CRUD under `/api/chats` in `app/api/chats.py`; the History page is a full-width table per AGENTS.md rule 5; `/?chat=<id>` re-opens a row pixel-identical through the phase-14 restore path). Phase 51 added the public snapshot read (`/shared/<token>` → `SharedChatOut`, no admin dependency). Sources are synced through two canonical paths: the admin **Sync** button (`POST /api/sync` → `app/api/sync.py::_run_sync`: `check_models` → resolve effective sources → clone/pull or local-dir re-verify → `import_sources(prune=True)` → change-gated `regenerate_overview`) and the CLI/quadlet `scripts/import_docs.py` (same `import_sources`; `--limit` debug runs and unchanged re-imports are change-gated on `added + updated`). Neither path records *when the KB last changed*, so a saved answer can silently predate the current index. `retryLastTurn(wrap)` in `frontend/assets/app.js` (phase 49) is the existing redo-in-place mechanism for the LAST brain bubble (re-asks the preceding user question, `runTurn(text, { reask: true })`, no user append, no scroll) — the Regenerate action reuses it. Single-row table precedent: `kb_overview` (id = 1, `app/models.py::KbOverview`).
|
||||
|
||||
## Objective
|
||||
Every sync that actually changes the knowledge base bumps a sources version; saved chats stamp that version at save time; a chat saved against an older version is surfaced as **stale** (History table badge + a banner when opened) with a **Regenerate** action that re-asks the last question against the new index and re-saves the row — a stale answer can no longer masquerade as current.
|
||||
|
||||
## Dependencies
|
||||
- `50_chat_history` (complete) — the `saved_chats` row, `/api/chats` CRUD, the `?chat=<id>` boot load, the linked-row Save upsert (`saveCurrentChat` in `app.js`).
|
||||
- `51_share_chat` (complete) — the public `/shared/<token>` snapshot read (stays a frozen snapshot; untouched by this phase).
|
||||
- `49_retry_answer` (complete) — the `retryLastTurn` redo-in-place the Regenerate action drives.
|
||||
- `52_pinned_composer` (todo, preceding) — the chat-page layout is stable while the banner lands; ordering keeps the chat UI quiet (no shared-file conflict beyond `styles.css`/`app.js` regions).
|
||||
|
||||
## Tasks
|
||||
1. `01_sources_version_and_migration.md` — the single-row `sources_meta` table + `saved_chats.sources_version` (migration 0010) + the current/bump helpers + the migration test.
|
||||
2. `02_sync_version_bump.md` — the version bump on both sync paths (Sync button + CLI), change-gated.
|
||||
3. `03_chats_api_staleness.md` — stamp-on-save + the `stale` flag on the list/detail responses.
|
||||
4. `04_history_stale_badge.md` — the Stale column on the History table.
|
||||
5. `05_stale_banner_and_regenerate.md` — the chat-page banner + Regenerate (`retryLastTurn` reuse) + the auto re-save.
|
||||
6. `06_e2e_stale_saved_chats.md` — the story Playwright suite + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_sources_meta.py` (helpers: absent row → 0, first bump → 1, second bump → 2, idempotent reads); the sync bump gates — extend the existing sync coverage (`tests/integration/test_sync_api.py` + the `tests/fakes.py` override patterns) and the CLI coverage (`tests/integration/test_import_docs_overview.py` pins the change-gating pattern; the bump asserts sit alongside).
|
||||
- Integration: `tests/integration/test_migration_0010.py` (house pattern, from `test_migration_0009.py`); extend `tests/integration/test_chats_api.py` (stamp + `stale` flag + share-unshare version immunity).
|
||||
- Frontend source pins (house style): `history.js` stale-cell branch, `app.js` banner reveal / Regenerate wiring / post-regenerate persist / no-brain-bubble guard (the `test_history_page.py` / `test_save_chat_ui.py` pin patterns).
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_stale_saved_chats.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A KB-changing sync (button or CLI) bumps `sources_meta.version` exactly once; an unchanged re-run, a `--limit` debug run, and a FAILED sync never bump.
|
||||
- [ ] A Save/Re-Save stamps the row's `sources_version`; `GET /api/chats` + `GET /api/chats/<id>` expose `stale` (true iff the row's version is behind the current one); share/unshare never touch the version.
|
||||
- [ ] The History table shows the Stale marker exactly on rows saved before the last KB-changing sync (full-width table geometry unchanged, AGENTS.md rule 5).
|
||||
- [ ] Opening a stale `/?chat=<id>` shows the banner; Regenerate re-streams the last answer in place against the new index and re-saves the row — the banner clears, `GET /api/chats/<id>` reports `stale: false`, the History marker is gone.
|
||||
- [ ] A `/shared/<token>` page is unchanged — the public snapshot carries no staleness surface.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_stale_saved_chats.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_chat_history.py`, `test_share_chat.py`, `test_sync_button.py`, `test_retry_answer.py`, `test_chat_persistence.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Recorded assumptions (TODO conversion, 2026-08-30 — owner asked for no confirmation):**
|
||||
1. The invalidation marker is a monotonically increasing `sources_meta.version` (single row, the `kb_overview` id=1 precedent), bumped ONLY when a sync changed the KB — the gate is `added + updated + pruned > 0` (pruned counts: a deleted doc can invalidate an answer that cited it — deliberately broader than the overview gate's `added + updated > 0`).
|
||||
2. `stale = row.sources_version < current`, computed server-side in the chats API; the client never computes staleness. Pre-existing rows stamp 0 (the pre-counter KB) and go stale on the first bump.
|
||||
3. Regenerate = the phase-49 redo-in-place of the LAST brain bubble (re-ask the last user question with the full conversation context) followed by an auto re-save of the linked row — the owner does not press Save again; a stale chat with no brain answer shows the banner text without a Regenerate button.
|
||||
4. The public shared snapshot (`/shared/<token>`) is deliberately untouched — a frozen snapshot by design; an owner who regenerates can re-share afterwards.
|
||||
5. A failed sync (git/embed/import error) aborts BEFORE the bump — a failed sync never invalidates chats; the bump commits even if the best-effort overview regeneration then fails (the index really did change).
|
||||
- **A10 honoured** — `/api/chat` stays stateless; staleness is a property of the explicitly saved row, not of the chat endpoint.
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ alembic/versions/ scripts/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): invalidate saved chats on sources sync — versioned stamps, stale marker, Regenerate against the new index"
|
||||
```
|
||||
@@ -0,0 +1,68 @@
|
||||
# Phase 54 — Never Serve Stale HTML: Fix the 304 Hole in Cache Busting
|
||||
|
||||
**Source:** owner instruction 2026-08-30 (chat) — found live while verifying phase 52: *"add that as phase 54"*. Not a TODO.md item (TODO.md is empty).
|
||||
**Story:** n/a (bug fix discovered by measurement, not a user story)
|
||||
**Context:** Phase 33 (`app/core/caching.py`, `.agent/reports/33_cache_busting__*`) ships two layers: `asset_version()` — a per-process token, `git rev-parse --short HEAD` in a git checkout (`functools.cache`, so it flips only on process start) — and `CachingMiddleware` (mounted in `app/main.py::create_app` via `configure_caching`, wrapping the `StaticFiles(directory=static_dir, html=True)` catch-all mounted at `/`). For every known page (`HTML_PAGES` + the dynamic `/shared/<token>` prefix) the middleware drains the body, rewrites each local `assets/…` `href`/`src` to carry `?v=<token>` (`rewrite_asset_refs`), and re-serves it with `Cache-Control: no-cache`. `/assets/*` gets `public, max-age=31536000, immutable` header-only; `/api/*` (incl. the SSE chat stream) passes through byte-identical. `_no_cache_headers()` copies **every** upstream header and overrides only `content-length` (dropped) and `Cache-Control` — so the static file's `etag` and `last-modified` travel with the rewritten body.
|
||||
|
||||
**The defect (measured on the running app, 2026-08-30):** the conditional-request validators describe the *static file*, but the *served bytes* are the rewritten body — a body this process built from its own token. A revalidation therefore 304s out of the rewrite and the browser keeps the HTML it already has, whose `?v=` points at the **previous** commit's CSS/JS, which is cached `immutable` for a year:
|
||||
|
||||
```
|
||||
GET / → 200, etag: "2a838bc8d1c4fc0e6cb75128913c3fb6", cache-control: no-cache
|
||||
GET / If-None-Match: "2a838bc8…" → 304 ← stale HTML kept, stale CSS pinned
|
||||
GET /history.html → 200, etag: "9b700148…"
|
||||
GET /history.html If-None-Match: "9b700148…" → 304 ← same
|
||||
```
|
||||
|
||||
The 304 is produced *downstream* (Starlette `FileResponse`/`StaticFiles` honours `If-None-Match` / `If-Modified-Since` before the middleware can touch the response), so the middleware must stop honouring those headers on page paths — stripping the outbound validators afterwards is too late: the body never arrives. The asymmetry to preserve: a 304 on `/assets/*` is **safe** (the URL itself is versioned, so a revalidated 304 re-serves the same versioned asset), a 304 on an HTML page is **never safe** (the served body depends on the process token, which the validator ignores). Observed user-visible symptom: after `aba8615` (the phase-52 fix) the bare `GET /` still served HTML referencing `styles.css?v=8207539`, i.e. the browser kept rendering the pre-fix layout until a hard reload.
|
||||
|
||||
## Objective
|
||||
HTML pages are revalidated against the bytes that are actually served, never against the static file underneath the rewrite: a conditional `GET` of any known page (`/`, every `HTML_PAGES` entry, `/shared/<token>`) returns **200 with the current `?v=<token>` body**, and those responses publish no `etag` / `last-modified`. `/assets/*` keeps its immutable-for-a-year + conditional-304 behavior and `/api/*` (incl. SSE) stays byte-identical, so a deploy can no longer leave a browser on stale CSS/JS.
|
||||
|
||||
## Dependencies
|
||||
- `33_cache_busting` (complete) — the module under fix: `asset_version()`, `rewrite_asset_refs()`, `CachingMiddleware`, `_no_cache_headers()`, `HTML_PAGES`, `ASSETS_PREFIX`, `ASSET_CACHE_CONTROL`, `HTML_CACHE_CONTROL`. Behavior contracts to preserve, not change.
|
||||
- `51_share_chat` (complete) — `/shared/<token>` joins the page contract by path prefix (a REAL route serving `shared.html`); it must get the same revalidation fix.
|
||||
- `50_chat_history` (complete) — `/history.html` is in `HTML_PAGES`; it is one of the reproduced 304s.
|
||||
- `52_pinned_composer` (complete) — the fix is what makes that fix actually reach the browser; its `#messages { flex: 1 1 auto }` rule is the "am I on current CSS?" probe the E2E suite asserts.
|
||||
- `16_admin_auth` (complete) — admin pages go through the same middleware; the signed session cookie path must not change.
|
||||
|
||||
## Tasks
|
||||
1. `01_page_paths_ignore_conditional_headers.md` — `app/core/caching.py`: drop `if-none-match` / `if-modified-since` on known-page requests before `call_next`, skip the rewrite for bodiless statuses (204/304), drop the outbound `etag` / `last-modified` on page responses.
|
||||
2. `02_unit_tests_conditional_pages.md` — extend `tests/unit/test_caching.py` with the conditional-request + validator pins on a `StaticFiles`-backed app.
|
||||
3. `03_integration_conditional_get.md` — new `tests/integration/test_caching_revalidation.py`: the real app, real `frontend/` tree, every page family, plus the `/assets/*` and `/api/*` non-regressions.
|
||||
4. `04_e2e_and_docs.md` — the dedicated Playwright suite `tests/e2e/test_asset_cache_revalidation.py` + the README cache-busting section + regressions + commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/test_caching.py`, extend — house style: bare `FastAPI` app + the middleware + `TestClient`): a conditional `GET /` against a `StaticFiles(html=True)` mount that already holds the etag returns **200 + the rewritten body** (this is the regression that reproduces the defect); a conditional `GET /` with `If-Modified-Since: <the mount's last-modified>` likewise 200s; page responses carry **no** `etag` and **no** `last-modified` while keeping `Cache-Control: no-cache`; a downstream bodiless 304 is passed through as a bodiless 304 (never a `Response(content=…, status_code=304)` with a body, which starlette rejects) with `no-cache` and no validators; `/api/*` and `/assets/*` still pass through byte-identical (conditional `GET /assets/…` may still 304 — that path is URL-versioned and safe); the inbound strip touches page paths only.
|
||||
- Integration (`tests/integration/test_caching_revalidation.py`, new — real `app.main:app`, real `frontend/` tree, no mocks needed): capture the etag of `GET /` then re-request it with `If-None-Match` → 200 whose body contains `styles.css?v=<asset_version()>` and whose headers have no `etag` / `last-modified`; the same pair for `/index.html`, `/sources.html`, `/tuning.html`, `/git-sources.html`, `/history.html`, `/login.html`, `/document.html`, `/shared.html`, and for a dynamic `/shared/<token>`; `GET /assets/styles.css?v=<token>` still carries `public, max-age=31536000, immutable` + `etag` + `last-modified` and still answers a conditional `GET` with 304; `POST /api/chat` keeps its SSE `text/event-stream` headers untouched.
|
||||
- Coverage: **>90%** on `app/` (the validate.sh gate) — every new branch in `caching.py` (strip, bodiless-status guard, outbound validator drop) must be covered.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_asset_cache_revalidation.py`, run in isolation (DB up).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Conditional `GET /` (and of every other known page, static and dynamic) returns 200 with the current `?v=<token>` references — no 304 on a page path, so the browser can never reuse HTML that points at a previous commit's assets.
|
||||
- [ ] Page responses publish no `etag` / `last-modified` and keep `Cache-Control: no-cache`.
|
||||
- [ ] `/assets/*` unchanged: `public, max-age=31536000, immutable`, validators intact, conditional `GET` still 304s.
|
||||
- [ ] `/api/*` unchanged byte-for-byte, including the SSE `POST /api/chat` stream and the signed-cookie admin paths.
|
||||
- [ ] A bodiless downstream 304/204 on a page path is passed through bodiless (with `no-cache`, no validators) instead of being rewritten into a bodiless status with a body.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_asset_cache_revalidation.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_cache_busting.py`, `test_smoke.py`, `test_shared_header.py`, `test_share_chat.py`, `test_chat_history.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `README.md` documents the rule (why pages never 304, why assets still may) and `git rev-parse --short HEAD` + hard-reload behaviour is recorded.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A2/A11 honoured** — the fix stays inside the existing Starlette middleware; no new dependency, no CDN, no new service (A12 unchanged: still no Valkey/CDN to hold asset versions).
|
||||
- **Recorded assumptions (owner asked for the phase in chat, no further interview):**
|
||||
1. **Fix the request, not the response.** The 304 is generated downstream by `FileResponse`/`StaticFiles`, before the middleware sees it, so the middleware removes `if-none-match` / `if-modified-since` from the *inbound* request for known page paths (via starlette's `MutableHeaders(scope=request.scope)`) so a full body always arrives to be rewritten. Post-hoc header rewriting cannot fix a body that was never sent.
|
||||
2. **Pages publish no validators.** `etag` / `last-modified` are dropped from page responses; combined with `Cache-Control: no-cache` this means every navigation re-fetches the ~10–18 KB page with the current token. Alternative considered and **rejected**: publish a weak `ETag` over the rewritten body so same-process revalidations still 304 — it only saves bandwidth on a LAN, and it is only correct if every layer (route, static mount, any proxy) computes the validator over the rewritten bytes, which cannot be guaranteed here. Correctness ("never stale") beats a 304.
|
||||
3. **`/assets/*` is left completely alone** — header-only caching as today; a conditional `GET` on a versioned asset URL may still 304 because the URL already encodes the version. The inbound strip is scoped to `HTML_PAGES` + `/shared/` and must not widen.
|
||||
4. **`/api/*` stays byte-identical** (A10/A15): no header changes, the SSE body is never drained, and no conditional-header stripping.
|
||||
5. **The token algorithm is not touched.** `asset_version()` keeps its per-process `functools.cache` semantics (the token changes on the next process start). Making it per-request was rejected: a per-request `git rev-parse` reopens the boot-hang cap `_GIT_TIMEOUT_S` on the hot path and would make one page load able to reference two versions.
|
||||
6. A bodiless downstream status (204/304) on a page path is **never** rewritten — starlette forbids a body on those statuses, and today the rewrite path would build exactly that. It is passed through with `no-cache` and no validators.
|
||||
- **No Regressions** — phase 33's observable contracts (`?v=` on every asset ref, `no-cache` on pages, `immutable` on assets, `/api/*` untouched) and phases 50/51/52's pages keep working; the only behavior change is that a page revalidation yields 200 instead of 304.
|
||||
- **A16/A17 honoured** — one story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agent/ app/ README.md tests/ && git commit --no-gpg-sign -m "fix(web): never 304 a rewritten page — pages drop conditional validators, assets keep them"
|
||||
```
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Task 01 — Page paths ignore conditional headers
|
||||
|
||||
**Phase:** `54_asset_cache_bust_revalidation` · **Story:** n/a (bug fix discovered by measurement)
|
||||
|
||||
## Objective
|
||||
`CachingMiddleware` stops honouring `If-None-Match` / `If-Modified-Since` on known HTML page paths so the full (rewritable) body always arrives, passes through any downstream bodiless 304/204 safely, and never publishes validators on page responses.
|
||||
|
||||
## Work
|
||||
1. `app/core/caching.py` — inside `CachingMiddleware.dispatch`, **before** `call_next`:
|
||||
- Reuse the existing page-path predicate: extract `is_known_page = path in HTML_PAGES or path.startswith("/shared/")` into a module-level `_is_known_page(path: str) -> bool` so the inbound check and the existing outbound branch share one source of truth (no behavior drift between the two).
|
||||
- When `_is_known_page(path)` is true, strip the conditional request headers before the downstream app runs. Starlette's `Request.headers` is read-only, but the middleware owns the scope:
|
||||
```python
|
||||
from starlette.datastructures import MutableHeaders
|
||||
headers = MutableHeaders(scope=request.scope)
|
||||
for name in ("if-none-match", "if-modified-since"):
|
||||
if name in headers:
|
||||
del headers[name]
|
||||
```
|
||||
`MutableHeaders` mutates `request.scope["raw_headers"]` in place, so the downstream `FileResponse`/route never sees the validators and can only return the full 200 body. `if-none-match` / `if-modified-since` are the two headers `FileResponse`/`StaticFiles` can act on for these paths.
|
||||
- **Do not** touch the request for any other path — `/api/*`, `/assets/*`, unknown paths keep their conditional headers (asset 304s are safe; the SSE stream must not be disturbed).
|
||||
2. `app/core/caching.py` — in the page branch, **guard the bodiless statuses before `_read_body`**:
|
||||
```python
|
||||
if response.status_code in (204, 304):
|
||||
response.headers["Cache-Control"] = HTML_CACHE_CONTROL
|
||||
response.headers.pop("etag", None)
|
||||
response.headers.pop("last-modified", None)
|
||||
return response
|
||||
```
|
||||
Today the rewrite path would build `Response(content=new_body, status_code=304)` — a body on a bodiless status, which starlette rejects and which would otherwise be the stale-304 bug re-served from the middleware. After the inbound strip this branch is a belt-and-braces guard (a future route or proxy could still produce it).
|
||||
3. `app/core/caching.py` — extend `_no_cache_headers` (used by every rewritten/fallback page response) to also drop the upstream validators, so a page response can never be 304'd against later:
|
||||
```python
|
||||
for name in ("etag", "last-modified"):
|
||||
headers.pop(name, None)
|
||||
```
|
||||
Keep the existing `content-length` drop and the `Cache-Control: no-cache` override. The three fallback returns in `dispatch` (buffer failure, non-HTML page body, rewrite failure) all flow through `_no_cache_headers`, so they lose the validators too.
|
||||
4. `app/core/caching.py` — module + method docstrings: record WHY (one paragraph, mirroring the phase overview): validators describe the static file, not the rewritten body, so pages must not 304; the asymmetry — `/assets/*` 304s are safe because the URL is versioned, pages are not. Update the class docstring's two-bullet contract ("always revalidated") to state "revalidated with a full 200 body (no 304), publishing no validators".
|
||||
- Do NOT change: `asset_version()` (token algorithm + `functools.cache`), `rewrite_asset_refs`, `_ASSET_REF_RE`, `HTML_PAGES`, `ASSETS_PREFIX`, `ASSET_CACHE_CONTROL`, `HTML_CACHE_CONTROL`, `_GIT_TIMEOUT_S`, or the `/api/*` pass-through.
|
||||
|
||||
## Testing & Quality
|
||||
- The branches this task adds (`_is_known_page`, the inbound strip, the bodiless guard, the validator drops) are covered by Task 02's unit tests and Task 03's integration tests — those must exist before this phase is complete; this task's own gate is that the existing suite stays green.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate); the new branches are expected to be exercised by Task 02/03 tests.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_caching.py -v` green (the pre-existing phase-33 pins still hold: `?v=` rewrite, `no-cache`, `/assets/*` headers, `/api/*` byte-identical).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] No change in `app/main.py` (the middleware is mounted there; the fix is fully inside `app/core/caching.py`).
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
# Task 02 — Unit: conditional page fetches + validator pins
|
||||
|
||||
**Phase:** `54_asset_cache_bust_revalidation` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
Extend `tests/unit/test_caching.py` (the house-style unit module for phase 33) so the 304 hole is reproduced as a failing test first, then pinned: known pages must 200 on conditional requests and publish no validators, while `/assets/*` and `/api/*` keep their existing conditional behavior.
|
||||
|
||||
## Work
|
||||
1. `tests/unit/test_caching.py` — new helper `_static_page_app(tmp_path)`:
|
||||
- build a mini static tree: `index.html` (containing `href="/assets/app.js"` and `src="/assets/app.js"` refs) + `assets/app.js` (`"x"`), `assets/app.css` ("y");
|
||||
- app: `app = FastAPI(); app.mount("/", StaticFiles(directory=<tmp frontend>, html=True), name="static")` + `app.get("/api/ping")` returning JSON + a `@app.get("/shared/{token}")` route that — like the phase-51 route — serves `text/html` **and honours a conditional header** (`if "if-none-match" in request.headers: return Response(status_code=304)`), so the inbound-strip behavior is observable at the route;
|
||||
- attach `CachingMiddleware` the same way `configure_caching` does;
|
||||
- `asset_version.cache_clear()` + monkeypatch the token to a deterministic value for this test only: monkeypatch `app.core.caching.get_settings` (the module-level import the middleware calls) to a settings stub whose `static_dir` is the tmp frontend — mirroring `test_default_static_dir_comes_from_settings`. Use token `"unit-token"` so assertions are exact.
|
||||
2. New tests (assertions against `TestClient(app)`):
|
||||
- `test_conditional_get_page_returns_200_with_rewrite` — `GET /` → 200, body contains `?v=unit-token`; note the response has NO `etag` (the fix drops it) — so derive the *file* validator the way a browser would have: `starlette.responses.FileResponse(<index.html>).headers["etag"]`; then `GET /` with `If-None-Match: <that etag>` → **200** (never 304), body still rewritten, headers: `cache-control == "no-cache"`, no `etag`, no `last-modified`. This test FAILS on the pre-fix code (it gets a 304) — it is the regression reproduction.
|
||||
- `test_conditional_get_page_with_if_modified_since_returns_200` — same pair using `If-Modified-Since` from `FileResponse(...).headers["last-modified"]` → 200 + rewrite.
|
||||
- `test_page_response_publishes_no_validators` — plain `GET /` → no `etag`, no `last-modified`, `cache-control == "no-cache"`.
|
||||
- `test_downstream_304_on_page_is_passed_bodiless` — a dedicated app whose `/` route returns `Response(status_code=304)` when `if-none-match` is present, else the full html; `GET /` with an `If-None-Match` header → the middleware must return a **304 with an empty body** (assert `resp.content == b""`), `cache-control == "no-cache"`, no `etag` — never a `Response(content=…, status_code=304)` (starlette raises on that; today's rewrite path would build it).
|
||||
- `test_shared_page_path_ignores_conditional_headers` — `GET /shared/abc123` with `If-None-Match: "whatever"` → 200 + rewrite (the conditional route above would have 304'd if the strip were missing).
|
||||
- `test_api_path_keeps_conditional_headers` — `/api/ping` route echoes 304 when `if-none-match` is present; `GET /api/ping` with `If-None-Match` → **304** (the strip must NOT widen to `/api/*`) and NO `cache-control` injected.
|
||||
- `test_assets_path_keeps_validators_and_304` — `GET /assets/app.css` → 200, `cache-control == ASSET_CACHE_CONTROL`, `etag` present; conditional `GET /assets/app.css` with that etag → **304** (versioned-URL 304s stay safe).
|
||||
3. Keep every existing test in the file green — phase 33's pins (token algorithm, `rewrite_asset_refs` idempotency, the buffer/rewrite failure fallbacks, `test_html_pages_include_history`, the shared-page contract) are unchanged by this phase.
|
||||
- The `MutableHeaders` import and the strip only exist in `app/core/caching.py` (task 01) — no new test helper modules.
|
||||
|
||||
## Testing & Quality
|
||||
- `uv run pytest tests/unit/test_caching.py -v` — every test above green; the first one fails on pre-fix code (run it against the pre-fix tree once to confirm it reproduces the defect before committing task 01's fix — the task files run in order, so task 01 lands first; this is a note for the executor, not an extra step).
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate) — these tests must cover every branch task 01 added.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_caching.py -v` green (all pre-existing + all new tests).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
# Task 03 — Integration: conditional GETs against the real app
|
||||
|
||||
**Phase:** `54_asset_cache_bust_revalidation` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
Prove the fix end-to-end against the REAL `app.main:app` (real `StaticFiles` mount on the real `frontend/` tree, real middleware, real `asset_version()` token): every known page — static and the dynamic share page — 200s on a conditional GET with the current `?v=<token>` body and no validators, while `/assets/*` still immutable-304s and `/api/*` stays untouched.
|
||||
|
||||
## Work
|
||||
1. `tests/integration/test_caching_revalidation.py` (new) — imports `client` (the real-app `TestClient`, `tests/conftest.py`) and `asset_version` from `app.core.caching`; a helper:
|
||||
```python
|
||||
def file_validators(page_file: Path) -> tuple[str, str]:
|
||||
"""The etag / last-modified Starlette would stamp on the underlying
|
||||
static file — exactly what a browser revalidates against."""
|
||||
from starlette.responses import FileResponse
|
||||
headers = FileResponse(page_file).headers
|
||||
return headers["etag"], headers["last-modified"]
|
||||
```
|
||||
and `FRONTEND = REPO / "frontend"` (`REPO = Path(__file__).resolve().parents[2]`, house pattern).
|
||||
2. Tests:
|
||||
- `test_every_known_page_200s_on_conditional_get` — `for path in app.core.caching.HTML_PAGES:` (all of `/`, `/index.html`, `/sources.html`, `/document.html`, `/login.html`, `/tuning.html`, `/git-sources.html`, `/history.html`, `/shared.html`): plain `GET` → 200, `cache-control == "no-cache"`, **no `etag`, no `last-modified`**, body contains `f"?v={asset_version()}"` (the current-process token the middleware used) and at least one `assets/` ref carrying it. Then `GET` the same path with `If-None-Match: <etag from file_validators(frontend/<file>)>` → **200**, same body, same headers. (On the pre-fix app this loop fails on the very first 304.)
|
||||
- `test_dynamic_shared_page_200s_on_conditional_get` — reuse the phase-51 save+share flow from `tests/integration/test_chats_api.py` (admin `client` fixture; `POST /api/chats` with `{messages, share: true}` → the row's `token`): `GET /shared/<token>` → 200, `?v=` refs, no validators; then the same GET with `If-None-Match` derived from `file_validators(FRONTEND / "shared.html")` → **200**, body still rewritten.
|
||||
- `test_assets_keep_immutable_validators_and_304` — `GET /assets/styles.css?v=<token>` → 200, `cache-control == "public, max-age=31536000, immutable"`, `etag` + `last-modified` present; `GET` the same URL with `If-None-Match: <that etag>` → **304** (the versioned-URL 304 stays — the inbound strip must not have widened to `/assets/*`).
|
||||
- `test_api_paths_get_no_cache_headers_and_untouched_stream` — `GET /api/health` → 200, no `cache-control` header injected, no `etag`; `GET /api/health` with `If-None-Match: "x"` → 200 (pass-through, the strip is page-scoped). The SSE stream itself is not re-tested here — the existing `tests/integration/test_chat_api.py` (phase 15/48) pins the byte-identical `/api/chat` pass-through and runs as a regression below.
|
||||
- `test_page_token_matches_process_token` — after `GET /`, the token embedded in `styles.css?v=…` equals `asset_version()` (guards the per-process `functools.cache` contract: one page load can never mix two versions — phase 54 assumption 5).
|
||||
3. Do not modify `tests/conftest.py`, `tests/integration/test_chats_api.py`, or `app/` — this task is tests only; if a pre-existing test breaks, the task is not done.
|
||||
- DB: these tests need the `client` fixture's app to boot, which requires the Postgres DB up (house pattern: `podman compose up -d db` before running, same prerequisite as the other integration files).
|
||||
|
||||
## Testing & Quality
|
||||
- `uv run pytest tests/integration/test_caching_revalidation.py -v` green with the DB up.
|
||||
- Regressions, in isolation: `uv run pytest tests/integration/test_chat_api.py -v` (SSE pass-through) and `uv run pytest tests/integration/test_chats_api.py -v` (the share flow the new test reuses).
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate) — the real-app path now also exercises the middleware branches the unit tests stub.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/integration/test_caching_revalidation.py -v` green (DB up).
|
||||
- [ ] `test_chat_api.py` + `test_chats_api.py` green (regressions).
|
||||
- [ ] `uv run pytest` (full unit + integration) green; coverage TOTAL >90%.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 04 — E2E: the browser can never be 304'd onto stale HTML + docs + commit
|
||||
|
||||
**Phase:** `54_asset_cache_bust_revalidation` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
One isolated Playwright story suite proving, through a real browser, that the document the browser receives is always the current one (200, `?v=<current token>`, no validators), that the CSS that actually renders is the current tree's, and that assets keep their immutable+validator behavior — then the regressions, the README section, and the single commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_asset_cache_revalidation.py` (new) — isolated run, `mock_llm` + `db_ready` prerequisites per house pattern; no KB seeding needed (the suite only loads pages, it asks no questions). The E2E app is a uvicorn subprocess of the SAME checkout (`tests/e2e/conftest.py` → `app_server`), so the expected token is deterministic: `subprocess.run(["git", "-C", str(REPO), "rev-parse", "--short", "HEAD"]).stdout.strip()`.
|
||||
- **`test_document_is_200_current_token_no_validators`** — `with page.expect_response("document") as info:` then `page.goto(app_url)`; `resp = info.value`: `resp.status == 200`; `resp.headers` has **no `etag`, no `last-modified`**; `resp.headers["cache-control"] == "no-cache"`; `resp.text()` contains `f"styles.css?v={token}"` and `f"app.js?v={token}"` (the chat page's two local refs). On the pre-fix app a browser that has seen the page once gets a 304 here instead — this is the suite's core assertion.
|
||||
- **`test_second_navigation_is_still_200_not_304`** — `page.goto(app_url)`, then `with page.expect_response("document") as info: page.goto(app_url)` again (the browser's own cache now holds the first document): second `resp.status == 200` and the body still carries the current token. This is the exact reproduction shape of the reported symptom (a plain re-navigation kept the previous commit's `?v=`).
|
||||
- **`test_browser_renders_current_css_not_stale`** — after `page.goto(app_url)`: `page.evaluate("() => getComputedStyle(document.querySelector('#messages')).flexGrow")` returns `"1"` — the phase-52 rule exists only in the CURRENT `styles.css`, so a green here proves the browser is not sitting on a stale, immutable-pinned asset from an earlier commit (the end-to-end consequence of the 304 hole).
|
||||
- **`test_other_pages_carry_the_contract`** — same document assertions (`200`, no validators, `no-cache`, `?v=` token) for `app_url + "/sources.html"`.
|
||||
- **`test_assets_still_immutable_with_validators`** — `with page.expect_response("**/assets/styles.css**") as info: page.goto(app_url)`; `resp.status == 200`; `resp.headers["cache-control"] == "public, max-age=31536000, immutable"`; `resp.headers` **has** `etag` (the asset keeps its validators — only pages dropped them).
|
||||
2. `README.md` — extend the deployment/caching documentation with a short **Cache busting** subsection (create it if the README has no caching section; place it near the deployment docs): the `?v=<git short SHA>` token flips on the next process start; `/assets/*` is cached `immutable` for a year under the versioned URL (a conditional `GET` may 304 — the URL already encodes the version); HTML pages are served `no-cache` and **never 304** — because the page body is rewritten per process, upstream validators would describe the file, not the bytes served (phase 54); during local development a `git` commit changes the token on server restart — if a browser still shows an old layout, hard-reload once (the fix guarantees the NEXT navigation is current, it cannot un-pin what a pre-54 deploy already 304'd).
|
||||
3. Regressions, each in isolation (`uv run pytest tests/e2e/<file> -v --no-cov`, DB up): `test_cache_busting.py` (the phase-33 story suite — the contract it pins must survive intact), `test_smoke.py`, `test_shared_header.py`, `test_share_chat.py` (the dynamic `/shared/<token>` page), `test_chat_history.py` (`/history.html`).
|
||||
4. One `--no-gpg-sign` commit staging `.agent/ app/ README.md tests/` (message per the phase overview); move `.agent/phases/todo/54_asset_cache_bust_revalidation/` to `.agent/phases/complete/`.
|
||||
- No `app/` change in this task; if one proves necessary, stop and flag it — the fix must already be complete from task 01.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_asset_cache_revalidation.py -v --no-cov` green in isolation (DB up).
|
||||
- The five regression suites above green in isolation.
|
||||
- Final gate before the move: `bash .agent/validate.sh` (unit + integration, coverage >90%, ruff, pyright).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_asset_cache_revalidation.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] `test_cache_busting.py`, `test_smoke.py`, `test_shared_header.py`, `test_share_chat.py`, `test_chat_history.py` green in isolation.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] README documents the no-304-pages / immutable-assets rule and the restart/hard-reload behavior.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Phase 55 — Save by Default, Share Anonymously
|
||||
|
||||
**Source:** `TODO.md` L3–L6 — "Share chat should work anonymously without login" / "Save shouldn't be a button, every chat should be saved by default" / "Need feedback (probably dropdown notification toast) to show share worked" / "New Chat and Share buttons should only be vertically stacked when in mobile, otherwise they should be horizontally next to each other"
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-31)
|
||||
**Context:** Phase 50 stores conversations in `saved_chats` through an explicit Save pill — the `saved_chats` CRUD lives under `/api/chats` in `app/api/chats.py` with a **router-wide** `dependencies=[Depends(require_admin)]` (line ~72), and the pill ships hidden, revealed at boot only when whoami says admin (`frontend/assets/app.js` boot IIFE, line ~1712). Phase 51 added save-then-share (`POST /api/chats` with `share: true` mints the 128-bit `uuid4` token in the same commit; idempotent `POST /{id}/share`) and the public read-only `/shared/<token>` page (already anonymous). The conversation itself lives in localStorage under `bor.chat.v1` (`{ v: 1, messages: [...] }` — **no row link is persisted**; `currentChatId` is module-scope only, so a reload unlinks the conversation). The chat-page pills (`#new-chat-btn`, `#save-chat-btn`, `#share-chat-btn`) are direct children of `.chat-shell` (a vertical flex column, `styles.css` line ~424), so they stack at **every** width. Feedback today is status-line text only (`#send-status`, the live region inside the composer's send button). Phase 53 (todo, preceding) stamps `sources_version` on save and re-saves the linked row after a stale Regenerate — its re-save must keep working once the Save pill is gone.
|
||||
|
||||
## Objective
|
||||
Every conversation on the chat page saves itself (no Save button), **any** visitor can turn the current conversation into a public link without signing in (with a visible toast confirming the share), and the New chat / Share pills sit horizontally on desktop, stacking vertically only on mobile.
|
||||
|
||||
## Dependencies
|
||||
- `53_stale_saved_chats` (todo, preceding) — the save-point machinery and the linked-row re-save (stale Regenerate's auto re-save must keep working after the Save pill is removed; task 02 re-points it at the shared upsert helper if it landed as a direct call).
|
||||
- `54_asset_cache_bust_revalidation` (todo, preceding) — the static-bundle / cache-busting contract the frontend changes ride on (the `?v=` rewrite picks up the changed `styles.css`/`app.js` automatically).
|
||||
- `50_chat_history` + `51_share_chat` (complete) — the `saved_chats` rows, the `/api/chats` surface, the save-then-share contract, the History page, the public `/shared/<token>` page.
|
||||
|
||||
## Tasks
|
||||
1. `01_anonymous_save_share_api.md` — open the save/share write surface to anonymous visitors (list/detail/delete/unshare stay admin-only).
|
||||
2. `02_auto_save_default.md` — retire the Save pill; every conversation auto-upserts at the existing save points and the row link survives reloads (no duplicate rows).
|
||||
3. `03_share_for_everyone.md` — ship the Share pill visible to all visitors; neutral error copy.
|
||||
4. `04_share_toast.md` — the top-right slide-down toast confirming a successful share (both success paths).
|
||||
5. `05_chat_actions_layout.md` — the `.chat-actions` row: horizontal on desktop, stacked at ≤640px.
|
||||
6. `06_e2e_save_share_ux.md` — the story Playwright suite + regressions + the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: `tests/integration/test_chats_api.py` — anonymous create (incl. save-then-share), anonymous update, anonymous share (idempotent); guest 403s on list/detail/delete/unshare; the existing admin pins stay green.
|
||||
- Frontend source pins (house style, `tests/unit/test_save_chat_ui.py` / `test_history_page.py` pattern): `app.js` (headless upsert helper, save-point triggers, `chatId` in the `bor.chat.v1` record, no Save-pill wiring, no admin-gated Share reveal, toast helper), `index.html` (no `#save-chat-btn`, `#share-chat-btn` not `hidden`, the `.chat-actions` wrapper), `styles.css` (`.save-chat-btn` gone, `.chat-actions` base + ≤640px rules, `.toast` + reduced-motion).
|
||||
- **Existing E2E pin adaptation (contract change):** `tests/e2e/test_chat_history.py` (the file-local `_save()` helper + the admin Save-pill assertions + the anonymous "absent" block) and `tests/e2e/test_share_chat.py` (the Save-click step in `test_share_from_history_and_unshare`) — the Save pill is replaced by waiting for the auto-saved row via the admin API; assertion edits limited to the new contract (tasks 02/03).
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_save_share_ux.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Without a session: `POST /api/chats`, `POST /api/chats` with `share: true`, `PUT /api/chats/<id>`, and `POST /api/chats/<id>/share` all succeed; the same guest gets 403 on `GET /api/chats`, `GET /api/chats/<id>`, `DELETE /api/chats/<id>`, and `POST /api/chats/<id>/unshare` (unshare still revokes — the public `/api/shared/<token>` read 404s afterwards).
|
||||
- [ ] The chat page has **no Save control** at any width; a signed-out visitor who sends one question produces exactly one `saved_chats` row (auto-title, both messages); the next brain answer updates the SAME row; a reload at `/` keeps the link (the next message does not create a second row); "New chat" unlinks (a fresh conversation creates a fresh row on its first message).
|
||||
- [ ] A signed-out visitor sees the Share pill; clicking it on a non-empty conversation yields the public link (clipboard or the inline fallback field) **and** a top-right toast; the link opens read-only in a fresh anonymous context; an admin unshare from the History page revokes it.
|
||||
- [ ] Desktop (>640px): the New chat and Share pills share one horizontal row (Share right of New chat); ≤640px: stacked vertically (New chat above Share); no horizontal overflow at 360px.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_save_share_ux.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_share_chat.py`, `test_chat_history.py`, `test_chat_persistence.py`, `test_stale_saved_chats.py`, `test_smoke.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked (2026-08-31, roadmap confirmation):**
|
||||
1. **A1 — the write surface is public.** `POST /api/chats` (create, incl. save-then-share), `PUT /api/chats/<id>`, and `POST /api/chats/<id>/share` require **no session**. Row ids stay unguessable `uuid4` — the same trust model as the share token (the token IS the credential, phase 51). The management surface stays admin-only: `GET /api/chats` (list), `GET /api/chats/<id>` (detail), `DELETE`, and `POST /<id>/unshare` — the owner's History surface. Guest chats appear in the admin's History (saved by default, per L4). **This supersedes the phase-50 owner lock "save/history is admin-only".**
|
||||
2. **A2 — auto-save contract.** Triggers: the first user message creates the row (auto-title as today); every brain-done save point and the pagehide partial update it. A failed auto-save **never blocks the conversation** — a one-line status note only (no error banner), retried at the next save point. Successful auto-saves are silent (the History page is the visible proof; the toast is reserved for share, per L5).
|
||||
3. **A3 — `/?chat=<id>` boot restore stays admin-only** (History "Open"); guests keep the localStorage restore exactly as today.
|
||||
4. **A4 — the toast.** Top-right, slides down, auto-dismisses ~4s, a single instance (a new toast replaces a pending one). **Visual only** (`aria-hidden`) — the existing `#send-status` live region remains the a11y announcer (no double screen-reader read). Shown on BOTH share-success paths (clipboard copied / fallback field). Never on failure (the error banner is the failure UI).
|
||||
5. **A5 — the layout.** A `.chat-actions` wrapper around New chat + Share; horizontal row on desktop, vertical stack at the existing 640px breakpoint; pill order New chat → Share in both orientations; the existing ≤640px pill rules (padding, icon/label handling, the `.chat-shell` label overrides) stay and apply to the stacked pills.
|
||||
- **A10 honoured** — `/api/chat` stays stateless; auto-save writes the `saved_chats` row, not the chat endpoint.
|
||||
- **A16/A17 honoured** — one dedicated story E2E suite, one atomic commit.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A app/ frontend/ tests/ && git add -f .agent/phases/todo/55_save_share_ux .agent/phases/complete && git commit --no-gpg-sign -m "feat(chat): save by default + share anonymously — auto-saved chats, guest-facing Share, success toast, action row"
|
||||
```
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 06 — E2E: save-by-default + anonymous share + regressions + commit
|
||||
|
||||
**Phase:** `55_save_share_ux` · **Source:** `TODO.md:3–6` — all four items (this task verifies the full loop in the browser)
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
One isolated Playwright story suite for the whole phase: auto-save without a button, share without login (with the toast), and the action-row layout at both breakpoints — plus the regression gate and the single atomic commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_save_share_ux.py` (new) — the story suite (isolated run; `mock_llm` deterministic; real DB per the e2e prerequisite; DB isolation follows the `test_share_chat.py` header pattern — the shared e2e Postgres keeps `saved_chats` rows between tests, so every row lookup selects by auto-title via the file's own `_chats`/`_find_row`-style helpers, and each test `_reset_db`s first):
|
||||
- **Anonymous auto-save (L4):** fresh anonymous context (no login) → `/` → assert **no** `#save-chat-btn` in the DOM → ask a question (mock answer) → wait for the turn to settle → `GET /api/chats` (admin cookies, file-local helper) reports exactly one row with the question's auto-title carrying both messages. No button press anywhere in this test.
|
||||
- **No duplicate across reload (L4):** same anonymous context → `page.reload()` → the conversation is restored from localStorage (both bubbles visible) → ask a second question → the row count for the auto-title is still **one**, and its message count grew by two (the link survived the reload — task 02's `chatId` persistence).
|
||||
- **Anonymous share + toast (L3 + L5):** fresh anonymous context → `/` → the Share pill is **visible without login** → ask a question → grant clipboard (the file's `_grant_clipboard` pattern) → click `#share-chat-btn` → the `.toast` becomes visible with the success text and auto-dismisses within ~5s → read the link (clipboard via Playwright, or the `.share-link-fallback` field text if the origin rejects the clipboard) → a **fresh incognito context** opens the `/shared/<token>` link → the conversation renders read-only (title + both bubbles, no composer/pills — the phase-51 zero-controls surface) → from an **admin** session, unshare via the History page's Unshare button → the same URL now shows the "invalid or revoked" state.
|
||||
- **Layout (L6):** desktop viewport (1280×800): `#new-chat-btn` and `#share-chat-btn` bounding boxes on one row (overlapping `y` bands; Share's `x` > New chat's `x` + width; each at intrinsic width, not the full 46rem column). Mobile viewport (390×844, `page.set_viewport_size`): stacked (Share's `y` > New chat's `y` + height). At 360px wide: no horizontal overflow (`document.documentElement.scrollWidth` ≤ 360).
|
||||
- **Admin still works (A1 sanity):** admin login → `/` → ask a question → the auto-saved row appears for the admin too (same machinery, session or not).
|
||||
2. Regressions, each in isolation (`uv run pytest tests/e2e/<file> -v --no-cov`, DB up): `test_share_chat.py` (Save-click pins already adapted in task 02), `test_chat_history.py` (auto-save pins adapted in task 02 + the anonymous Share-pill pin from task 03), `test_chat_persistence.py`, `test_stale_saved_chats.py` (phase 53 — the stale Regenerate's auto re-save must survive task 02's helper rename; if it still references the removed Save pill or `saveCurrentChat` by name, adapt those pins to the auto-save contract, edits limited to the rename/removal), `test_smoke.py`.
|
||||
3. One `--no-gpg-sign` commit staging `app/ frontend/ tests/` + the phase dir force-added (`.agent/` is gitignored by design — AGENTS.md rule 8: `git add -f .agent/…`):
|
||||
```bash
|
||||
git add -A app/ frontend/ tests/ && git add -f .agent/phases/complete/55_save_share_ux && git commit --no-gpg-sign -m "feat(chat): save by default + share anonymously — auto-saved chats, guest-facing Share, success toast, action row"
|
||||
```
|
||||
then move `.agent/phases/todo/55_save_share_ux/` → `.agent/phases/complete/55_save_share_ux/` (stage the move with `git add -f` on the new path before committing, so the tracked phase files land under `complete/`).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E (mandatory, A16): `tests/e2e/test_save_share_ux.py` green in isolation.
|
||||
- The five regression suites green in isolation; no assertion edits outside the new contract (phase-48/49/51 behavior pins — streaming, retry, the shared page's zero controls, unshare revocation — stay intact).
|
||||
- Full gate: `uv run pytest` + `uv run pytest --cov=app --cov-report=term-missing` (TOTAL >90%) + `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_save_share_ux.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] `test_share_chat.py`, `test_chat_history.py`, `test_chat_persistence.py`, `test_stale_saved_chats.py`, `test_smoke.py` green in isolation.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Phase 56 — User-Extensible Import Extensions
|
||||
|
||||
**Source:** `TODO.md` L6 — "Allow the user to specify extensions to be read in .env, don't hard-code working extensions. Supply some by default in .env.example"
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-31)
|
||||
**Context:** `BOR_IMPORT_EXTENSIONS` already exists: the `import_extensions` CSV field in `app/config.py` (default = the A9 family), the `import_extension_set` property, the `_import_extensions_known` validator (`app/config.py:172`), and a commented default line in `.env.example`. The one gap versus the TODO is the A9 lock: the allowed set is hard-coded in `_ALLOWED_IMPORT_EXTENSIONS` (`app/config.py:20`), so the env var may only **narrow** the family, never **add** an extension ("may only narrow, never widen" — `app/config.py` module docstring, `.env.example` "Import scope" comment). The importer filters with `path.suffix.lower() in extensions` (`app/rag/importer.py:137`); non-markdown formats are chunked as plain text and (phase 30) get a `lite` summary, so any new extension flows through the existing pipeline with no importer changes.
|
||||
|
||||
## Objective
|
||||
`BOR_IMPORT_EXTENSIONS` may name **any** extension (the A9 family becomes the built-in default, not the ceiling), fail-loud validation is kept for typos, and `.env.example` ships the default list as a usable, documented example.
|
||||
|
||||
## Dependencies
|
||||
- — (none)
|
||||
|
||||
## Tasks
|
||||
1. `01_lift_extension_allowlist.md` — config change: A9 family becomes the default, the validator accepts any well-formed extension, `.env.example` documents it.
|
||||
2. `02_e2e_extensions_env.md` — integration + Playwright proof (a novel `.sh` extension imports end-to-end), regressions, atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_config*.py` — the validator now **accepts** novel extensions (`md,sh` → `{'.md', '.sh'}`), still rejects an empty list and malformed tokens; default parsing unchanged.
|
||||
- Integration: `tests/integration/test_import_extensions_env.py` — in-process `import_sources` against a story-dedicated fixture dir with `import_extensions="md,sh"` (novel extension imported, plain-text chunked, mock summary) and `import_extensions="md"` (narrowing still works).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_import_extensions_env.py`, run in isolation (DB up) — Sources page (admin) lists the novel-extension doc with its format badge; anonymous still gets the sign-in gate.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `BOR_IMPORT_EXTENSIONS=md,sh` (or any other well-formed novel extension) starts the app and imports `.sh` files; the Sources page shows them with format badge `sh`.
|
||||
- [ ] `BOR_IMPORT_EXTENSIONS=` (empty) or a malformed token (e.g. `md,sh!`) fails loudly at startup with a named value.
|
||||
- [ ] `.env.example` carries the default list (A9 family + quadlet + jinja) with a comment stating any extension is allowed.
|
||||
- [ ] Existing suites stay green: `test_import_documents.py`, `test_quadlet_jinja_import.py` (isolated runs).
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_import_extensions_env.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/` (`.agent/` stays untracked — owner instruction, commit 281f355).
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked (2026-08-31, roadmap confirmation, D1):** `BOR_IMPORT_EXTENSIONS` may **extend** beyond the A9 family — the hard-coded set becomes the default, not the ceiling. This revises the A9 "narrow-only" clause (2026-08-21/27) by owner permission; the A9 list remains the built-in default and the documented example.
|
||||
- **Fail-loud kept (house style):** empty list and malformed tokens are rejected at startup (validator), so a typo never walks zero files silently.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add app/ tests/ .env.example && git commit --no-gpg-sign -m "feat(import): user-extensible BOR_IMPORT_EXTENSIONS — any well-formed extension, A9 family stays the default"
|
||||
```
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 01 — Lift the extension allowlist to a default
|
||||
|
||||
**Phase:** `56_import_extensions_env` · **Source:** `TODO.md:6` — "Allow the user to specify extensions to be read in .env, don't hard-code working extensions. Supply some by default in .env.example"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
`app/config.py` treats the A9 family as the **default** import scope and accepts any well-formed extension named in `BOR_IMPORT_EXTENSIONS`; `.env.example` documents the setting with the default list.
|
||||
|
||||
## Work
|
||||
1. `app/config.py`:
|
||||
- Rename `_ALLOWED_IMPORT_EXTENSIONS` → `_DEFAULT_IMPORT_EXTENSIONS` (same A9 list, incl. the quadlet family + `j2`). Update its comment: this is the built-in default and the `.env.example` example — no longer a ceiling.
|
||||
- Rewrite the module docstring and the field comment around `import_extensions` (currently "may only narrow, never widen" — module header ~L12–19, field docstring ~L146): the user may name **any** extension; the default shown is the A9 family.
|
||||
- Rewrite the validator `_import_extensions_known`: parse parts (strip, `lstrip(".")`, lowercase); reject an empty result (`"import_extensions must name at least one format"`); reject any token that does not match `^[a-z0-9]{1,16}$` with a `ValueError` naming the offending token(s) (fail loud, `agent_max_rounds`-style). Delete the membership test against the fixed set.
|
||||
- `ASSUMPTION:` the token shape guard `^[a-z0-9]{1,16}$` is the typo guard — it keeps path-ish values (`../x`, `/etc/passwd`, `sh!`) out of the set while still allowing anything a file could actually be suffixed with.
|
||||
2. `.env.example`: uncomment the `BOR_IMPORT_EXTENSIONS=…` line (full default list) and rewrite the "Import scope" section comment: extensions the importer reads; **any** extension is allowed (lowercase, no dot); the value shown is the built-in default (A9 family + quadlet + jinja).
|
||||
3. Sweep stale references: `rg -n "narrow|_ALLOWED_IMPORT_EXTENSIONS" app/ scripts/ frontend/ tests/` — update every hit that still claims the set can only narrow (known candidates: `app/config.py` header, `scripts/import_docs.py` docstrings/comments, `app/rag/importer.py` docstring).
|
||||
4. Adjust the config unit tests that assert unknown-extension rejection (they now assert acceptance of novel extensions — see Testing & Quality).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/test_config*.py`, adapt in place):
|
||||
- default value parses to the full A9 family set (unchanged pin).
|
||||
- `import_extensions="md,sh"` → set `{".md", ".sh"}` (novel extension accepted).
|
||||
- `"MD,.Py"` normalizes to `{".md", ".py"}` (case + leading-dot tolerance unchanged).
|
||||
- `""` / `",,"` → startup `ValueError` naming the field.
|
||||
- `"md,sh!"` (and `"md,../x"`) → `ValueError` naming the malformed token.
|
||||
- Existing importer unit tests stay green (they exercise `iter_importable_files` with explicit sets — no behavior change for a fixed set).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Validator accepts novel extensions, rejects empty/malformed, with named-value error messages.
|
||||
- [ ] `_DEFAULT_IMPORT_EXTENSIONS` is the only surviving constant; no "narrow-only" wording remains anywhere.
|
||||
- [ ] `.env.example` ships the default list as an active, documented example.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 02 — Import proof (novel extension) + E2E + commit
|
||||
|
||||
**Phase:** `56_import_extensions_env` · **Source:** `TODO.md:6` — "Allow the user to specify extensions to be read in .env, don't hard-code working extensions. Supply some by default in .env.example"
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove a **novel** (non-A9) extension flows end-to-end — config → import → chunks → mock summary → Sources page — run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. Story-dedicated fixture dir `tests/fixtures/extension_kb/` (house pattern: `summary_kb/` keeps the shared `docs/` fixtures pinned):
|
||||
- `homelab/scripts/uptime.sh` — a handful of distinctive lines incl. a sentinel word (e.g. `UPTIME-PROBE-SENTINEL-9c2f`) so retrieval/row assertions are unambiguous.
|
||||
- `homelab/notes/note.md` — a small markdown control doc.
|
||||
2. `tests/integration/test_import_extensions_env.py` (in-process, mock LLM port — the `_import_fixtures` pattern from `tests/e2e/test_import_documents.py`):
|
||||
- `Settings(_env_file=None, llm_base_url=<mock>, import_extensions="md,sh")` → `import_sources([FIXTURE], LLMClient(settings))` → a `homelab/scripts/uptime.sh` row with format `sh`, non-empty `summary` (mock `SUMMARY_MODE` digest), chunked; `note.md` imported as control.
|
||||
- Same fixture with `import_extensions="md"` → **no** `.sh` row (narrowing still works — the A9-era behavior is preserved as a special case).
|
||||
3. `tests/e2e/test_import_extensions_env.py` (Playwright, DB up, mock LLM; admin login via `e2e.auth_helpers.login`):
|
||||
- Seed the same fixture in-process with `import_extensions="md,sh"` (the `test_import_documents.py` seeding thread pattern — the fixture, not the subject).
|
||||
- Admin: `/sources.html` lists `homelab/scripts/uptime.sh` with format badge `sh` (the row's path cell + badge, house assertion style).
|
||||
- Anonymous fresh context: the Sources gate renders and **no** `/api/docs` request fires (`page.on("request")` pin — the `test_import_documents.py`/phase-16 pattern).
|
||||
- DB isolation: the fixture's `source` name (`extension_kb`) is distinctive — never assert on absolute row counts; delete the rows it creates in a `finally` (admin cookie).
|
||||
4. Regression pass (isolation runs): `test_import_documents.py`, `test_quadlet_jinja_import.py` (both pin the A9 family with the **default** config — must be byte-for-byte unchanged in behavior).
|
||||
5. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
6. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section; move `.agent/phases/todo/56_import_extensions_env/` → `.agent/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_import_extensions_env.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A `.sh` file imports with `import_extensions="md,sh"` (row, summary, chunks) and is visible on the admin Sources page with badge `sh`.
|
||||
- [ ] `import_extensions="md"` excludes the `.sh` file (narrowing unchanged).
|
||||
- [ ] Anonymous Sources gate regression holds (no `/api/docs` call).
|
||||
- [ ] Regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user