feat: scaffold Brain of Reese — FastAPI RAG chat over Postgres 17 + pgvector
Foundation (phase 01, verified): - FastAPI app: /api/health, /api/suggestions, /api/chat (placeholder), static frontend served locally (no CDN) - Postgres 17 + pgvector via db/Containerfile + compose.yaml (podman compose up -d db), Alembic initial migration (documents, chunks with vector(768), query_log) - LLM client targeting https://aipi.reeseapps.com/v1 (turbo/embed); scripts/llm_probe.py verified models + 768-dim embeddings live - Conditional debugpy: imported only when DEBUGPY=1 (attach on demand, :5678); logging config for clean single-line logs - Frontend shell: mobile-first chat + Sources pages, tokens, a11y baselines - Tests: 24 unit+integration (99% coverage on app/), ruff + pyright clean, Playwright smoke E2E (3 tests) against a deterministic mock LLM - Planning: .agent/PLAN.md (architecture + LOCKED decisions), AGENTS.md, 6 user stories, 7 phase files (one story / one phase / one Playwright suite each)
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
# Phase 04 — Story: Honest Deflection
|
||||
|
||||
**Story:** `.agent/user_stories/honest-deflection.md`
|
||||
**Context:** `.agent/PLAN.md` §4, §6 (honesty gate), §9
|
||||
|
||||
## Goal
|
||||
When retrieval finds nothing relevant, Brain says so — plainly, chippily —
|
||||
and offers real alternatives. No hallucinated confidence.
|
||||
|
||||
## Implementation steps
|
||||
1. `app/api/chat.py` — apply the gate: `best_score < settings.relevance_
|
||||
threshold` ⇒ build LOW prompt (`DEFLECT_MODE`, weak-hit titles only),
|
||||
else HIGH prompt. Set `deflected` on the `done` event + `query_log`.
|
||||
2. Deflection `suggestions[]`: ask `turbo` (same stream) to include 2–3
|
||||
alternative questions; simplest robust approach — have the LLM emit them
|
||||
inline in the answer AND have the server derive 2–3 chips from the
|
||||
weak-hit document titles (deterministic fallback if the model doesn't
|
||||
produce a parsable list). Ship the deterministic title-derived chips as
|
||||
the v1 behavior; model-generated list is a bonus if trivially parseable.
|
||||
3. `frontend/assets/app.js` — on `done.deflected`: add `.is-deflected`
|
||||
class to the bubble, render "Maybe try:" chips below it (same
|
||||
`.suggestion-chip` component; clicking fills the input — full submit
|
||||
behavior lands with Phase 05's chip component; wire what exists).
|
||||
4. `README.md` — document `BOR_RELEVANCE_THRESHOLD` tuning + the
|
||||
deflection behavior in Troubleshooting.
|
||||
|
||||
## UI Verification
|
||||
Against the story: amber bubble (`#fff7e8` bg / `#f59e0b` border) distinct
|
||||
from normal answers; "Maybe try:" chips ≥44px, brand-soft/brand-ink;
|
||||
contrast pairs verified (ink on accent-bg ≥ 9:1, accent-ink ≥ 8:1);
|
||||
chip group has an accessible name; mobile wraps cleanly.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: gate boundary with a fake retriever — score exactly 0.30 → HIGH;
|
||||
0.2999 → LOW; LOW prompt contains `DEFLECT_MODE` + titles, no full docs;
|
||||
HIGH unaffected. Suggestions derivation (2–3, non-empty, derived from
|
||||
titles).
|
||||
- Integration: mock LLM — off-topic question ("sourdough") ⇒ `done`
|
||||
`deflected: true`, `query_log.deflected=true`, weak `top_score` stored;
|
||||
on-topic question ⇒ `deflected: false`.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Playwright Execution Phase
|
||||
Run ONLY this story's suite:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov
|
||||
```
|
||||
|
||||
Implements the story mapping: off-topic question ⇒ `.is-deflected` bubble
|
||||
matching /haven't done anything like that/i + ≥2 "Maybe try:" chips; chip
|
||||
click behavior; (unit boundary test lives in pytest, not here).
|
||||
|
||||
## Success criteria
|
||||
- [ ] off-topic question never gets a confident fake answer
|
||||
- [ ] deflected bubble visually distinct + alternative chips render
|
||||
- [ ] `query_log.deflected` accurate; threshold env-tunable
|
||||
- [ ] unit + integration green, coverage >90%
|
||||
- [ ] UI verification passed
|
||||
- [ ] story E2E green in isolation
|
||||
- [ ] committed
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A && git commit --no-gpg-sign -m "feat(rag): honest deflection gate with amber UI state and alternative-question chips"
|
||||
```
|
||||
Reference in New Issue
Block a user