Files

T

ducoterra 022da8e2bc feat: scaffold Brain of Reese — FastAPI RAG chat over Postgres 17 + pgvector

Foundation (phase 01, verified):
- FastAPI app: /api/health, /api/suggestions, /api/chat (placeholder),
  static frontend served locally (no CDN)
- Postgres 17 + pgvector via db/Containerfile + compose.yaml
  (podman compose up -d db), Alembic initial migration (documents,
  chunks with vector(768), query_log)
- LLM client targeting https://aipi.reeseapps.com/v1 (turbo/embed);
  scripts/llm_probe.py verified models + 768-dim embeddings live
- Conditional debugpy: imported only when DEBUGPY=1 (attach on demand,
  :5678); logging config for clean single-line logs
- Frontend shell: mobile-first chat + Sources pages, tokens, a11y baselines
- Tests: 24 unit+integration (99% coverage on app/), ruff + pyright clean,
  Playwright smoke E2E (3 tests) against a deterministic mock LLM
- Planning: .agent/PLAN.md (architecture + LOCKED decisions), AGENTS.md,
  6 user stories, 7 phase files (one story / one phase / one Playwright
  suite each)

2026-08-21 13:42:21 -04:00

2.9 KiB

Raw Blame History

Phase 04 — Story: Honest Deflection

Story: .agent/user_stories/honest-deflection.md Context: .agent/PLAN.md §4, §6 (honesty gate), §9

Goal

When retrieval finds nothing relevant, Brain says so — plainly, chippily — and offers real alternatives. No hallucinated confidence.

Implementation steps

app/api/chat.py — apply the gate: best_score < settings.relevance_ threshold ⇒ build LOW prompt (DEFLECT_MODE, weak-hit titles only), else HIGH prompt. Set deflected on the done event + query_log.
Deflection suggestions[]: ask turbo (same stream) to include 2–3 alternative questions; simplest robust approach — have the LLM emit them inline in the answer AND have the server derive 2–3 chips from the weak-hit document titles (deterministic fallback if the model doesn't produce a parsable list). Ship the deterministic title-derived chips as the v1 behavior; model-generated list is a bonus if trivially parseable.
frontend/assets/app.js — on done.deflected: add .is-deflected class to the bubble, render "Maybe try:" chips below it (same .suggestion-chip component; clicking fills the input — full submit behavior lands with Phase 05's chip component; wire what exists).
README.md — document BOR_RELEVANCE_THRESHOLD tuning + the deflection behavior in Troubleshooting.

UI Verification

Against the story: amber bubble (#fff7e8 bg / #f59e0b border) distinct from normal answers; "Maybe try:" chips ≥44px, brand-soft/brand-ink; contrast pairs verified (ink on accent-bg ≥ 9:1, accent-ink ≥ 8:1); chip group has an accessible name; mobile wraps cleanly.

Testing & Quality

Unit: gate boundary with a fake retriever — score exactly 0.30 → HIGH; 0.2999 → LOW; LOW prompt contains DEFLECT_MODE + titles, no full docs; HIGH unaffected. Suggestions derivation (2–3, non-empty, derived from titles).
Integration: mock LLM — off-topic question ("sourdough") ⇒ done deflected: true, query_log.deflected=true, weak top_score stored; on-topic question ⇒ deflected: false.
Coverage: >90% on app/.

Playwright Execution Phase

Run ONLY this story's suite:

uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov

Implements the story mapping: off-topic question ⇒ .is-deflected bubble matching /haven't done anything like that/i + ≥2 "Maybe try:" chips; chip click behavior; (unit boundary test lives in pytest, not here).

Success criteria

off-topic question never gets a confident fake answer
deflected bubble visually distinct + alternative chips render
query_log.deflected accurate; threshold env-tunable
unit + integration green, coverage >90%
UI verification passed
story E2E green in isolation
committed

Commit

git add -A && git commit --no-gpg-sign -m "feat(rag): honest deflection gate with amber UI state and alternative-question chips"

2.9 KiB Raw Blame History Unescape Escape