Files
brain-of-reese/.agent/user_stories/chat-rag-answer.md
T
ducoterra 022da8e2bc feat: scaffold Brain of Reese — FastAPI RAG chat over Postgres 17 + pgvector
Foundation (phase 01, verified):
- FastAPI app: /api/health, /api/suggestions, /api/chat (placeholder),
  static frontend served locally (no CDN)
- Postgres 17 + pgvector via db/Containerfile + compose.yaml
  (podman compose up -d db), Alembic initial migration (documents,
  chunks with vector(768), query_log)
- LLM client targeting https://aipi.reeseapps.com/v1 (turbo/embed);
  scripts/llm_probe.py verified models + 768-dim embeddings live
- Conditional debugpy: imported only when DEBUGPY=1 (attach on demand,
  :5678); logging config for clean single-line logs
- Frontend shell: mobile-first chat + Sources pages, tokens, a11y baselines
- Tests: 24 unit+integration (99% coverage on app/), ruff + pyright clean,
  Playwright smoke E2E (3 tests) against a deterministic mock LLM
- Planning: .agent/PLAN.md (architecture + LOCKED decisions), AGENTS.md,
  6 user stories, 7 phase files (one story / one phase / one Playwright
  suite each)
2026-08-21 13:42:21 -04:00

3.3 KiB
Raw Blame History

Story: Chat RAG Answer (happy path)

Phase: 03_story_chat_rag.md · E2E: tests/e2e/test_chat_rag.py

Narrative

As a user (friend, colleague, future me), I want to ask Brain a question about Reese's setup and get a grounded, chippy answer that points me at the exact documentation — so I can actually do the thing.

  • Given the knowledge base is imported and I type "How is my Kubernetes cluster set up?"
  • When Brain embeds the question, retrieves the top chunks by cosine similarity, maps them to their parent documents, and feeds the full document text to turbo
  • Then I see a streamed, upbeat answer that cites the source (Homelab/kubernetes.md), grounded in the doc's specifics (Talos, Cilium, the node list) — and never in anything the docs don't say.

Acceptance criteria

  1. POST /api/chat streams SSE: delta events then a final done event carrying {deflected, sources[], suggestions[]} (PLAN §4).
  2. Retrieval: top-4 chunks (BOR_TOP_K_CHUNKS), cosine via pgvector <=>, score = 1 − distance.
  3. Context assembly: top-2 distinct documents by best-chunk score, full content, capped at BOR_MAX_CONTEXT_CHARS with truncation marker.
  4. System prompt = locked persona + HONESTY GATE rules (PLAN §6), with <relevance>HIGH</relevance> and <documents>…</documents>.
  5. The answer arrives streamed (multiple deltas), rendered live.
  6. Source chips (mono, source/path) render under the answer bubble.
  7. Per-turn log line emitted (PLAN §9) and a query_log row inserted (deflected=false, top_score, sources, latency).
  8. LLM/embedding failure → JSON/SSE error the UI turns into the error banner (no hang, no stale button).

UI Visualization & Structure

  • Chat column centered at 46rem (PLAN §7.1); user bubble right (brand indigo, white text ≥4.5:1), Brain bubble left (white, ink text, avatar 🧠).
  • While generating: typing indicator → live-appended text (see loading-feedback story for the full state machine — this story only needs "deltas render as they arrive and the button is busy throughout").
  • Source chips: pill, font-family: mono, bg --brand-soft, color --brand-ink (6.3:1), max-width + ellipsis; each shows Homelab/kubernetes.md. aria-label when truncated.
  • Bubble content is safe-rendered markdown (escape-first local renderer — <script> in an LLM answer must NOT execute).
  • On mobile the bubbles expand to ~92% width; chips wrap.

Playwright Mapping Rule

Test Scenario → tests/e2e/test_chat_rag.py (mock LLM, seeded KB):

  1. test_on_topic_question_streams_grounded_answer — type "How is my Kubernetes cluster set up?", submit; assert: answer bubble appears with streamed content (mock's answer references the question), a .source-chip containing kubernetes.md is present, send button returns to enabled "Send".
  2. test_chat_logs_query — after the turn, GET /api/health is still ok AND (via a test-only detail: query the DB directly) a query_log row exists with deflected=false and sources including kubernetes.md.
  3. test_sse_stream_shape — raw httpx streaming request to /api/chat: assert multiple data: delta events precede a done event with deflected: false and a non-empty sources list.