Phase 04 — Story: Honest Deflection

Story: .agent/user_stories/honest-deflection.md Context: .agent/PLAN.md §4, §6 (honesty gate), §9

Goal

When retrieval finds nothing relevant, Brain says so — plainly, chippily — and offers real alternatives. No hallucinated confidence.

Implementation steps

app/api/chat.py — apply the gate: best_score < settings.relevance_ threshold ⇒ build LOW prompt (DEFLECT_MODE, weak-hit titles only), else HIGH prompt. Set deflected on the done event + query_log.
Deflection suggestions[]: ask turbo (same stream) to include 2–3 alternative questions; simplest robust approach — have the LLM emit them inline in the answer AND have the server derive 2–3 chips from the weak-hit document titles (deterministic fallback if the model doesn't produce a parsable list). Ship the deterministic title-derived chips as the v1 behavior; model-generated list is a bonus if trivially parseable.
frontend/assets/app.js — on done.deflected: add .is-deflected class to the bubble, render "Maybe try:" chips below it (same .suggestion-chip component; clicking fills the input — full submit behavior lands with Phase 05's chip component; wire what exists).
README.md — document BOR_RELEVANCE_THRESHOLD tuning + the deflection behavior in Troubleshooting.

UI Verification

Against the story: amber bubble (#fff7e8 bg / #f59e0b border) distinct from normal answers; "Maybe try:" chips ≥44px, brand-soft/brand-ink; contrast pairs verified (ink on accent-bg ≥ 9:1, accent-ink ≥ 8:1); chip group has an accessible name; mobile wraps cleanly.

Testing & Quality

Unit: gate boundary with a fake retriever — score exactly 0.30 → HIGH; 0.2999 → LOW; LOW prompt contains DEFLECT_MODE + titles, no full docs; HIGH unaffected. Suggestions derivation (2–3, non-empty, derived from titles).
Integration: mock LLM — off-topic question ("sourdough") ⇒ done deflected: true, query_log.deflected=true, weak top_score stored; on-topic question ⇒ deflected: false.
Coverage: >90% on app/.

Playwright Execution Phase

Run ONLY this story's suite:

uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov

Implements the story mapping: off-topic question ⇒ .is-deflected bubble matching /haven't done anything like that/i + ≥2 "Maybe try:" chips; chip click behavior; (unit boundary test lives in pytest, not here).

Success criteria

off-topic question never gets a confident fake answer
deflected bubble visually distinct + alternative chips render
query_log.deflected accurate; threshold env-tunable
unit + integration green, coverage >90%
UI verification passed
story E2E green in isolation
committed

Commit

git add -A && git commit --no-gpg-sign -m "feat(rag): honest deflection gate with amber UI state and alternative-question chips"

2.9 KiB Raw Blame History Unescape Escape