feat(rag): honest deflection gate with amber UI state and alternative-question chips

This commit is contained in:
2026-08-21 17:50:33 -04:00
parent 396e4d47fb
commit cbf8e39e63
11 changed files with 779 additions and 34 deletions
+15 -1
View File
@@ -210,9 +210,23 @@ served locally (no CDN), `BOR_ENVIRONMENT=production`.
`uv run python -m scripts.llm_probe`, update `BOR_EMBEDDING_DIM`, then
drop + recreate the chunks table (new migration or manual `TRUNCATE
chunks, documents`).
- **Honest deflection (the amber “I haven't done anything like that”
bubble)** — every question passes the honesty gate: when the best
cosine similarity is below `BOR_RELEVANCE_THRESHOLD` (default `0.30`),
Brain switches to deflection mode instead of guessing. The LLM prompt
then carries weak-hit *titles only* (no document content), the reply
opens with “I haven't done anything like that”, the bubble renders
amber with “Maybe try” chips derived from the closest indexed titles,
the SSE `done` event carries `deflected: true` + `suggestions[]`, and
the `query_log` row records `deflected=true` + the weak `top_score`.
This is a feature, not a bug — the KB simply has no notes that close;
the chips always point at topics Brain really covers.
- **Answers deflect too often / too rarely** — tune
`BOR_RELEVANCE_THRESHOLD` (lower = answers more, higher = more honest
deflection). Check `query_log` for the actual scores:
deflection): `0.0` ⇒ every question gets answered, even unknown topics
(expect confident-sounding guesses); `1.0` ⇒ everything deflects
(nothing but a perfect 1.0 score counts as relevant). After changing
it, check the real scores:
`psql … -c 'SELECT question, top_score, deflected FROM query_log ORDER BY created_at DESC LIMIT 20'`
- **KB offline banner in the chat** — Postgres isn't running:
`podman compose up -d db`.