feat(rag): honest deflection gate with amber UI state and alternative-question chips
This commit is contained in:
@@ -210,9 +210,23 @@ served locally (no CDN), `BOR_ENVIRONMENT=production`.
|
||||
`uv run python -m scripts.llm_probe`, update `BOR_EMBEDDING_DIM`, then
|
||||
drop + recreate the chunks table (new migration or manual `TRUNCATE
|
||||
chunks, documents`).
|
||||
- **Honest deflection (the amber “I haven't done anything like that”
|
||||
bubble)** — every question passes the honesty gate: when the best
|
||||
cosine similarity is below `BOR_RELEVANCE_THRESHOLD` (default `0.30`),
|
||||
Brain switches to deflection mode instead of guessing. The LLM prompt
|
||||
then carries weak-hit *titles only* (no document content), the reply
|
||||
opens with “I haven't done anything like that”, the bubble renders
|
||||
amber with “Maybe try” chips derived from the closest indexed titles,
|
||||
the SSE `done` event carries `deflected: true` + `suggestions[]`, and
|
||||
the `query_log` row records `deflected=true` + the weak `top_score`.
|
||||
This is a feature, not a bug — the KB simply has no notes that close;
|
||||
the chips always point at topics Brain really covers.
|
||||
- **Answers deflect too often / too rarely** — tune
|
||||
`BOR_RELEVANCE_THRESHOLD` (lower = answers more, higher = more honest
|
||||
deflection). Check `query_log` for the actual scores:
|
||||
deflection): `0.0` ⇒ every question gets answered, even unknown topics
|
||||
(expect confident-sounding guesses); `1.0` ⇒ everything deflects
|
||||
(nothing but a perfect 1.0 score counts as relevant). After changing
|
||||
it, check the real scores:
|
||||
`psql … -c 'SELECT question, top_score, deflected FROM query_log ORDER BY created_at DESC LIMIT 20'`
|
||||
- **KB offline banner in the chat** — Postgres isn't running:
|
||||
`podman compose up -d db`.
|
||||
|
||||
Reference in New Issue
Block a user