Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched.
4.2 KiB
4.2 KiB
Phase 15 — Tune How Brain Answers (Steering Notes)
Story: .agents/user_stories/steering-notes.md
Context: owner report 2026-08-22 — "the answers are off. Add a
feature that lets me 'tune' the output if I think an answer isn't quite
right. This tuning should be added to the database and read in with the
system prompt to steer the replies."
Goal
A first-class steering loop: under any answer, "Tune" → short
instruction → stored in Postgres (steering_notes) → injected into the
system prompt of every subsequent turn → observable in the reply.
Notes are listed and deletable in a "Tuning" panel.
Implementation steps
- Schema —
alembic/versions/0003_steering_notes.py:steering_notes(id UUID PK, note TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now()); downgrade drops the table. ModelSteeringNoteinapp/models.py. - API —
app/api/steering.py(stateless, A10):GET /api/steering→{notes: [{id, note, created_at}]}newest first.POST /api/steering{note}(trimmed, 1–2000 chars; 422 on empty/over-long) → 201 with the created note.DELETE /api/steering/{note_id}→ 204; 404 unknown id. Mount inapp/main.py; schemas inapp/schemas.py.
- Prompt (
app/rag/prompts.py):build_steering_section(notes: Sequence[str]) -> str—""when empty; else<tuning>…</tuning>with numbered notes, capped atsteering_max_chars(new setting, default 8000;[…truncated…]marker). Both HIGH and DEFLECT prompts carry it (after<relevance>…</relevance>, before<documents>/DEFLECT_MODE body). With zero notes the prompt is byte-identical to today.- Preserve the owner's working-tree persona edits (no "you've got
this" / no mandated deflection opening — align
tests/unit/test_prompts.pyverbatim check with the current text; record as a PLAN §6 revision).
- Chat turn (
app/api/chat.py): load notes (created_at asc), pass intoplan_turn→ prompts; per-turn log line gainstuning=N(PLAN §9). - E2E mock (
tests/e2e/mock_llm.py): when the system prompt contains<tuning>, the composed answer ends with(tuning: <first note line>)— makes prompt injection observable in the UI deterministically. - UI (
frontend/index.html,app.js,styles.css):- "Tune" button (
.tune-btn, ghost, ≥44px) in the meta row of every completed brain bubble (deflected included). - Inline
.tune-form: labeled textarea (maxlength 2000) + Save / Cancel →POST /api/steering→ success.tune-saved(role=status: "Saved — future answers will follow this.") or inline error (role=alert), form kept on failure. - Header
#steering-toggle"Tuning" + count badge;#steering-panel(region) above the messages: notes newest-first, per-note delete (labeled); empty text; count updates on add/delete. - a11y: aria-expanded/controls on the toggle, aria-live announcements for save/delete, focus-visible, Phase-08 tokens (all ≥4.5:1).
- "Tune" button (
- Docs: README "Tuning your answers" section; PLAN §4/§5/§6/§9/§7.5
- roadmap rows 11–15.
Locked decisions
None broken: A10 (stateless endpoints), A13 (alembic), A16 (one E2E suite), A11 (no library). New table + new setting only.
Testing & Quality
- Unit: steering section (empty/one/many/budget truncation), prompt placement in both modes, persona-verbatim check aligned to the owner's current persona.
- Integration: steering CRUD (201/200/204/404/422, ordering,
validation); chat turn with a stored note → fake LLM's captured system
prompt contains the note (HIGH and LOW); log line
tuning=N. - Coverage:
uv run pytest --cov=app --cov-report=term-missing>90%. - E2E:
tests/e2e/test_steering.pyper the story mapping. - Regression:
test_chat_rag.py,test_honest_deflection.py,test_retrieval_quality.pygreen in isolation (zero-note prompt is byte-identical, so behavior is unchanged until a note exists).
Commit
git add -A .agents/ alembic/ app/ frontend/ tests/ README.md && git commit --no-gpg-sign -m "feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt"