Files
brain-of-reese/.agent/phases/complete/15_steering_notes/00_phase.md
T
ducoterra 4971e2859d
Build and Push Containers / build-and-push-app (push) Successful in 12s
Build and Push Containers / build-and-push-db (push) Successful in 10s
chore(agent): track .agent/ planning tree in git
Remove the blanket .agent/ gitignore so the phase roadmap, user
stories, reports, and PLAN.md are versioned with the code. Only
runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log)
remain ignored. Update AGENTS.md git protocol rule to match.
2026-09-01 10:18:22 -04:00

4.2 KiB
Raw Blame History

Phase 15 — Tune How Brain Answers (Steering Notes)

Story: .agent/user_stories/steering-notes.md Context: owner report 2026-08-22 — "the answers are off. Add a feature that lets me 'tune' the output if I think an answer isn't quite right. This tuning should be added to the database and read in with the system prompt to steer the replies."

Goal

A first-class steering loop: under any answer, "Tune" → short instruction → stored in Postgres (steering_notes) → injected into the system prompt of every subsequent turn → observable in the reply. Notes are listed and deletable in a "Tuning" panel.

Implementation steps

  1. Schema — alembic/versions/0003_steering_notes.py: steering_notes(id UUID PK, note TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now()); downgrade drops the table. Model SteeringNote in app/models.py.
  2. API — app/api/steering.py (stateless, A10):
    • GET /api/steering → {notes: [{id, note, created_at}]} newest first.
    • POST /api/steering {note} (trimmed, 1–2000 chars; 422 on empty/over-long) → 201 with the created note.
    • DELETE /api/steering/{note_id} → 204; 404 unknown id. Mount in app/main.py; schemas in app/schemas.py.
  3. Prompt (app/rag/prompts.py):
    • build_steering_section(notes: Sequence[str]) -> str — "" when empty; else <tuning>…</tuning> with numbered notes, capped at steering_max_chars (new setting, default 8000; […truncated…] marker). Both HIGH and DEFLECT prompts carry it (after <relevance>…</relevance>, before <documents>/DEFLECT_MODE body). With zero notes the prompt is byte-identical to today.
    • Preserve the owner's working-tree persona edits (no "you've got this" / no mandated deflection opening — align tests/unit/test_prompts.py verbatim check with the current text; record as a PLAN §6 revision).
  4. Chat turn (app/api/chat.py): load notes (created_at asc), pass into plan_turn → prompts; per-turn log line gains tuning=N (PLAN §9).
  5. E2E mock (tests/e2e/mock_llm.py): when the system prompt contains <tuning>, the composed answer ends with (tuning: <first note line>) — makes prompt injection observable in the UI deterministically.
  6. UI (frontend/index.html, app.js, styles.css):
    • "Tune" button (.tune-btn, ghost, ≥44px) in the meta row of every completed brain bubble (deflected included).
    • Inline .tune-form: labeled textarea (maxlength 2000) + Save / Cancel → POST /api/steering → success .tune-saved (role=status: "Saved — future answers will follow this.") or inline error (role=alert), form kept on failure.
    • Header #steering-toggle "Tuning" + count badge; #steering-panel (region) above the messages: notes newest-first, per-note delete (labeled); empty text; count updates on add/delete.
    • a11y: aria-expanded/controls on the toggle, aria-live announcements for save/delete, focus-visible, Phase-08 tokens (all ≥4.5:1).
  7. Docs: README "Tuning your answers" section; PLAN §4/§5/§6/§9/§7.5
    • roadmap rows 11–15.

Locked decisions

None broken: A10 (stateless endpoints), A13 (alembic), A16 (one E2E suite), A11 (no library). New table + new setting only.

Testing & Quality

  • Unit: steering section (empty/one/many/budget truncation), prompt placement in both modes, persona-verbatim check aligned to the owner's current persona.
  • Integration: steering CRUD (201/200/204/404/422, ordering, validation); chat turn with a stored note → fake LLM's captured system prompt contains the note (HIGH and LOW); log line tuning=N.
  • Coverage: uv run pytest --cov=app --cov-report=term-missing >90%.
  • E2E: tests/e2e/test_steering.py per the story mapping.
  • Regression: test_chat_rag.py, test_honest_deflection.py, test_retrieval_quality.py green in isolation (zero-note prompt is byte-identical, so behavior is unchanged until a note exists).

Commit

git add -A .agent/ alembic/ app/ frontend/ tests/ README.md && git commit --no-gpg-sign -m "feat(rag): steering notes — tune how Brain answers, stored in Postgres and injected into every system prompt"