chore(agent): phase roadmap from TODO.md — 8 phases (40–47), 24 tasks
Converts the 9 TODO items into an executable phase roadmap (Protocol B, appended after phase 39): - 40 tuning toggle anonymous flash (TODO L3) - 41 sync fail-fast + modal when a model is down (TODO L4) - 42 no reply autoscroll (TODO L5) - 43 thinking scroll back — user scroll + gated autoscroll (TODO L7) - 44 markdown tables (TODO L6) - 45 agent unlimited tool calls behind BOR_AGENT_MAX_ROUNDS (TODO L8) - 46 mobile hamburger nav (TODO L9) - 47 quadlet + jinja import formats, A9 revision (TODO L10–L11) Each phase carries a user story, a dedicated Playwright E2E suite plan, and owner-locked decisions (R1 A9 format extension, R2 phase-37 budget revision, A1–A5 scope decisions) confirmed 2026-08-27. Also records the completed phases 30–39 todo/ -> complete/ moves that were pending in the working tree. TODO.md is cleared (items now live in .agent/phases/todo/).
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Phase 31 — KB Overview in the System Prompt (lite-generated knowledge-base outline)
|
||||
|
||||
**Source:** `TODO.md L4 — "The system prompt should inject basic categories of everything that's been read so the agent knows roughly what its knowledge base contains before the rag retrieval returns documents. This part of the system prompt should be generated by the lite model and should be stored somewhere so it can be updated whenever we import new documents."`
|
||||
**Story:** `.agent/user_stories/kb-overview-prompt.md`
|
||||
**Context:** Phase 15 steering notes (the `<tuning>` prompt section, its char budget, and the **byte-identical-when-absent** convention — `app/rag/prompts.py::build_steering_section`), phase 30 (the `lite` client method `LLMClient.chat`, and per-document summaries that make a much better overview input than raw titles), `scripts/import_docs.py` (the place "whenever we import new documents" happens), and the E2E mock's answer-echo convention (the `(tuning: …)` suffix — `tests/e2e/mock_llm.py`).
|
||||
|
||||
## Objective
|
||||
Store a lite-generated, plain-text outline of the knowledge base's basic **categories** in a single-row `kb_overview` table, inject it into **both** chat prompts (HIGH and LOW) as a `<knowledge_base>` section so the agent knows roughly what the KB contains before retrieval, and regenerate it automatically whenever an import changes the KB.
|
||||
|
||||
## Dependencies
|
||||
- `30_document_summaries` (todo) — `LLMClient.chat` + `BOR_LLM_SUMMARY_MODEL` (task 01) and the stored per-document summaries (task 04) that feed the overview input.
|
||||
- `15_steering_notes` (complete) — the prompt-section pattern this phase mirrors (budget, marker, byte-identical-when-absent, per-turn load in `app/api/chat.py`).
|
||||
- `11_long_answers` / README import workflow (complete) — `scripts/import_docs.py`'s `main()` structure, which this phase extends with the post-import regeneration.
|
||||
|
||||
## Tasks
|
||||
1. `01_migration_kb_overview.md` — Alembic 0005: single-row `kb_overview` table + `KbOverview` model.
|
||||
2. `02_overview_generator.md` — `app/rag/overview.py`: `KB_OVERVIEW_MODE` prompt builder, `load_kb_overview`, `regenerate_overview` (best-effort upsert).
|
||||
3. `03_prompt_injection.md` — `<knowledge_base>` section in HIGH + LOW prompts (budgeted, byte-identical when absent); `plan_turn`/chat wire it in; `kb_chars` in the per-turn log.
|
||||
4. `04_import_trigger.md` — `import_docs` regenerates the overview after a KB-changing import (shared with phase 32's sync).
|
||||
5. `05_mock_and_e2e.md` — deterministic `KB_OVERVIEW_MODE` mock + `(kb: …)` echo, `tests/e2e/test_kb_overview.py`, story file, commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: overview generator (prompt build/cap, load, regenerate upsert/fail-soft/zero-docs), prompts (section present/budgeted/absent → byte-identical, ordering vs `<tuning>`), chat gate (`kb_chars`, prompt carries the section).
|
||||
- Integration: migration 0005 up/down; `import_docs` regeneration trigger (changed vs unchanged imports, failure isolation).
|
||||
- Coverage: **>90%** on `app/` (`app/` TOTAL ≥ pre-change).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_kb_overview.py` — one story, run **in isolation**; the injected section is observable in the mock answer via the `(kb: …)` echo (steering precedent).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] After a KB-changing import, `kb_overview` holds a fresh outline (log line `overview: regenerated docs=… chars=…`); an unchanged re-import does **not** call the lite model.
|
||||
- [ ] Every chat turn's system prompt (HIGH and LOW) contains the `<knowledge_base>` section when a row exists; with no row, both prompts are **byte-identical** to the pre-phase text (unit-asserted).
|
||||
- [ ] The per-turn log line records `kb_chars=<n>`; section overflow beyond `BOR_KB_OVERVIEW_MAX_CHARS` is capped with the shared `[…truncated…]` marker.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL ≥ pre-change number (app/ >90%).
|
||||
- [ ] `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` green in isolation; existing prompt/steering/chat suites stay green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `.agent/user_stories/kb-overview-prompt.md` exists; `.env.example` + README document `BOR_KB_OVERVIEW_MAX_CHARS` / `BOR_OVERVIEW_INPUT_MAX_CHARS` and the regeneration behavior.
|
||||
- [ ] One `--no-gpg-sign` commit staging only this phase's files (e.g. `feat(rag): lite-generated KB overview in the system prompt — stored single row, regenerated on import, <knowledge_base> section in HIGH+LOW prompts`); `.agent/phases/todo/31_kb_overview_prompt/` moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A13** — migration 0005 adds one single-row table `kb_overview` (`id INTEGER PK DEFAULT 1`, `content TEXT NOT NULL DEFAULT ''`, `updated_at TIMESTAMPTZ`); no other schema change.
|
||||
- **A5 extended** — the overview is generated by the same `lite` model via the same `BOR_LLM_SUMMARY_MODEL` setting and `LLMClient.chat` (phase 30); no new model or package.
|
||||
- **Prompt-section convention (phase 15 precedent)** — the section is budgeted by `BOR_KB_OVERVIEW_MAX_CHARS` (default **4000**) with the shared `TRUNCATION_MARKER` overflow; **zero/empty row → prompts byte-identical** to pre-phase text. Section order: `<relevance>` → `<knowledge_base>` → `<tuning>` → mode body.
|
||||
- **Regeneration is best-effort and change-gated** — runs only when an import added/updated at least one document (or no row exists yet); a lite failure logs and leaves the previous overview intact (an old outline is better than none).
|
||||
- **Overview input is capped** — `BOR_OVERVIEW_INPUT_MAX_CHARS` (default **40 000**) on the document list (source/path/title/first summary line) sent to the model.
|
||||
- **No per-turn LLM call** — chat turns only *read* the stored row (one indexed PK lookup); generation happens at import/sync time (phase 32's button triggers the same `regenerate_overview`).
|
||||
- **A16 / A17 honoured** — one dedicated story E2E suite; one atomic `--no-gpg-sign` commit.
|
||||
Reference in New Issue
Block a user