chore(agent): track .agent/ planning tree in git
Remove the blanket .agent/ gitignore so the phase roadmap, user stories, reports, and PLAN.md are versioned with the code. Only runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log) remain ignored. Update AGENTS.md git protocol rule to match.
This commit is contained in:
@@ -0,0 +1,49 @@
|
||||
# Phase 37 — Agent document tools (list + read)
|
||||
|
||||
**Source:** `TODO.md` L3–L9 — "Still having trouble referencing specific documents. The agent should be able to list the available sources as a tool and the read the ones it thinks are relevant… it will need a basic agent loop. I'm thinking it gets one opportunity to list documents and then one opportunity to add exactly one extra document to its context before being required to answer. These values should be configured by environment variables." (L3; L5–L9 is the failure example: `aws-route53.md` references `example-record-file.json` whose contents are not in context, so the model refuses to guess)
|
||||
**Story:** `.agent/user_stories/agent-document-tools.md`
|
||||
**Context:** PLAN §3/§4 (chat flow + SSE contract), §6 (locked persona prompt), §9 (per-turn log line); `app/api/chat.py` (`plan_turn` + stream), `app/rag/llm.py` (`StreamPiece`, `chat_stream`), `app/rag/prompts.py`, `app/schemas.py`, `frontend/assets/app.js` (chat feedback state machine + thinking block), `tests/e2e/mock_llm.py`.
|
||||
|
||||
## Objective
|
||||
Give the chat model two server-side tools on **grounded** turns — `list_documents` (list the indexed sources) and `read_document(source, path)` (add exactly one more indexed document, full text) — with the opportunity counts tunable by env vars (`BOR_AGENT_LIST_CALLS`, `BOR_AGENT_READ_CALLS`, default 1 each); once both budgets are spent the tools are dropped and the model must answer. The UI shows a "calling tool" state in addition to "thinking".
|
||||
|
||||
## Dependencies
|
||||
- `03_story_chat_rag` (complete) — the A7/A8/A15 pipeline this phase extends.
|
||||
- `14_chat_persistence` (complete) — the saved chat record shape this phase extends with `tools`.
|
||||
- `17_thinking_display` (complete) — the SSE `thinking` event + `StreamPiece` kinds + the UI scratchpad the `tool` event sits beside; the `BOR_STREAM_THINKING` kill-switch pattern.
|
||||
- `24_whole_document_context` (complete) — the whole-document context contract (`read_document` never truncates).
|
||||
- `31_kb_overview_prompt` (complete) — HIGH-prompt section order the `<tools>` instructions join.
|
||||
- No dependency on 34/35/36 (frontend touch surface is `app.js`/`styles.css` only).
|
||||
|
||||
## Tasks
|
||||
1. `01_probe_tool_calling.md` — extend `scripts/llm_probe.py` with a live `--tools` probe against `turbo`; record the verdict (tool-calling vs documented prompt-based fallback).
|
||||
2. `02_llm_client_tools.md` — `app/rag/llm.py`: `chat_stream(messages, tools=None)` accumulates `tool_calls` deltas into `ToolCallPiece`; `tools=None` stays byte-identical.
|
||||
3. `03_agent_loop.md` — `app/rag/agent.py`: `run_agent` loop with env-tuned budgets, DB accessors, the `<tools>` prompt section, `app/config.py` settings.
|
||||
4. `04_api_sse_tool_event.md` — `app/api/chat.py` + `app/schemas.py`: SSE `tool` events, agent on grounded turns, `done.sources`/`query_log` include the read doc, `tool_calls=N` log field, PLAN §4/§9 revision notes.
|
||||
5. `05_frontend_tool_states.md` — `app.js`/`styles.css`: "calling tool" state + `.tool-call` lines + persistence; UI Structure Check.
|
||||
6. `06_e2e_docs_commit.md` — mock-LLM tool behavior, the story E2E, README + `.env.example`, commit, move the phase dir.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `app/rag/agent.py` loop mechanics (budgets, forced answer, dedupe, unknown tool, missing doc, round cap) with a scripted fake LLM + monkeypatched DB accessors; `llm.py` tool-call delta accumulation; config defaults.
|
||||
- Integration: the agent's DB accessors against real Postgres; the `/api/chat` SSE contract with `tool` events (mock LLM); no schema change in this phase.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_agent_document_tools.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run python -m scripts.llm_probe --tools` runs against the live endpoint; the verdict is recorded in the `app/rag/agent.py` docstring + the commit message.
|
||||
- [ ] `POST /api/chat` streams `{"type":"tool","name":"list_documents"}` and `{"type":"tool","name":"read_document","argument":"<source/path>"}` frames for a tool-using model; the non-tool path (deflected, or no tool call) is byte-identical to today's SSE.
|
||||
- [ ] `BOR_AGENT_LIST_CALLS=0 BOR_AGENT_READ_CALLS=0` reproduces pre-phase behavior (no `tools` in the LLM request, no `tool` events).
|
||||
- [ ] `done.sources` + `query_log.sources` include the read document (deduped); the per-turn log line carries `tool_calls=N`.
|
||||
- [ ] The UI shows the "calling tool" label + tool lines while tools run; tool lines re-render after a reload; WCAG basics hold.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_agent_document_tools.py -v --no-cov` green in isolation; existing chat E2E suites green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5) + no CDN (rule 6).
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agent/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **A7/A8 honoured** — the retrieval gate is untouched; `read_document` appends the **full** document text (no truncation, A7-revised contract); the deflection path is byte-identical.
|
||||
- **A15 extended (revision note, owner permission 2026-08-26)** — the SSE contract gains `{"type":"tool","name":…,"argument":…}`; `delta`/`done` shapes are unchanged.
|
||||
- **A5 honoured** — same aipi endpoint/models; tools are plain OpenAI `tools`/`tool_calls`. If the task-01 probe shows `turbo` lacks tool-calling, the phase falls back to the documented prompt-based structured call (task 03) — recorded in the `agent.py` docstring, never silent.
|
||||
- **No schema change** — the tools query existing tables; no migration in this phase.
|
||||
- **Budgets-as-kill-switch** — both budgets at 0 disables the tools entirely (request byte-identical to pre-phase); no separate kill-switch env var.
|
||||
- **A16/A17 honoured** — one new story E2E suite + one atomic `--no-gpg-sign` commit.
|
||||
Reference in New Issue
Block a user