brain-of-reese/.agent/phases/complete/37_agent_document_tools/00_phase.md

Phase 37 — Agent document tools (list + read)

Source: TODO.md L3–L9 — "Still having trouble referencing specific documents. The agent should be able to list the available sources as a tool and the read the ones it thinks are relevant… it will need a basic agent loop. I'm thinking it gets one opportunity to list documents and then one opportunity to add exactly one extra document to its context before being required to answer. These values should be configured by environment variables." (L3; L5–L9 is the failure example: aws-route53.md references example-record-file.json whose contents are not in context, so the model refuses to guess) Story: .agent/user_stories/agent-document-tools.md Context: PLAN §3/§4 (chat flow + SSE contract), §6 (locked persona prompt), §9 (per-turn log line); app/api/chat.py (plan_turn + stream), app/rag/llm.py (StreamPiece, chat_stream), app/rag/prompts.py, app/schemas.py, frontend/assets/app.js (chat feedback state machine + thinking block), tests/e2e/mock_llm.py.

Objective

Give the chat model two server-side tools on grounded turns — list_documents (list the indexed sources) and read_document(source, path) (add exactly one more indexed document, full text) — with the opportunity counts tunable by env vars (BOR_AGENT_LIST_CALLS, BOR_AGENT_READ_CALLS, default 1 each); once both budgets are spent the tools are dropped and the model must answer. The UI shows a "calling tool" state in addition to "thinking".

Dependencies

• 03_story_chat_rag (complete) — the A7/A8/A15 pipeline this phase extends.
• 14_chat_persistence (complete) — the saved chat record shape this phase extends with tools.
• 17_thinking_display (complete) — the SSE thinking event + StreamPiece kinds + the UI scratchpad the tool event sits beside; the BOR_STREAM_THINKING kill-switch pattern.
• 24_whole_document_context (complete) — the whole-document context contract (read_document never truncates).
• 31_kb_overview_prompt (complete) — HIGH-prompt section order the <tools> instructions join.
• No dependency on 34/35/36 (frontend touch surface is app.js/styles.css only).

Tasks

1. 01_probe_tool_calling.md — extend scripts/llm_probe.py with a live --tools probe against turbo; record the verdict (tool-calling vs documented prompt-based fallback).
2. 02_llm_client_tools.md — app/rag/llm.py: chat_stream(messages, tools=None) accumulates tool_calls deltas into ToolCallPiece; tools=None stays byte-identical.
3. 03_agent_loop.md — app/rag/agent.py: run_agent loop with env-tuned budgets, DB accessors, the <tools> prompt section, app/config.py settings.
4. 04_api_sse_tool_event.md — app/api/chat.py + app/schemas.py: SSE tool events, agent on grounded turns, done.sources/query_log include the read doc, tool_calls=N log field, PLAN §4/§9 revision notes.
5. 05_frontend_tool_states.md — app.js/styles.css: "calling tool" state + .tool-call lines + persistence; UI Structure Check.
6. 06_e2e_docs_commit.md — mock-LLM tool behavior, the story E2E, README + .env.example, commit, move the phase dir.

Testing & Quality

• Unit: app/rag/agent.py loop mechanics (budgets, forced answer, dedupe, unknown tool, missing doc, round cap) with a scripted fake LLM + monkeypatched DB accessors; llm.py tool-call delta accumulation; config defaults.
• Integration: the agent's DB accessors against real Postgres; the /api/chat SSE contract with tool events (mock LLM); no schema change in this phase.
• Coverage: >90% on app/.
• E2E (mandatory, A16): tests/e2e/test_agent_document_tools.py, run in isolation.

Completion Criteria

• uv run python -m scripts.llm_probe --tools runs against the live endpoint; the verdict is recorded in the app/rag/agent.py docstring + the commit message.
• POST /api/chat streams {"type":"tool","name":"list_documents"} and {"type":"tool","name":"read_document","argument":"<source/path>"} frames for a tool-using model; the non-tool path (deflected, or no tool call) is byte-identical to today's SSE.
• BOR_AGENT_LIST_CALLS=0 BOR_AGENT_READ_CALLS=0 reproduces pre-phase behavior (no tools in the LLM request, no tool events).
• done.sources + query_log.sources include the read document (deduped); the per-turn log line carries tool_calls=N.
• The UI shows the "calling tool" label + tool lines while tools run; tool lines re-render after a reload; WCAG basics hold.
• uv run pytest green; uv run pytest --cov=app --cov-report=term-missing >90%; uv run pytest tests/e2e/test_agent_document_tools.py -v --no-cov green in isolation; existing chat E2E suites green.
• uv run ruff check . && uv run pyright clean.
• UI Structure Check (AGENTS.md rule 5) + no CDN (rule 6).
• One --no-gpg-sign commit; phase directory moved to .agent/phases/complete/.

Locked decisions

• A7/A8 honoured — the retrieval gate is untouched; read_document appends the full document text (no truncation, A7-revised contract); the deflection path is byte-identical.
• A15 extended (revision note, owner permission 2026-08-26) — the SSE contract gains {"type":"tool","name":…,"argument":…}; delta/done shapes are unchanged.
• A5 honoured — same aipi endpoint/models; tools are plain OpenAI tools/tool_calls. If the task-01 probe shows turbo lacks tool-calling, the phase falls back to the documented prompt-based structured call (task 03) — recorded in the agent.py docstring, never silent.
• No schema change — the tools query existing tables; no migration in this phase.
• Budgets-as-kill-switch — both budgets at 0 disables the tools entirely (request byte-identical to pre-phase); no separate kill-switch env var.
• A16/A17 honoured — one new story E2E suite + one atomic --no-gpg-sign commit.