brain-of-reese/.agents/phases/complete/68_search_tool/00_phase.md

Phase 68 — Agent Search Tool: grep the Indexed Documents

Source: TODO.md L4 — "Add a search tool that allows the LLM to grep through the uploaded documents for a given string" Story: n/a (TODO-derived — owner roadmap confirmation 2026-09-01) Context:

• app/rag/agent.py — AGENT_TOOLS currently defines two OpenAI functions (list_documents, read_document); _execute_tool runs them server-side against the DB with fixed refusal strings (ALREADY_IN_CONTEXT, UNKNOWN_TOOL, MISSING_READ_ARGS); AgentHolder counts executed calls (tool_calls) and context additions (read_docs).
• documents table (app/models.py) — content is the FULL document text (Text column), so a grep is a plain in-process scan: no new index, no migration.
• app/schemas.py — ChatToolEvent ({type: "tool", name, argument}): argument is "source/path" for read_document, null otherwise.
• frontend/assets/app.js — runTurn's tool branch builds the status label (${brand()} is reading … / … is listing documents) and appendToolLine (L796) renders the per-call line (📄 Reading <code>path</code> / 🔎 Listing documents); both special-case by tool name.
• tests/e2e/mock_llm.py — the marker-driven deterministic tool flow (use your tools → list → read → answer) is the template for a search flow; tests/e2e/test_agent_document_tools.py is the pattern for the E2E assertions (.tool-call lines, #send-status recording).
• Phases 37/45/63 (complete) established the tool infrastructure: native tool-calling, unlimited calls bounded by the round cap, labeled source:/path: catalog lines.

Objective

A third agent tool, search_documents, lets the model grep every indexed document (or one named document) for an exact string and get back path:line: text matches — so it can LOCATE content cheaply and then read_document the winner, instead of reading whole documents hoping the string is in them.

Dependencies

• 67_llm_retry (todo, preceding — no functional dependency; ordering by number)

Tasks

1. 01_search_tool_backend.md — the tool definition, the grep_document helper, and the _execute_tool branch.
2. 02_search_tool_api_ui.md — the SSE tool argument mapping and the frontend status/tool-line for the search.
3. 03_e2e_and_commit.md — the mock search flow, the dedicated E2E suite, regressions, commit.

Testing & Quality

• Unit: tests/unit/test_agent.py — match semantics (case-insensitivity, 1-based line numbers, the 20-match cap, 200-char line truncation, scoped single-doc search, no-match/missing-arg/unknown-doc refusals, tool_calls counting, read_docs untouched).
• Integration: tests/integration/test_agent_tools.py — the tool appears in AGENT_TOOLS with the locked parameter shape; tests/integration/test_chat_api.py (or the SSE pin file) — a search_documents call streams argument = pattern.
• E2E (mandatory, house rule): tests/e2e/test_search_tool.py, run in isolation (deterministic mock flow: the model searches, sees the match line, answers from it).
• Coverage: >90% on app/ (validate.sh gate).

Completion Criteria

• search_documents is the third entry in AGENT_TOOLS; a model call with pattern (optionally source+path) returns grep-style matches or a no-match line.
• The UI shows Brain is searching for '…' in the status line and a 🔎 Searching for '<pattern>' tool line, persisted/restored like the other tool lines.
• uv run pytest green; coverage TOTAL >90%; uv run ruff check . && uv run pyright clean.
• uv run pytest tests/e2e/test_search_tool.py -v --no-cov green in isolation (DB up).
• Regression E2E suites green in isolation: test_agent_document_tools.py, test_agent_unlimited_tools.py.
• One --no-gpg-sign commit; phase dir moved to .agents/phases/complete/.

Locked decisions

• Owner-locked (2026-09-01, roadmap confirmation, A5): the match is a case-insensitive fixed substring (no regex — no ReDoS surface, a simple contract for the model); output is grep-style source/path:LINE: text lines; 20 matches per call maximum (global cap across documents, in catalog order), each line truncated to 200 chars; a search does not add the document to the answer context (read_document remains the only context-adder — holder.read_docs is untouched by a search).
• Owner-locked (2026-09-01, roadmap confirmation, A6): the tool name is search_documents (alongside list_documents / read_document).
• Scope: search is offered on grounded (HIGH) turns only, exactly like the existing tools — deflected turns keep tools=None (A8 byte-identical deflection path), and BOR_AGENT_MAX_ROUNDS=0 stays the no-tools kill switch (phase 45).

Commit

git add -A .agents/ app/ tests/ frontend/ && git commit --no-gpg-sign -m "feat(agent): search_documents tool — the model can grep the indexed documents for an exact string"