feat(agent): align the document tools with the harness-trained shape — ls, read(path), grep(pattern, path?)

This commit is contained in:
2026-09-03 11:17:47 -04:00
parent 16f1cfbcaf
commit 801639efcc
55 changed files with 4031 additions and 1466 deletions
+27 -15
View File
@@ -163,28 +163,40 @@ exactly as before.
To hide it, set `BOR_STREAM_THINKING=0` — the `thinking` events stop
(the per-turn log line still counts `thinking_chars`).
## Agent document tools (list + read)
## Agent document tools (ls + read + grep)
Retrieval only puts the top documents in context. When an answer depends
on a file a note *references* ("the exact JSON shape is in
example-record-file.json"), the model can extend its own context with two
server-side tools — on **grounded** (high-relevance) turns only:
example-record-file.json"), the model can extend its own context with
three server-side tools — on **grounded** (high-relevance) turns only.
The surface mirrors the shape the chat model was trained on (the pi.dev
harness tools, owner decision 2026-09-03), and the canonical document
identity everywhere is the combined `source/path` string:
* **`list_documents`** — lists every indexed document, one
`source: X | path: Y | title: Z` line each (the same order as the
Sources page);
* **`read_document(source, path)`** — appends the **full** text of
one more indexed document to the context (never truncated).
* **`ls`** — lists every indexed document, one `source: X | path: Y |
title: Z` line each (the same order as the Sources page); pass a
source name as `path` to list one source's documents;
* **`read(path)`** — appends the **full** text of one more indexed
document to the context (never truncated); `path` is the combined
`source/path` string exactly as shown in the `ls` output;
* **`grep(pattern, path?)`** — searches the indexed documents for an
exact string (case-insensitive fixed substring) and returns up to 20
matching `source/path:line: text` lines; an optional `path` limits the
search to one document. A locator, not a context-adder: it never adds
to the answer context — the model `read`s the winner.
Each call the model requests is executed against Postgres only (no extra
LLM round trip) and streamed as an SSE `tool` frame ahead of the answer —
`{"type": "tool", "name": …, "argument": "source/path" | null}`. In the
chat, each call shows a **"calling tool" state** in addition to
"thinking": the send button keeps its busy state ("Calling tool…") and a
visible tool line (`🔎 Listing documents` / `📄 Reading source/path`) lands
above the answer, one per call, in order. The tool lines persist with the
message, so a reloaded conversation re-renders them. The read document is
reflected in the answer's **source chips** and in the `query_log` row.
`{"type": "tool", "name": "ls" | "read" | "grep", "argument": … | null}`
(`argument` is the single string the model passed — `read`'s `path`,
`grep`'s `pattern`, `ls`'s scope — or null). In the chat, each call shows
a transient **calling-tool status** in addition to "thinking" (the send
button keeps its busy state — "Stop" — for the whole turn) and a visible
tool line (`🔎 Listing documents` / `📄 Reading source/path` /
`🔎 Searching for pattern`) lands above the answer, one per call, in
order. The tool lines persist with the message, so a reloaded
conversation re-renders them. The read document is reflected in the
answer's **source chips** and in the `query_log` row.
The tools stay offered for the whole turn — the model may call them as
many times as it needs (re-lists included), bounded only by a round cap