feat(rag): unbounded agent tool calls behind a round cap (owner revision)
Phase 45 (owner permission 2026-08-27, TODO.md L8: "allow the LLM
to make as many tool calls as it wants"): the phase-37 per-turn tool
budgets (BOR_AGENT_LIST_CALLS / BOR_AGENT_READ_CALLS, default 1 each)
and their exhaustion refusals are removed — a grounded turn now offers
list_documents / read_document for the whole turn (re-lists included),
bounded only by the round cap:
- app/config.py: agent_max_rounds (BOR_AGENT_MAX_ROUNDS, default 10,
negative rejected) replaces agent_list_calls / agent_read_calls;
.env.example + README document the single knob; app/rag/prompts.py
docstrings follow.
- app/rag/agent.py: the loop runs tools until the model answers or
rounds >= max_rounds, at which point it forces one final no-tools
answer (the cap is the only forced exit); 0 = no tools — exactly one
tools=None request, byte-identical to the pre-phase-37 path (the
kill switch). Rejected calls (unknown tool / missing args /
already-in-context / unknown path) still consume a round, so
pathological rejected-call streams are bounded by the cap. The
per-call log line is now tool/args/round=N/M; the per-turn
tool_calls=N field and the tool SSE event are unchanged.
- tests/e2e/mock_llm.py: MULTI_READ_TRIGGER ("read two documents") —
the deterministic list -> read #1 -> read #2 -> forced-answer flow
(byte-stable "I read <sp1> and <sp2>." line), classified by the
count of tool-role read results; the phase-37 single-read flow stays
byte-identical (unit-pinned in tests/unit/test_mock_tool_flow.py).
- tests/e2e/test_agent_unlimited_tools.py (new, story suite,
mock-only): three tool frames/lines in order (one list, two reads —
the second read is what the old read budget refused) + the
both-named non-deflected answer; done.sources + chips = retrieval
doc + both reads, deduped; no budget refusal rendered; the
single-read marker flow regression (exactly one read, single tool
pair).
- .agent/PLAN.md: the phase-45 SSE revision note (owner-locked, R2) —
the only PLAN edit this phase; the phase-37 note's budget clause is
marked removed.
Unit/integration rewrites (test_agent.py round-cap matrix incl. the
kill switch and rejected-call spam, test_config.py, test_chat_api.py
agent_max_rounds=0 fixtures) landed with the server core so every gate
stays green.
uv run pytest: 756 passed, app/ coverage 99%; ruff + pyright clean;
story E2E 4/4 in isolation (ran twice); regression E2E suites
(agent_document_tools unmodified, chat_rag, smoke) green in isolation.
Also records the 45_agent_unlimited_tools todo/ -> complete/ task-file
moves (00/01/02 pending in the working tree, task 03 moves on success).
This commit is contained in:
+61
-67
@@ -15,38 +15,42 @@ probe came back "supported".
|
||||
Loop contract (one grounded chat turn; the API layer wires this in,
|
||||
task 04):
|
||||
|
||||
1. While budget remains the model is offered the two OpenAI functions in
|
||||
:data:`AGENT_TOOLS`: up to ``settings.agent_list_calls``
|
||||
(``BOR_AGENT_LIST_CALLS``, default 1) ``list_documents`` calls and up
|
||||
to ``settings.agent_read_calls`` (``BOR_AGENT_READ_CALLS``, default 1)
|
||||
``read_document`` calls. With both budgets at 0 the loop makes exactly
|
||||
one request with ``tools=None`` — byte-identical to the pre-phase chat
|
||||
path (budgets-as-kill-switch, phase 37 locked decision).
|
||||
1. The model is offered the two OpenAI functions in :data:`AGENT_TOOLS`
|
||||
for the whole turn — phase 45 removed the phase-37 per-tool budgets
|
||||
(owner permission 2026-08-27, ``TODO.md`` L8: "allow the LLM to make
|
||||
as many tool calls as it wants"): ``list_documents`` and
|
||||
``read_document`` can each be called as many times as the model needs,
|
||||
re-lists included. With ``settings.agent_max_rounds``
|
||||
(``BOR_AGENT_MAX_ROUNDS``, default 10) at 0 the loop makes exactly one
|
||||
request with ``tools=None`` — byte-identical to the pre-phase-37 chat
|
||||
path (the kill switch).
|
||||
2. Each tool call the model emits is executed server-side against
|
||||
Postgres only (no LLM, no network): ``list_documents`` returns the
|
||||
indexed catalog — one ``source/path — title`` line per document,
|
||||
``GET /api/docs`` order (uncapped in v1; the UI never shows it, only
|
||||
the model does) — and ``read_document`` returns the document's **full**
|
||||
content (A7-revised contract: never truncated).
|
||||
3. Rejected calls consume **no** budget and get a one-line refusal:
|
||||
unknown tool name → ``"Unknown tool."``; missing ``source``/``path``
|
||||
arguments; a document already in context (seed or previously read) →
|
||||
``"Already in your context."``; an unknown ``source/path`` →
|
||||
``"No document at …"``; an exhausted list/read budget → the matching
|
||||
``"No … budget left"`` refusal.
|
||||
4. Every executed call is appended back to the message history as the
|
||||
assistant tool-call message + the tool result, and the model is called
|
||||
again. Once **both** budgets are spent, ``tools`` is dropped from the
|
||||
request and the model must answer. Belt-and-braces round cap:
|
||||
``max_rounds = 2 + agent_list_calls + agent_read_calls`` (every tool
|
||||
round consumes a budget, so the cap only catches pathological streams
|
||||
that keep calling rejected tools) — at the cap the loop forces one
|
||||
final ``chat_stream(messages, tools=None)`` and returns.
|
||||
3. Rejected calls get a one-line refusal and count in nothing
|
||||
(``holder.tool_calls`` tracks executed calls only): unknown tool name
|
||||
→ ``"Unknown tool."``; missing ``source``/``path`` arguments; a
|
||||
document already in context (seed or previously read) → ``"Already in
|
||||
your context."``; an unknown ``source/path`` → ``"No document at …"``.
|
||||
A rejected call still consumes a *round* in the loop, so a
|
||||
pathological stream that keeps emitting rejected calls is bounded by
|
||||
the cap (point 4).
|
||||
4. Every call the model emits is appended back to the message history as
|
||||
the assistant tool-call message + the tool result (refusals included),
|
||||
consumes one round, and the model is called again. At the round cap —
|
||||
``max_rounds = settings.agent_max_rounds`` (``BOR_AGENT_MAX_ROUNDS``,
|
||||
default 10) — the loop forces one final ``chat_stream(messages,
|
||||
tools=None)`` and returns: the cap is the **only** forced exit
|
||||
(besides "the stream carried no calls"), and it bounds pathological
|
||||
rejected-call streams.
|
||||
5. A rare stream that carries both content and a tool call keeps the
|
||||
content (it was already emitted) **and** still runs the tool.
|
||||
6. *holder* (an :class:`AgentHolder`) records the read documents and the
|
||||
number of budget-consuming tool executions; the API layer (task 04)
|
||||
reads it after the stream to extend ``done.sources`` /
|
||||
number of executed tool calls (re-lists included); the API layer
|
||||
(task 04) reads it after the stream to extend ``done.sources`` /
|
||||
``query_log.sources`` and the per-turn log line (``tool_calls=N``).
|
||||
|
||||
The DB accessors (:func:`list_catalog`, :func:`find_document`) are
|
||||
@@ -71,8 +75,9 @@ from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
|
||||
logger = logging.getLogger("app.agent")
|
||||
|
||||
#: The two agent tools (phase 37): OpenAI function definitions passed as
|
||||
#: ``tools=AGENT_TOOLS`` to ``chat_stream`` while the per-turn budgets
|
||||
#: (``BOR_AGENT_LIST_CALLS`` / ``BOR_AGENT_READ_CALLS``) remain.
|
||||
#: ``tools=AGENT_TOOLS`` to ``chat_stream`` for the whole grounded turn —
|
||||
#: phase 45 removed the per-tool budgets; the round cap
|
||||
#: (``BOR_AGENT_MAX_ROUNDS``) is the only bound.
|
||||
AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
{
|
||||
"type": "function",
|
||||
@@ -90,7 +95,7 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"function": {
|
||||
"name": "read_document",
|
||||
"description": (
|
||||
"Add the full content of exactly one more indexed document "
|
||||
"Add the full content of one more indexed document "
|
||||
"to your context"
|
||||
),
|
||||
"parameters": {
|
||||
@@ -117,9 +122,9 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
},
|
||||
]
|
||||
|
||||
#: Tool refusal texts (phase 37): rejected calls consume no budget.
|
||||
LIST_EXHAUSTED = "No listing budget left — answer with what you have."
|
||||
READ_EXHAUSTED = "No reading budget left — answer with what you have."
|
||||
#: Tool refusal texts (phase 37): rejected calls count in nothing
|
||||
#: (``holder.tool_calls`` tracks executed calls); the round cap bounds
|
||||
#: their pathological repetition (phase 45).
|
||||
ALREADY_IN_CONTEXT = "Already in your context."
|
||||
UNKNOWN_TOOL = "Unknown tool."
|
||||
MISSING_READ_ARGS = "read_document requires string arguments 'source' and 'path'."
|
||||
@@ -155,8 +160,8 @@ class AgentHolder:
|
||||
|
||||
``read_docs``: the documents ``read_document`` added to the context,
|
||||
in read order (deduped — re-reading a document appends nothing).
|
||||
``tool_calls``: how many tool executions consumed budget; rejected
|
||||
calls (unknown tool, unknown/missing document, exhausted budget,
|
||||
``tool_calls``: how many tool calls executed (re-lists included);
|
||||
rejected calls (unknown tool, unknown/missing arguments or document,
|
||||
already-in-context) do not count. Drives the per-turn log line's
|
||||
``tool_calls=N`` field (task 04).
|
||||
"""
|
||||
@@ -170,47 +175,40 @@ def _execute_tool(
|
||||
call: ToolCallPiece,
|
||||
seed_docs: Sequence[Document],
|
||||
holder: AgentHolder,
|
||||
list_left: int,
|
||||
read_left: int,
|
||||
) -> tuple[str, int, int]:
|
||||
) -> str:
|
||||
"""Execute one tool call server-side (DB only).
|
||||
|
||||
Returns ``(result, list_left, read_left)``. Rejected calls consume no
|
||||
budget; a successful read appends the :class:`Document` to
|
||||
``holder.read_docs`` and bumps ``holder.tool_calls``.
|
||||
Returns the tool result text. A successful call bumps
|
||||
``holder.tool_calls`` (a successful read also appends the
|
||||
:class:`Document` to ``holder.read_docs``); rejected calls return
|
||||
their refusal line and count in nothing.
|
||||
"""
|
||||
if call.name == "list_documents":
|
||||
if list_left <= 0:
|
||||
return LIST_EXHAUSTED, list_left, read_left
|
||||
rows = list_catalog(db)
|
||||
listing = f"{len(rows)} documents:\n" + "\n".join(
|
||||
f"{source}/{path} — {title}" for source, path, title in rows
|
||||
)
|
||||
holder.tool_calls += 1
|
||||
return listing, list_left - 1, read_left
|
||||
return listing
|
||||
if call.name == "read_document":
|
||||
raw_source = call.arguments.get("source")
|
||||
raw_path = call.arguments.get("path")
|
||||
source = raw_source.strip() if isinstance(raw_source, str) else ""
|
||||
path = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
if not source or not path:
|
||||
return MISSING_READ_ARGS, list_left, read_left
|
||||
return MISSING_READ_ARGS
|
||||
known = {(doc.source, doc.path) for doc in (*seed_docs, *holder.read_docs)}
|
||||
if (source, path) in known:
|
||||
return ALREADY_IN_CONTEXT, list_left, read_left
|
||||
if read_left <= 0:
|
||||
return READ_EXHAUSTED, list_left, read_left
|
||||
return ALREADY_IN_CONTEXT
|
||||
doc = find_document(db, source, path)
|
||||
if doc is None:
|
||||
return (
|
||||
f"No document at {source}/{path} — check the list_documents output.",
|
||||
list_left,
|
||||
read_left,
|
||||
f"No document at {source}/{path} — check the list_documents output."
|
||||
)
|
||||
holder.read_docs.append(doc)
|
||||
holder.tool_calls += 1
|
||||
return f"Document {source}/{path}:\n{doc.content}", list_left, read_left - 1
|
||||
return UNKNOWN_TOOL, list_left, read_left
|
||||
return f"Document {source}/{path}:\n{doc.content}"
|
||||
return UNKNOWN_TOOL
|
||||
|
||||
|
||||
async def run_agent(
|
||||
@@ -228,24 +226,23 @@ async def run_agent(
|
||||
Every piece (``thinking`` / ``content`` / tool calls) is yielded as it
|
||||
arrives; the API layer (task 04) turns tool-call pieces into SSE
|
||||
``tool`` events. After the loop finishes, *holder* carries the read
|
||||
documents and the budget-consuming tool count.
|
||||
documents and the executed tool-call count (re-lists included).
|
||||
|
||||
``seed_docs`` are the documents the retrieval already put in context
|
||||
(they shape the *system_prompt* the caller built); re-reading one of
|
||||
them is rejected as "Already in your context." without spending budget.
|
||||
them is rejected as "Already in your context." — the rejection counts
|
||||
in nothing, but it still consumes a round.
|
||||
"""
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": user_message},
|
||||
]
|
||||
list_left = settings.agent_list_calls
|
||||
read_left = settings.agent_read_calls
|
||||
tools: list[dict[str, Any]] | None = AGENT_TOOLS if (list_left or read_left) else None
|
||||
# Every tool round consumes a budget, so this cap only catches
|
||||
# pathological streams that keep calling rejected tools (belt and
|
||||
# braces — the budgets already force the answer after
|
||||
# list + read rounds).
|
||||
max_rounds = 2 + settings.agent_list_calls + settings.agent_read_calls
|
||||
# Phase 45: no per-tool budgets — the tools stay offered for the
|
||||
# whole turn, bounded by the round cap. ``0`` is the no-tools kill
|
||||
# switch: exactly one request with ``tools=None`` (the pre-phase-37
|
||||
# path).
|
||||
max_rounds = settings.agent_max_rounds
|
||||
tools: list[dict[str, Any]] | None = AGENT_TOOLS if max_rounds > 0 else None
|
||||
rounds = 0
|
||||
while True:
|
||||
calls: list[ToolCallPiece] = []
|
||||
@@ -258,15 +255,14 @@ async def run_agent(
|
||||
if not calls:
|
||||
return # the answer was streamed
|
||||
call = calls[0] # a stream can carry several calls; run the first
|
||||
result, list_left, read_left = _execute_tool(
|
||||
db, call, seed_docs, holder, list_left, read_left
|
||||
)
|
||||
result = _execute_tool(db, call, seed_docs, holder)
|
||||
rounds += 1 # every call the model emits consumes a round
|
||||
logger.info(
|
||||
"agent tool=%s args=%s budget list_left=%d read_left=%d",
|
||||
"agent tool=%s args=%s round=%d/%d",
|
||||
call.name,
|
||||
json.dumps(call.arguments, ensure_ascii=False)[:200],
|
||||
list_left,
|
||||
read_left,
|
||||
rounds,
|
||||
max_rounds,
|
||||
)
|
||||
messages.append(
|
||||
{
|
||||
@@ -285,8 +281,6 @@ async def run_agent(
|
||||
}
|
||||
)
|
||||
messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
|
||||
tools = None if (list_left == 0 and read_left == 0) else AGENT_TOOLS
|
||||
rounds += 1
|
||||
if rounds >= max_rounds:
|
||||
logger.warning(
|
||||
"agent round cap reached (rounds=%d) — forcing a final "
|
||||
|
||||
+7
-7
@@ -26,9 +26,9 @@ prompt is byte-identical to the pre-phase text.
|
||||
|
||||
Agent tools (phase 37): the **HIGH** prompt only carries a ``<tools>``
|
||||
section after the ``<documents>`` body — the grounded turn may call the
|
||||
server-side ``list_documents`` / ``read_document`` tools (budgeted, see
|
||||
:mod:`app.rag.agent`). The LOW/deflection prompt never carries it and
|
||||
stays byte-identical to the pre-phase text.
|
||||
server-side ``list_documents`` / ``read_document`` tools (round-capped,
|
||||
see :mod:`app.rag.agent`). The LOW/deflection prompt never carries it
|
||||
and stays byte-identical to the pre-phase text.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -74,8 +74,8 @@ _KB_INTRO = (
|
||||
|
||||
#: The ``<tools>`` instructions section — **HIGH prompt only** (phase 37,
|
||||
#: task 03): a grounded turn may extend its context through the two
|
||||
#: server-side tools (budgets: ``BOR_AGENT_LIST_CALLS`` /
|
||||
#: ``BOR_AGENT_READ_CALLS``, see :mod:`app.rag.agent`). Appended after
|
||||
#: server-side tools (round cap: ``BOR_AGENT_MAX_ROUNDS``, see
|
||||
#: :mod:`app.rag.agent`). Appended after
|
||||
#: the mode body (``<documents>``), so the instructions are the last
|
||||
#: thing the model reads. The LOW/deflection prompt never carries it —
|
||||
#: a deflection has no grounded context to extend — and stays
|
||||
@@ -184,8 +184,8 @@ texts of the top documents + the ``<tools>`` instructions (phase 37).
|
||||
|
||||
Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
|
||||
→ ``<documents>`` → ``<tools>``; empty steering/overview omit their
|
||||
section. ``<tools>`` is always present in the HIGH prompt (the
|
||||
budgets — not the prompt — decide whether the tools are actually
|
||||
section. ``<tools>`` is always present in the HIGH prompt (the round
|
||||
cap — not the prompt — decides whether the tools are actually
|
||||
offered to the model, see :mod:`app.rag.agent`).
|
||||
"""
|
||||
blocks = [
|
||||
|
||||
Reference in New Issue
Block a user