phase: 160_tool_budget_telemetry
All green. Final report: **Phase 160 completion pass — verified (final verification; all 5 tasks already in `complete/`)** - **Verified, no code defects found:** budget trailers (`TOOL_BUDGET_REMAINING/LAST/SPENT`), cap-path rejection + `ROUND_CAP_FALLBACK_ANSWER` + `ROUND_CAP_WRAPUP_INSTRUCTION`, `done.budget_exhausted` (ABSENT-when-false), `rounds=/cap_hit=` log fields, `agent_max_rounds=10` default, frontend Continue button + restore + shared-page exclusion, mock markers. - **Tests/lint:** `uv run pytest` 153/153 green (EXIT 0); `uv run pytest --cov=app --cov-report=term-missing` TOTAL **99%** (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors/0 warnings. - **Criteria:** ① decreasing trailer pinned by unit tests (9→…→spent) ✅ ② cap path ends readable, no tool lines after cap (E2E) ✅ ③ `budget_exhausted` true-only-on-cap, absent otherwise (integration) ✅ ④ Continue button → one fresh `continue` turn, fresh budget, record intact (E2E) ✅ ⑤ flag survives refresh; shared chat never renders it (`shared.js` has no `budget_exhausted` path) ✅ ⑥ `max_rounds=0` byte-identical single `tools=None` request; default still 10 ✅ ⑦ full suite/coverage/lint green ✅ ⑧ `test_tool_budget_telemetry.py` **7/7** in isolation; all 6 regression E2E suites green in isolation ✅ ⑨ commit + phase move = harness (I make no git history/index changes). - **Docs:** README.md — fixed: the "Continue researching" button and cap behavior were undocumented (added a Chat-section paragraph + an Agent-tools-section paragraph, incl. the phase-71 recorded-revision note); env-var table, commands, and architecture all verified consistent otherwise. AGENTS.md — verified consistent, no changes needed. - **Decisions:** no code changes required — phase was fully implemented; only README docs added. No live-infrastructure touched. - **Next pending phase:** `161_reasoning_bubble`.
This commit is contained in:
+70
-4
@@ -122,6 +122,29 @@ the LOW prompt never carries tools, and with ``agent_max_rounds`` at
|
||||
**0** ``run_agent`` makes exactly one ``tools=None`` request,
|
||||
reproducing the pre-phase behavior (the kill switch).
|
||||
|
||||
No ``tool`` frame without tools (phase 160, task 02): a request made
|
||||
WITHOUT tools never produces a ``tool`` frame — a structured ``tool_calls``
|
||||
reply to a ``tools=None`` request (the documented ``lite`` behavior, the
|
||||
phase-71 incident class) is rejected at the transport: ``_pump``
|
||||
(``tools_offered=False`` on the deflected turn and its recovery) drops the
|
||||
``ToolCallPiece`` (counted in a local, named by one debug line) instead of
|
||||
rendering it as a tool line, and the grounded cap path's rejected calls are
|
||||
suppressed inside ``run_agent`` (``holder.rejected_calls``). The ``tool``
|
||||
event shape (A15) is unchanged — a frame is simply no longer emitted for a
|
||||
call that cannot run.
|
||||
|
||||
Budget-exhausted report (phase 160, task 03, TODO L4): a turn that
|
||||
hits ``BOR_AGENT_MAX_ROUNDS`` reports itself — the done frame carries
|
||||
one additive key, ``budget_exhausted: true`` (driven by
|
||||
``AgentHolder.cap_reached``; ABSENT — never ``false`` — on every other
|
||||
turn, so non-cap frames stay byte-identical to pre-phase; the UI's
|
||||
"Continue researching" affordance keys on it, task 04). The per-turn
|
||||
log line records ``rounds=N`` and ``cap_hit=0|1`` after
|
||||
``tool_calls=N`` (PLAN §9): ``N`` is the rounds CONSUMED (an executed
|
||||
call or a rejection; a cap-path rejected call is not a round) and
|
||||
``cap_hit=1`` only on a cap-reached turn — ``rounds=0 cap_hit=0`` on
|
||||
deflected turns (the agent never runs) and kill-switch turns.
|
||||
|
||||
LLM retries (phase 67, ``BOR_LLM_RETRIES`` / ``BOR_LLM_RETRY_DELAY``,
|
||||
owner-locked 2026-09-01): when the aipi endpoint dies before a request
|
||||
has streamed its first output frame (locked A2), the request is
|
||||
@@ -890,11 +913,21 @@ async def chat(
|
||||
thinking_chars = 0
|
||||
content_chars = 0 # phase 71: the turn's visible (clean) content
|
||||
scaffold_stripped = 0 # phase 71: sum across the turn's requests
|
||||
# Phase 160 (task 02): the transport half of the cap path — a
|
||||
# request made WITHOUT tools (the deflected turn, A8, and its
|
||||
# recovery) never produces a ``tool`` frame: a structured
|
||||
# ``tool_calls`` reply is rejected at ``_pump`` (dropped,
|
||||
# counted in a local, named by one debug line). The grounded
|
||||
# first pass keeps its current behavior (``run_agent``
|
||||
# suppresses the cap-path calls internally, and the kill
|
||||
# switch is byte-identical — unchanged, phase 45).
|
||||
tools_offered = not plan.deflected
|
||||
|
||||
async def _pump(
|
||||
pieces: AsyncIterator[
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
|
||||
],
|
||||
tools_offered: bool = True,
|
||||
) -> AsyncIterator[str]:
|
||||
"""One request's piece loop (phase 71 extraction): the
|
||||
thinking/tool/retry/tool_result/delta handling shared by
|
||||
@@ -903,10 +936,27 @@ async def chat(
|
||||
(pinned by the existing integration suite). Phase 95:
|
||||
the ``ToolResultPiece`` branch emits the additive
|
||||
``tool_result`` SSE frame (the seventh, optional event
|
||||
type — the A15 extension)."""
|
||||
type — the A15 extension). Phase 160 (task 02):
|
||||
``tools_offered=False`` (a request made WITHOUT tools —
|
||||
the deflected turn and its recovery) drops any
|
||||
``ToolCallPiece`` — a ``tools=None`` request cannot run a
|
||||
call, so it is rejected at the transport (no ``tool``
|
||||
frame, no visible markup) and counted in a local so one
|
||||
debug line can name it. The grounded first pass keeps its
|
||||
current behavior (``tools_offered=True``; ``run_agent``
|
||||
suppresses the cap-path calls internally)."""
|
||||
nonlocal thinking_chars, content_chars, retries_used
|
||||
dropped_calls = 0 # phase 160: tools=None requests only
|
||||
async for piece in pieces: # StreamPiece | ToolCallPiece | RetryPiece
|
||||
if isinstance(piece, ToolCallPiece):
|
||||
if not tools_offered:
|
||||
# Phase 160 (task 02): a tools=None request
|
||||
# cannot run a call — reject it (no ``tool``
|
||||
# frame, no visible markup); count it so one
|
||||
# debug line can name it. Never executed,
|
||||
# never rendered.
|
||||
dropped_calls += 1
|
||||
continue
|
||||
# Phase 37 (PLAN §4 extension; phase 70): one SSE
|
||||
# ``tool`` frame per model-requested call.
|
||||
# ``argument`` is the single string the model
|
||||
@@ -1013,9 +1063,15 @@ async def chat(
|
||||
else:
|
||||
content_chars += len(piece.text)
|
||||
yield sse_event({"type": "delta", "text": piece.text})
|
||||
if dropped_calls:
|
||||
logger.debug(
|
||||
"chat: %d tool call(s) from a tools=None request "
|
||||
"rejected, not shown (phase 160)",
|
||||
dropped_calls,
|
||||
)
|
||||
|
||||
try:
|
||||
async for frame in _pump(answer_stream):
|
||||
async for frame in _pump(answer_stream, tools_offered=tools_offered):
|
||||
yield frame
|
||||
if plan.deflected and deflected_filter is not None:
|
||||
scaffold_stripped = deflected_filter.stripped_chars
|
||||
@@ -1052,7 +1108,7 @@ async def chat(
|
||||
delay=settings.llm_retry_delay,
|
||||
scaffolding=recovery_filter,
|
||||
)
|
||||
async for frame in _pump(recovery_stream):
|
||||
async for frame in _pump(recovery_stream, tools_offered=False):
|
||||
yield frame
|
||||
scaffold_stripped += recovery_filter.stripped_chars
|
||||
if content_chars == 0:
|
||||
@@ -1197,7 +1253,8 @@ async def chat(
|
||||
"question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d "
|
||||
"suggested=%d tuning=%d "
|
||||
"kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r "
|
||||
"thinking_chars=%d tool_calls=%d total_ms=%d retries=%d scaffold_stripped=%d",
|
||||
"thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d "
|
||||
"retries=%d scaffold_stripped=%d",
|
||||
request.message,
|
||||
embed_ms,
|
||||
plan.top_score,
|
||||
@@ -1212,6 +1269,8 @@ async def chat(
|
||||
source_paths,
|
||||
thinking_chars,
|
||||
holder.tool_calls,
|
||||
holder.rounds,
|
||||
int(holder.cap_reached),
|
||||
total_ms,
|
||||
retries_used,
|
||||
scaffold_stripped,
|
||||
@@ -1269,6 +1328,13 @@ async def chat(
|
||||
related=related_refs,
|
||||
suggestions=plan.suggestions,
|
||||
answer_image=answer_ref,
|
||||
# Phase 160 (task 03): the turn reports the cap
|
||||
# hit — ``True`` only when the agent's round
|
||||
# cap forced the final no-tools answer.
|
||||
# Deflected turns run no agent, so the default
|
||||
# (``False`` → ABSENT on serialization) is
|
||||
# correct by construction.
|
||||
budget_exhausted=holder.cap_reached,
|
||||
).model_dump()
|
||||
)
|
||||
finally:
|
||||
|
||||
Reference in New Issue
Block a user