brain-of-reese/.agent/phases/complete/71_scaffolding_guardrails/03_recovery_policy.md

Task 03 — Recovery Policy: One Bounded tools=None Retry + Terminal Error

Phase: 71_scaffolding_guardrails · Story: n/a (owner request from chat, 2026-09-03)

Objective

When a round/turn's visible content ends up empty because scaffolding was the whole answer, run exactly one deterministic recovery (same turn, tools=None, fixed correction line in the system prompt, fresh filter); a second empty reply settles with a dedicated error frame. Both the grounded agent loop and the deflected path get the policy; the per-turn log line gains scaffold_stripped=N.

Work

1. app/rag/agent.py:
   • CORRECTION_INSTRUCTION: str — the harness-owned constant (verbatim, single line): "Your previous reply contained raw tool-call markup, which is not interpreted here. Answer the user's question directly in plain text — no tool syntax." (The E2E mock in task 05 keys on a stable substring of it — pick the exact constant now; the mock copies it.)
   • class MalformedReplyError(LLMError) — raised only by the recovery policy (never from inside a stream, so chat_stream_retried's retry rule never sees it). Module docstring: the phase-71 note (deterministic-only, owner permission 2026-09-03).
   • run_agent — per round: create a fresh ScaffoldingFilter, pass it to chat_stream_retried(..., scaffolding=round_filter), and count the round's visible content (sum of the lengths of the yielded StreamPiece("content", …) texts — the filtered ones). After the round, when not calls (today's "the answer was streamed" exit):
     • if round content > 0 → return (as today).
     • if round content == 0 and round_filter.stripped_chars > 0 → one recovery: messages_recovered = [*messages[:-1], {"role": "system", "content": system_prompt + "\n" + CORRECTION_INSTRUCTION}, messages[-1]] (the correction folds into the ORIGINAL single system message — provider-safe; the user message stays last) — one chat_stream_retried request with tools=None, a fresh filter, the same retries/delay budget; yield its pieces through the normal piece flow. If the recovery's visible content > 0 → return. Otherwise → logger.warning + raise MalformedReplyError(…).
     • if round content == 0 and nothing was stripped → return (today's empty/thinking-only answer behavior — the UI handles it; unchanged).
     • Log one warning per strip event here: logger.warning("agent: stripped N chars of tool-scaffolding in round %d: %r", …) with the stripped span truncated to 200 chars (the capture mechanism for new registry entries — the filter exposes the stripped spans for this; add a stripped_spans: list[str] to the filter if needed).
     • A scaffolding-only round that also carried tool calls needs no recovery (the clean content stands / the tool ran) — the policy keys on the not calls exit only (pinned).
2. app/api/chat.py — the deflected path (the grounded path is covered by run_agent):
   • Create one ScaffoldingFilter for the turn's request, pass it to chat_stream_retried(..., scaffolding=filter); count visible content across the piece loop (a content_chars counter next to thinking_chars).
   • After the piece loop (deflected branch only): content == 0 and filter.stripped_chars > 0 → one recovery request: the same messages with the system prompt extended by CORRECTION_INSTRUCTION (import from app.rag.agent), tools=None, a fresh filter, the same retry budget; stream its pieces through the SAME piece-handling code (extract the piece loop into a small inner helper/coroutine to avoid duplicating the thinking/tool/retry/delta handling — the extraction must be behavior-preserving for the first pass, pinned by the existing integration suite). If the recovery content > 0 → continue to the normal done flow; else → the terminal path below.
   • Catch MalformedReplyError before the generic LLMError handler: settled = True, yield ChatErrorEvent(detail="The model returned a malformed reply — please try again."), return (no query_log row, no done — the existing terminal-error semantics; the generic "dropped the connection" copy stays for transport failures).
   • Per-turn log line: append scaffold_stripped=N after retries=N — the sum across the turn's requests (rounds + any recovery; 0 on clean turns — uniform field, the phase-67 retries=N pattern). The recovery does not bump retries=N (it is not a phase-67 endpoint-retry).
   • Module docstring: the phase-71 paragraph (deterministic guardrail + recovery + log field).
3. tests/unit/test_agent.py — the grounded matrix (scripted fake LLM, monkeypatched DB): scaffolding-only round → exactly two model requests, the second tools=None with CORRECTION_INSTRUCTION in its system prompt → clean answer ends the turn, done-side state normal (holder untouched by the recovery); scaffolding twice → MalformedReplyError (assert it subclasses LLMError); scaffolding + real content → one request only, clean content yielded, no recovery; clean turn → one request, no correction in any system prompt; the round cap + kill-switch tests stay green.
4. tests/integration/test_chat_api.py — the deflected matrix over POST /api/chat (mock LLM): scaffolding-only deflected turn → clean recovery answer + done frame; scaffolding twice → the error frame with the dedicated copy, no done, no query_log row (assert against the table); mixed scaffolding+content → clean answer, scaffold_stripped>0 in the log line; the log-line pin gains scaffold_stripped=0 on clean turns and the summed value on stripped turns.

Testing & Quality

• Unit: tests/unit/test_agent.py (grounded matrix).
• Integration: tests/integration/test_chat_api.py (deflected matrix + log line).
• Coverage: >90% on this task's modified code (app/rag/agent.py, app/api/chat.py).

Completion Criteria

• uv run pytest tests/unit/test_agent.py tests/integration/test_chat_api.py -v --no-cov green.
• Exactly one recovery per turn, on both paths; the recovery request is tools=None + correction line; a second empty reply → the dedicated error frame, no query_log row.
• Clean turns: no correction in any system prompt, scaffold_stripped=0, request counts unchanged (kill-switch/deflection byte-identical pins green).
• uv run ruff check . && uv run pyright clean.