feat(agent): strip raw tool-scaffolding from streamed answers — deterministic filter with one bounded recovery

2026-09-03 13:39:15 -04:00
parent 801639efcc
commit 575d6c88d0
38 changed files with 2793 additions and 50 deletions
@@ -0,0 +1,61 @@
+# Task 02 — Filter Integration in `chat_stream` / `chat_stream_retried`
+
+**Phase:** `71_scaffolding_guardrails` · **Story:** n/a (owner request from chat, 2026-09-03)
+
+## Objective
+Route `delta.content` through the caller-supplied `ScaffoldingFilter` in
+`app/rag/llm.py` — content filtered, thinking raw, tail flushed before tool-call
+materialization — with `None` keeping today's byte-identical raw path.
+
+## Work
+1. `app/rag/llm.py` — `chat_stream(..., scaffolding: "ScaffoldingFilter | None =
+   None)"` (type imported under `TYPE_CHECKING` or as a string annotation to keep
+   the module's import graph clean — the filter type is only needed for typing):
+   - Content path: `content = delta.content` → when a filter is present,
+     `cleaned = scaffolding.feed(content)`; yield `StreamPiece("content",
+     cleaned)` only when `cleaned` is non-empty (an empty clean result yields
+     **nothing** — no empty `delta` frames). Without a filter the existing
+     `if content: yield` stands untouched (byte-identical).
+   - Thinking path: unchanged — `reasoning_content` pieces are never filtered
+     (locked: the scratchpad stays raw).
+   - End of stream: after the `async for` exhausts and **before** the tool-call
+     materialization block (`if calls and not emitted: …` and the
+     finish-reason emission), `tail = scaffolding.flush()` when a filter is
+     present; yield a content piece for the tail when non-empty. Order pinned:
+     flushed-tail content precedes `ToolCallPiece`s (content-before-tools wire
+     convention).
+   - Teardown (phase 48) untouched: the `finally: await stream.close()` behavior is
+     independent of the filter.
+   - Docstring: a short "Scaffolding guardrail (phase 71)" paragraph — the filter
+     is caller-owned (one per request), content-only, `None` = raw path.
+2. `app/rag/llm.py` — `chat_stream_retried(..., scaffolding=None)` — pass-through
+   to `llm.chat_stream`. The same filter object is used across retry attempts of
+   one logical request: safe by construction (a restarted attempt only happens
+   when no piece was emitted, i.e. the filter was never fed — note this in the
+   docstring).
+3. `tests/unit/test_llm_client.py` — scripted-chunk tests (the existing
+   fake-client pattern):
+   - with a filter: a chunk carrying a span mid-stream → the `delta` pieces carry
+     only the clean text; a span split across two chunks → no partial emit;
+     `filter.stripped_chars` correct after consumption.
+   - thinking pieces pass through raw even when they contain a span (pinned).
+   - `scaffolding=None` → the yielded pieces are byte-identical to the pre-phase
+     raw path (a span in content is yielded verbatim — the raw contract for
+     callers that opt out).
+   - flush ordering: a trailing partial-then-complete tail yields its content
+     piece **before** the materialized `ToolCallPiece` (a chunk stream that ends
+     with `…clean tail` + tool_calls deltas).
+   - teardown interaction: the phase-48 `aclose()`/abandon tests
+     (`tests/unit/test_llm_stream_teardown.py`) stay green with and without a
+     filter (run, don't rewrite).
+
+## Testing & Quality
+- Unit: `tests/unit/test_llm_client.py` (+ the untouched teardown suite green).
+- Coverage: **>90%** on this task's modified code (`app/rag/llm.py`).
+
+## Completion Criteria
+- [ ] `uv run pytest tests/unit/test_llm_client.py tests/unit/test_llm_stream_teardown.py
+      -v --no-cov` green.
+- [ ] `scaffolding=None` callers (including the kill-switch no-tools requests) are
+      byte-identical (pinned).
+- [ ] `uv run ruff check . && uv run pyright` clean.