brain-of-reese/.agents/phases/complete/71_scaffolding_guardrails/02_llm_integration.md

Task 02 — Filter Integration in chat_stream / chat_stream_retried

Phase: 71_scaffolding_guardrails · Story: n/a (owner request from chat, 2026-09-03)

Objective

Route delta.content through the caller-supplied ScaffoldingFilter in app/rag/llm.py — content filtered, thinking raw, tail flushed before tool-call materialization — with None keeping today's byte-identical raw path.

Work

1. app/rag/llm.py — chat_stream(..., scaffolding: "ScaffoldingFilter | None = None)" (type imported under TYPE_CHECKING or as a string annotation to keep the module's import graph clean — the filter type is only needed for typing):
   • Content path: content = delta.content → when a filter is present, cleaned = scaffolding.feed(content); yield StreamPiece("content", cleaned) only when cleaned is non-empty (an empty clean result yields nothing — no empty delta frames). Without a filter the existing if content: yield stands untouched (byte-identical).
   • Thinking path: unchanged — reasoning_content pieces are never filtered (locked: the scratchpad stays raw).
   • End of stream: after the async for exhausts and before the tool-call materialization block (if calls and not emitted: … and the finish-reason emission), tail = scaffolding.flush() when a filter is present; yield a content piece for the tail when non-empty. Order pinned: flushed-tail content precedes ToolCallPieces (content-before-tools wire convention).
   • Teardown (phase 48) untouched: the finally: await stream.close() behavior is independent of the filter.
   • Docstring: a short "Scaffolding guardrail (phase 71)" paragraph — the filter is caller-owned (one per request), content-only, None = raw path.
2. app/rag/llm.py — chat_stream_retried(..., scaffolding=None) — pass-through to llm.chat_stream. The same filter object is used across retry attempts of one logical request: safe by construction (a restarted attempt only happens when no piece was emitted, i.e. the filter was never fed — note this in the docstring).
3. tests/unit/test_llm_client.py — scripted-chunk tests (the existing fake-client pattern):
   • with a filter: a chunk carrying a span mid-stream → the delta pieces carry only the clean text; a span split across two chunks → no partial emit; filter.stripped_chars correct after consumption.
   • thinking pieces pass through raw even when they contain a span (pinned).
   • scaffolding=None → the yielded pieces are byte-identical to the pre-phase raw path (a span in content is yielded verbatim — the raw contract for callers that opt out).
   • flush ordering: a trailing partial-then-complete tail yields its content piece before the materialized ToolCallPiece (a chunk stream that ends with …clean tail + tool_calls deltas).
   • teardown interaction: the phase-48 aclose()/abandon tests (tests/unit/test_llm_stream_teardown.py) stay green with and without a filter (run, don't rewrite).

Testing & Quality

• Unit: tests/unit/test_llm_client.py (+ the untouched teardown suite green).
• Coverage: >90% on this task's modified code (app/rag/llm.py).

Completion Criteria

• uv run pytest tests/unit/test_llm_client.py tests/unit/test_llm_stream_teardown.py -v --no-cov green.
• scaffolding=None callers (including the kill-switch no-tools requests) are byte-identical (pinned).
• uv run ruff check . && uv run pyright clean.