# Task 04 — Mock Flow, Dedicated E2E Suite **Phase:** `72_teaching_refusals` · **Story:** `.agent/user_stories/agent-document-tools.md` ## Objective Prove the self-correction loop deterministically through the real UI: a mock-LLM flow that reproduces the incident's `ls(path='.')` misuse, receives the teaching refusal, corrects to a no-arg `ls()`, and answers from the catalog — a dedicated Playwright suite pinning the two-round shape on the SSE wire. (The live real-model acceptance gate is task 05 — this task is the deterministic half of the proof.) ## Work 1. `tests/e2e/mock_llm.py` — one new deterministic flow, checked in the flow table **before** the plain `TOOLS_TRIGGER` flow (the trigger phrases are disjoint substrings; the ordering rule follows the phase-71 convention): - `LS_TEACH_TRIGGER = "list the files in this directory"` — **and** the system prompt carries the `` section (grounded turn): * request 1 (tools offered, no `tool`-role result in the messages yet): stream ONLY `tool_calls` deltas — `ls` with `{"path": "."}` (synthetic id `call_0`), `finish_reason: "tool_calls"`, no content (the incident's misuse, deterministic); * request 2 (a `tool`-role result present that is **not** a catalog listing — i.e. the teaching refusal): stream a `tool_calls` delta — `ls` with no arguments (id `call_1`); * request 3 (a `tool`-role result whose first line matches the `^\d+ documents:` catalog header): a deterministic content answer — `These are the indexed documents: ` (the `source: X | path: Y | title: Z` line, parsed with the existing `_CATALOG_LINE_RE` machinery), `finish_reason: "stop"`. - Update the module docstring's flow table with the phase-72 note. 2. `tests/e2e/test_tool_path_teaching.py` (NEW — the phase's dedicated suite, house pattern, run in isolation; DB up, mock LLM): - Import a small fixture document set (house fixture pattern: one source, two documents with known `source`/`path`/`title`) and ask a question containing `LS_TEACH_TRIGGER`. - **Self-correction** — the turn settles (composer re-enables, `done` observed); the answer bubble contains the first document's `source:` and `path:` fields (the catalog reached the model and landed in the answer); no error banner. - **Two rounds on the wire** (the house SSE-capture pattern): the `tool` frames arrive in order — first `name:"ls"` with `argument:"."`, then `name:"ls"` with `argument:null` — and there is **no** third `tool` frame (the loop ended in one correction, not at the round cap). - **No regression to the plain flow** — a follow-up question containing `TOOLS_TRIGGER` (the single-read flow) in the same session still settles with the read flow's answer (the new flow did not swallow the existing trigger). ## Testing & Quality - E2E: the dedicated suite proves the loop shape (misuse → teaching refusal → corrected call → answer) through the real UI and the SSE wire; the existing regression E2E suites (mock-driven) stay green — run them as the regression check for this task. - Coverage: unit/integration coverage of `app/` stays >90% (this task adds test-only code; `uv run pytest --cov=app --cov-report=term-missing` as the check). ## Completion Criteria - [ ] `uv run pytest tests/e2e/test_tool_path_teaching.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`, mock LLM) - [ ] Regression E2E suites green in isolation: `test_harness_aligned_tools.py`, `test_agent_document_tools.py`, `test_agent_unlimited_tools.py`, `test_search_tool.py`, `test_chat_rag.py` - [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean - [ ] No commit in this task (the commit happens in task 05, after the real-model gate passes)