Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(agent): strip raw tool-scaffolding from streamed answers — deterministic filter with one bounded recovery

Browse Source
This commit is contained in:
ducoterra
2026-09-03 13:39:15 -04:00
parent 801639efcc
commit 575d6c88d0
38 changed files with 2793 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agent/reports/70_harness_aligned_tools/70_harness_aligned_tools__00_phase.a1.err
View File
.agent/reports/70_harness_aligned_tools/70_harness_aligned_tools__00_phase.a1.md
+20
View File
@@ -0,0 +1,20 @@
All criteria verified green; no defects found, no fixes needed.
## Phase 70 — Harness-Aligned Tools: verification report
**Verified (final pass — all 5 tasks already complete in `complete/`):**
- `AGENT_TOOLS` = exactly `ls`(path?) / `read`(path) / `grep`(pattern, path?); `rg "list_documents|read_document|search_documents" app/` → no matches
- `read` splits combined `source/path` at first slash, full content (A7); `grep` locked A5 (fixed substring, case-insens., 20×200, locator-only); `ls` prints `source: X | path: Y | title: Z`
- SSE `tool` frames: new names; `argument` = the single string passed (`grep`→pattern, else path) or null
- Kill switch (`MAX_ROUNDS=0` → one `tools=None` request) and LOW/deflected byte-identity pinned by unit tests and green
- `TOOLS_SECTION` rewritten for the new surface; frontend `read`/`grep`/`ls` render (old names still render for persisted chats); README §"Agent document tools (ls + read + grep)" updated
**Gates (exact commands):**
- `uv run pytest --cov=app` → exit 0, all pass; TOTAL coverage **99%** (>90%)
- `uv run ruff check . && uv run pyright` → "All checks passed!" / "0 errors"
- `uv run pytest tests/e2e/test_harness_aligned_tools.py -v --no-cov` → **3 passed** (ls→read, grep→read, wire-argument rule)
- Isolated regressions: `test_agent_document_tools` **4 passed**, `test_agent_unlimited_tools` **4 passed**, `test_search_tool` **3 passed**, `test_chat_rag` **3 passed**
- Commit `801639e` exists with the exact Commit-block message; phase dir in `.agent/phases/complete/`
**Deviations:** none — no code changes required this pass.
**Next pending phase:** `71_scaffolding_guardrails` (in `todo/`).
.agent/reports/70_harness_aligned_tools/70_harness_aligned_tools__00_phase.a1.validate
+75
View File
@@ -0,0 +1,75 @@
........................................................................ [ 5%]
........................................................................ [ 10%]
........................................................................ [ 15%]
........................................................................ [ 20%]
........................................................................ [ 25%]
........................................................................ [ 31%]
........................................................................ [ 36%]
........................................................................ [ 41%]
........................................................................ [ 46%]
........................................................................ [ 51%]
........................................................................ [ 57%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 83%]
........................................................................ [ 88%]
........................................................................ [ 93%]
........................................................................ [ 98%]
................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 149 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 139 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 213 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2722 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agent/reports/70_harness_aligned_tools/70_harness_aligned_tools__05_mock_e2e_commit.a1.md
+20
View File
@@ -0,0 +1,20 @@
Done — the empty `.err` is a harness runtime artifact (tracked-report convention, same as prior phases); working tree is clean.
**Phase 70, task 05 complete**
- `tests/e2e/mock_llm.py`: deterministic flows now emit `ls` (no args, `call_0`), `read` with the **combined** `source/path` (mock joins the two catalog fields — only parse change), `grep` with `pattern`; docstrings + phase-70 notes updated
- Existing E2E suites updated to new frames: `test_agent_document_tools.py` (`ls`→`read`), `test_agent_unlimited_tools.py` (multi-read `ls`+2×`read`), `test_search_tool.py` (`grep` + pattern argument)
- NEW `tests/e2e/test_harness_aligned_tools.py` (3 tests): READ flow (ls line → 📄 Reading line → quoted answer → done.sources incl. read doc → no raw `<|…|>`/`tool_call` markup in DOM), SEARCH flow (🔎 Searching for sentinel → answer), wire argument rule across both flows (ls→null, read→combined path, grep→pattern; names ⊆ {ls, read, grep})
- README agent-tools section rewritten for `ls`/`read(path)`/`grep(pattern, path?)` (combined identity + A5 locator semantics); `.env.example` header comment refreshed
- Old-name cleanup so the rg criterion holds: `app/api/docs.py` route fn → `list_indexed_documents`, `app/rag/llm.py` docstrings, `test_llm_client.py`, `test_llm_stream_teardown.py`, `test_sse_events.py`, `test_mock_tool_flow.py`, `test_api.py` docstring. Remaining matches: only the explicitly-marked legacy frontend branches/pins + the marked legacy-chat persistence pin + the mandated "old names absent" regression pins from tasks 01–02
**Gates (all green):**
- `uv run pytest` → exit 0; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- E2E in isolation: `test_harness_aligned_tools` 3 passed, `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_search_tool` 3, `test_chat_rag` 3
**Decisions:** mock `_tool_flow` tuple kept `(source, path)`; the join happens at the wire-emission point (keeps `test_mock_tool_flow.py` pins valid). `.env.example` included in the commit for atomicity (not in the block's path list, but task 05 touched it). Phase-dir move left to the harness per executor rules.
**Commit:** `801639e feat(agent): align the document tools with the harness-trained shape — ls, read(path), grep(pattern, path?)` (55 files, `--no-gpg-sign`)
**Next pending:** `71_scaffolding_guardrails/01_filter.md`
.agent/reports/70_harness_aligned_tools/70_harness_aligned_tools__05_mock_e2e_commit.a1.validate
+75
View File
@@ -0,0 +1,75 @@
........................................................................ [ 5%]
........................................................................ [ 10%]
........................................................................ [ 15%]
........................................................................ [ 20%]
........................................................................ [ 25%]
........................................................................ [ 31%]
........................................................................ [ 36%]
........................................................................ [ 41%]
........................................................................ [ 46%]
........................................................................ [ 51%]
........................................................................ [ 57%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 83%]
........................................................................ [ 88%]
........................................................................ [ 93%]
........................................................................ [ 98%]
................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 149 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 139 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 213 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2722 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK