Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(agents): migrate .agent/ planning tree to .agents/

Browse Source 
Standardize on the .agents/ directory (shared with project skills):
phases/, user_stories/, reports/, screenshots/, validate.sh, and
phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves
history; runtime artifacts move alongside).

Updates every reference in AGENTS.md, README.md, .gitignore, app
docstrings, and test story headers. Historical KB content in data/
and the runtime pipeline.log transcript are left untouched.
This commit is contained in:
ducoterra
2026-09-05 10:57:07 -04:00
parent 766702c750
commit dbf2af26c6
1118 changed files with 664 additions and 664 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/reports/72_teaching_refusals/72_teaching_refusals__00_phase.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__00_phase.a1.md
+12
View File
@@ -0,0 +1,12 @@
All verification complete. Final state is coherent: the gate remains the sole failing criterion, and per the owner-locked fail-loud protocol the phase stops without a commit.
## Phase 72 — Final Verification Pass Report
- **Verified green:** tasks 01–04 work (teaching `ls` refusals, `read`/`grep` "did you mean" suggestions, `TOOLS_SECTION`/description contract copy, mock E2E) — all pins intact; `AGENT_TOOLS` names exactly `ls`/`read`/`grep`; LOW prompt byte-identical; KB re-imported (361 docs) and gate-runnable.
- **Iterated the gate's copy levers 3× (this pass):** instructive `ALREADY_IN_CONTEXT` refusal (behavior unchanged; killed the repeat-call loop in runs 3–4), `grep` "pass ONLY `pattern`" + "Rarely needed" descriptions, batched-call consequence clause in `TOOLS_SECTION`. Tried + **reverted** a `<document>`-anchor/`<documents>`-intro variant: it never stopped seed-doc reads and regressed the incident turn (cap-reached loop, run 4).
- **Gate result — FAIL (pass condition):** final run `gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 9/21 executed (43%) 2026-09-04`. Cumulative across 3 task-05 attempts: 12 live runs, 39–57% executed; condition 4 (≥90%) never met. Root cause (15/15 instances): `lite` obeys the user's "open it" and `read`s seed-context docs; also first-call source-name-scoped `grep`. Copy-resistant — a model-capability limit, not a copy lever.
- **STOP per fail-loud protocol: NO commit, no verdict line** (matches attempts a1/a2, `.agents/reports/72_teaching_refusals/`).
- `uv run pytest --cov=app --cov-report=term-missing` → **1489 passed, TOTAL 99%** (>90% ✓); `uv run ruff check . && uv run pyright` → clean ✓.
- E2E in isolation: `test_tool_path_teaching.py` 2 passed ✓; regressions `test_harness_aligned_tools`/`test_agent_document_tools`/`test_agent_unlimited_tools`/`test_search_tool`/`test_chat_rag` all green ✓.
- Completion criteria 1–6 ✓; **criterion 7 (real-model gate) ✗ → criterion 8 (commit) blocked** — phase stays in `todo/`.
- **Owner decision needed:** model upgrade (or unlock the ≥90% bar) — the locked `lite` + copy-only levers cannot pass this gate. Next pending phase: none (72 is the only `todo/` entry).
.agents/reports/72_teaching_refusals/72_teaching_refusals__00_phase.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__01_ls_teaching_refusal.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__01_ls_teaching_refusal.a1.md
+18
View File
@@ -0,0 +1,18 @@
Task 01 is complete — all gates green.
## Report
**Implemented (task 01 — `ls` teaching refusals):**
- `app/rag/agent.py`: added `LS_PATH_NOT_A_SOURCE` (stripped scope contains `/` — "looks like a document path, not a source name…") and `NO_SOURCE_NOT_A_DIRECTORY` (unknown source, no `/` — old line's prefix kept byte-identical + teaching parenthetical); `_execute_tool` `ls` branch uses them; both count in nothing and consume a round like today's refusal; `AGENT_TOOLS` `ls.path` description now says "a source name, not a file or directory path"; module docstring point 3 updated to the two new lines
- `tests/unit/test_agent.py`: byte-for-byte template pins; new pins for `ls(path='app/rag/importer.py')` (echoed, `tool_calls == 0`, tools stay offered, no registry lookup) and `ls(path='.')`; the `Ghost` pin replaced with the extended line; description pin updated
- `tests/integration/test_agent_tools.py`: `Ghost` pin updated to the constant; new path-like-scope case through `run_agent` on real Postgres (`kb`/`src`)
**Test / lint / coverage:**
- `uv run pytest tests/unit/test_agent.py tests/integration/test_agent_tools.py -v --no-cov` → 86 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1476 passed, TOTAL **99%** (agent.py 100%)
- `uv run ruff check . && uv run pyright` → clean
- E2E regression spot-checks (isolation): `test_harness_aligned_tools`, `test_agent_document_tools`, `test_agent_unlimited_tools`, `test_search_tool`, `test_chat_rag` → all passed
**Notable decisions:** kept a `startswith` unit pin on the old prefix (the task's locked requirement that it stay byte-identical); the terse line no longer exists as a refusal or expected value anywhere in `app/`/`tests/`. Path-like scopes refuse before any registry lookup (boom-pinned). Tool names/argument shapes unchanged (`ls`/`read`/`grep`).
**Next pending task:** `.agents/phases/todo/72_teaching_refusals/02_read_grep_path_suggestion.md` (not started, per protocol).
.agents/reports/72_teaching_refusals/72_teaching_refusals__01_ls_teaching_refusal.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 63%]
........................................................................ [ 68%]
........................................................................ [ 73%]
........................................................................ [ 78%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 92%]
........................................................................ [ 97%]
.................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 185 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2853 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__02_read_grep_path_suggestion.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__02_read_grep_path_suggestion.a1.md
+18
View File
@@ -0,0 +1,18 @@
Everything is green and the diff is coherent. Task 02 is complete.
## Report — Task 02: `read`/`grep` "did you mean 'source/path'?" suggestions
**Implemented**
- `app/rag/agent.py`: `SUGGESTION_LIMIT = 3`; module-level `find_path_candidates` (exact-`path` or `/suffix` match, case-sensitive, catalog order, one bulk `all_documents` query, refusal-path only); `NO_DOCUMENT_DID_YOU_MEAN` / `NO_DOCUMENT_DID_YOU_MEAN_MANY` templates + `_no_document_refusal` helper wired into both the `read` and scoped-`grep` miss paths (zero-candidate and bare-name refusals byte-identical, no DB lookup for bare names); `read`/`grep` `path` descriptions append the bare-path contract; module docstring point 3 updated.
- `tests/unit/test_agent.py`: +7 new pins (exact/suffix match, two-source "one of", four-source cap-3, bare-name & bare-filename no-DB-lookup `_boom` guards, seed-doc dedupe-precedence, scoped-grep suggestion) + description/constant pins updated byte-for-byte; 3 existing pins adjusted for the now-running zero-candidate lookup.
- `tests/integration/test_agent_tools.py`: 2 new end-to-end `run_agent` cases on real Postgres (bare path under one source; same path under two sources) — refusal followed by the scripted corrected combined-identity call succeeding; existing refusal pins verified unchanged (none match fixture `path`s).
**Results**
- `uv run pytest tests/unit/test_agent.py tests/integration/test_agent_tools.py -v --no-cov` → 96 passed (DB up)
- `uv run pytest --cov=app --cov-report=term-missing` → 1486 passed, TOTAL **99%** (>90%; `agent.py` 100%)
- `uv run ruff check . && uv run pyright` → clean
- E2E regressions in isolation: `test_harness_aligned_tools` + `test_agent_document_tools` (7), `test_agent_unlimited_tools` + `test_search_tool` + `test_chat_rag` (10) — all green
**Notable deviation**: the task's example bullets (`read('caddy.md')`, `'A/x.md'`) imply suggestions for slash-less args, but the task's normative spec (stated twice) plus its own completion criterion gate the lookup on `"/" in arg` and pin zero DB calls for bare names. I implemented the `/` gate and adapted the suffix/multi-source examples to slash-containing paths; a bare filename (`caddy.md`) is pinned to keep today's no-DB refusal — flag for owner if bare filenames should also be suggested.
**Next pending task**: `.agents/phases/todo/72_teaching_refusals/03_prompt_copy.md`
.agents/reports/72_teaching_refusals/72_teaching_refusals__02_read_grep_path_suggestion.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 92%]
........................................................................ [ 96%]
.............................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__03_prompt_copy.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__03_prompt_copy.a1.md
+17
View File
@@ -0,0 +1,17 @@
Task 03 complete — all gates green.
**Implemented**
- Rewrote `TOOLS_SECTION` in `app/rag/prompts.py`: `ls` clause now states its optional `path` is a source name (e.g. `'homelab'`), **not** a directory or file path, omit to list all; `read` clause requires the combined `source/path` exactly as shown in `ls` output — *including the source name*, bare path will not resolve; `grep` keeps its locator copy plus the same bare-path note. Single paragraph, `<tools>`/`</tools>` shape, HIGH-only, same as before.
- Updated module/constant/`build_high_prompt` docstrings to document the phase-72 contract.
- `tests/unit/test_prompts.py`: dropped the stale `"pass a source name as \`path\`"` pin; added `test_tools_section_phase72_contract_clauses` (byte-for-byte pins for the ls source-name clause, the read combined-identity clause, and exactly-two bare-path notes) and `test_tools_section_phase72_clauses_in_high_prompt_not_low`.
**Test / lint results**
- `uv run pytest tests/unit/test_prompts.py -v --no-cov` → 32 passed
- `uv run pytest` → 1488 passed
- `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%)
- `uv run ruff check . && uv run pyright` → clean (0 errors)
- E2E regression (mock-keying check): `test_harness_aligned_tools` + `test_agent_document_tools` + `test_agent_unlimited_tools` → 11 passed; `test_search_tool` + `test_chat_rag` → 6 passed (isolation)
**Decisions**: kept all pre-existing substrings the `test_agent.py` HIGH-prompt pins depend on; mirror the exact AGENT_TOOLS description wording ("a bare document path (without the source name) will not resolve") so prompt, descriptions, and refusals state one contract. No deviations.
**Next pending task**: `.agents/phases/todo/72_teaching_refusals/04_mock_e2e.md` (mock `ls`-misuse flow + dedicated E2E suite).
.agents/reports/72_teaching_refusals/72_teaching_refusals__03_prompt_copy.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__04_mock_e2e.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__04_mock_e2e.a1.md
+17
View File
@@ -0,0 +1,17 @@
All green. Task 04 complete.
**Implemented**
- `tests/e2e/mock_llm.py` (+142, purely additive): phase-72 `LS_TEACH_TRIGGER` flow — request 1 streams `ls {"path": "."}` (`call_0`, the incident's misuse), request 2 (teaching refusal present, not a catalog listing) streams no-arg `ls()` (`call_1`), request 3 (first tool result line matches `^\d+ documents:`) answers `These are the indexed documents: <first catalog line>` via `_CATALOG_LINE_RE`/new `_CATALOG_HEADER_RE` machinery; checked before the plain `TOOLS_TRIGGER` flow (disjoint triggers); docstring flow table updated.
- `tests/e2e/test_tool_path_teaching.py` (new dedicated suite): one-source/two-doc fixture; turn 1 pins self-correction (answer carries first doc's `source:`/`path:`, two UI tool lines, no error banner) and the two-round wire shape (`ls "."` → `ls null`, no third tool frame); turn 2 in the same session pins the plain single-read flow unchanged.
**Tests / lint / coverage**
- `uv run pytest tests/e2e/test_tool_path_teaching.py -v --no-cov` → 2 passed (isolated, DB up, mock LLM)
- Regression suites isolated: `test_harness_aligned_tools` 3✓, `test_agent_document_tools` 4✓, `test_agent_unlimited_tools` 4✓, `test_search_tool` 3✓, `test_chat_rag` 3✓
- `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%)
- `uv run ruff check . && uv run pyright` → clean (0 errors)
**Decisions**
- Fixture: catalog-first doc is catalog-only (no chunks); the *second* doc carries the retrieval chunk — a seed that is also catalog-first makes the follow-up single-read flow dedupe ("Already in your context") and loop to the round cap (caught and fixed on first run).
- Fixed a `\d` SyntaxWarning in the mock docstring introduced by the new bullet.
**Next pending task:** `.agents/phases/todo/72_teaching_refusals/05_real_model_gate.md`
.agents/reports/72_teaching_refusals/72_teaching_refusals__04_mock_e2e.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a1.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a1.md
+12
View File
@@ -0,0 +1,12 @@
All gates verified. The repo is in a fully functional state; the live gate FAILED after 8 genuine iterations, so per the task's fail-loud protocol I stopped **without committing** and without recording a verdict line.
## Report
- **Implemented:** `scripts/agent_realmodel_check.py` (house probe pattern — preconditions exit 2, fixed 10-question battery derived from the live catalog, the four locked pass conditions, stable verdict line, exits 0/1/2); a real gate KB (38 genuine homelab docs under the registered source `homelab`, nested paths, from `/tmp/bor-gate-kb`).
- **Gate result:** 8 live runs vs `lite` (temp 0.4): **41 / 53 / 39 / 42 / 57 / 47 / 53 / 48 %** executed. Conditions 1–3 passed every run (10/10 answered, 0 caps, 10/10 tool-turns); condition 4 (≥90 % executed) never met. Final verdict: `gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 10/21 executed (48%) 2026-09-03`.
- **Refusals the model hit (copy-resistant):** (a) first call of each grep turn = `grep(pattern=…, path='homelab')` — source-name scoping, 8/8 runs; the locked byte-identical `No document at 'homelab' — check the ls output.` can't teach bare args (locked decision) and the model reads it as "no matches in the homelab directory"; (b) reads of docs already in the `<documents>` seed (turns 4/5/9, 8/8 runs) → locked `Already in your context.`; (c) occasional repeated-refused calls / batched multi-call extras.
- **Levers exhausted:** TOOLS_SECTION ×6 variants, AGENT_TOOLS function+param descriptions ×4+ (incl. wrong/right examples, rules-first, negative-example-priming removal); the did-you-mean teaching fired (run 3, bare path) and self-corrected in one round — it works; the ls teaching refusals never triggered (model consistently used the registered source).
- **Tests/lint/coverage:** `uv run pytest` → 1488 passed; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99 %** (>90 %); `uv run ruff check . && uv run pyright` → clean.
- **E2E in isolation:** `test_tool_path_teaching.py` 2 passed; regressions `test_harness_aligned_tools` 3, `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_search_tool` 3, `test_chat_rag` 3 — all passed.
- **Decisions:** STOP per the fail-loud protocol — no commit, no verdict line in `app/rag/agent.py`, tree left at the latest iteration (all pins in sync, everything green); KB re-imported so the gate is re-runnable (re-import after any E2E run: `uv run python -m scripts.import_docs --source /tmp/bor-gate-kb/homelab`).
- **Next pending task:** none — task 05 is phase 72's last. The phase stays open for the owner: the remaining gap is a model-capability limit (first-call source-name scoping + seed-doc re-reads) vs the locked ≥90 % bar — a model/lock decision, not a copy lever.
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a1.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a2.err
View File
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a2.md
+13
View File
@@ -0,0 +1,13 @@
All clean — no strays, preconditions pass, repo gate-runnable. Everything is verified green; here is the final report.
## Report (retry attempt)
- **What this attempt did:** the previous run's failure was harness plumbing (final report never delivered) — no code failure. Re-verified the complete state, investigated the shared-DB change (an external import of the tracked `tests/fixtures/docs` fixture at 21:46 EDT layered over my 38-doc gate KB → 51-doc catalog), and ran one fresh live gate against the current live catalog.
- **Live gate (run 9, 51-doc catalog):** `gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 11/29 executed (38%) 2026-09-03` — condition 4 (≥90 % executed) missed; conditions 1–3 passed. Full dataset (9 runs): 41/53/39/42/57/47/53/48 (38-doc KB) + 38 (mixed). Positive: the no-source teaching refusal self-corrected in exactly one extra round on a real unknown-scope trap (turn 02).
- **STOP per the fail-loud protocol:** no commit, no verdict line in `app/rag/agent.py`, phase stays in `todo/`. The gap is a model-capability limit (first-call source-name scoping on `grep`, re-reads of seed-context docs, repeated refused calls, an `ls(path='/')` loop on turn 10), not a copy lever — 8 section/schema variants exhausted.
- **Tests:** `uv run pytest` → 1488 passed (exit 0).
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99 %** (>90 %).
- **Lint:** `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E in isolation:** `test_tool_path_teaching` 2 passed; regressions `test_harness_aligned_tools` 3, `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_search_tool` 3, `test_chat_rag` 3 — all passed.
- **Repo state:** `scripts/agent_realmodel_check.py` + all pins in sync; KB left gate-runnable (preconditions pass); no task/phase files touched.
- **Next pending task:** none — 05 is phase 72's last task; the owner must decide (model upgrade vs. the locked ≥90 % bar) before the gate can pass.
.agents/reports/72_teaching_refusals/72_teaching_refusals__05_real_model_gate.a2.validate
+77
View File
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
........................................................................ [ 29%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 43%]
........................................................................ [ 48%]
........................................................................ [ 53%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 67%]
........................................................................ [ 72%]
........................................................................ [ 77%]
........................................................................ [ 82%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 96%]
................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 200 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 94 3 97%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2868 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK