Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 112_honesty_gate_weak_hits
Build and Push Containers / build-and-push-app (push) Successful in 2m15s
Details
Build and Push Containers / build-and-push-db (push) Successful in 14s
Details

Browse Source 
**Phase 112 — final verification pass (all 4 tasks already complete in `complete/`):**

- Verified gate fix: `app/api/chat.py::plan_turn` — HIGH iff `best_cosine >= relevance_threshold` OR (`fts_hits > 0` AND `best_cosine >= lexical_support_floor`); `lexical_support_floor` (default 0.35, `BOR_LEXICAL_SUPPORT_FLOOR`, bounds-validated) in `app/config.py` + `.env.example`; A8 revision note (2026-09-14) in `.agents/PLAN.md`.
- Verified prompt contract: `app/rag/prompts.py` diff is docstring-only (dated owner-decision-iii entry); `tests/unit/test_prompt_lock.py` byte-pins PERSONA/TOOLS_SECTION/DEFLECT body (sha256+length).
- Verified README: L11 + L575 deflection copy refreshed; `grep "haven't done anything" README.md` → no hits; disclosed-answer behavior documented.
- Tests: `uv run pytest --cov=app --cov-report=term-missing` → **2378 passed, 99% coverage (>90%)**; includes Mongolia-quadrant unit pins (fts>0 + cosine<floor → LOW).
- E2E in isolation: `uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov` → **4 passed** (out-of-KB question: `deflected=true`, `sources==[]`, 2–3 suggestions); regression `test_chat_rag.py` + `test_retrieval_quality.py` → **7 passed**.
- Lint/types: `uv run ruff check .` → clean; `uv run pyright` → 0 errors.

**Completion criteria:** weak-FTS→LOW unit-pinned ✅ · no false citations + 2–3 alternatives E2E ✅ · prompts byte-identical (test-pinned) + README matches ✅ · suite/coverage/e2e/lint all green ✅ · commit + phase move → left to harness (no `git commit` run, per rules; changes in working tree).

**Deviations:** none. Next pending phase: `113_source_chip_quality`.
This commit is contained in:
ducoterra
2026-09-15 00:37:38 -04:00
parent 2683128876
commit 1374faf136
36 changed files with 1240 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__00_phase.a1.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__00_phase.a1.md
+12
View File
@@ -0,0 +1,12 @@
**Phase 112 — final verification pass (all 4 tasks already complete in `complete/`):**
- Verified gate fix: `app/api/chat.py::plan_turn` — HIGH iff `best_cosine >= relevance_threshold` OR (`fts_hits > 0` AND `best_cosine >= lexical_support_floor`); `lexical_support_floor` (default 0.35, `BOR_LEXICAL_SUPPORT_FLOOR`, bounds-validated) in `app/config.py` + `.env.example`; A8 revision note (2026-09-14) in `.agents/PLAN.md`.
- Verified prompt contract: `app/rag/prompts.py` diff is docstring-only (dated owner-decision-iii entry); `tests/unit/test_prompt_lock.py` byte-pins PERSONA/TOOLS_SECTION/DEFLECT body (sha256+length).
- Verified README: L11 + L575 deflection copy refreshed; `grep "haven't done anything" README.md` → no hits; disclosed-answer behavior documented.
- Tests: `uv run pytest --cov=app --cov-report=term-missing` → **2378 passed, 99% coverage (>90%)**; includes Mongolia-quadrant unit pins (fts>0 + cosine<floor → LOW).
- E2E in isolation: `uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov` → **4 passed** (out-of-KB question: `deflected=true`, `sources==[]`, 2–3 suggestions); regression `test_chat_rag.py` + `test_retrieval_quality.py` → **7 passed**.
- Lint/types: `uv run ruff check .` → clean; `uv run pyright` → 0 errors.
**Completion criteria:** weak-FTS→LOW unit-pinned ✅ · no false citations + 2–3 alternatives E2E ✅ · prompts byte-identical (test-pinned) + README matches ✅ · suite/coverage/e2e/lint all green ✅ · commit + phase move → left to harness (no `git commit` run, per rules; changes in working tree).
**Deviations:** none. Next pending phase: `113_source_chip_quality`.
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__00_phase.a1.validate
+102
View File
@@ -0,0 +1,102 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 93%]
........................................................................ [ 96%]
........................................................................ [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 208 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4079 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a1.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a1.md
+1
View File
@@ -0,0 +1 @@
All 2373 tests pass with 99% coverage. Now let me run the linter:
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a1.validate
+101
View File
@@ -0,0 +1,101 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 94%]
........................................................................ [ 97%]
..................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 205 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4076 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a2.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a2.md
+1
View File
@@ -0,0 +1 @@
All 2373 tests pass with 99% coverage. Now let me run the linter:
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__01_gate_meaningful_fts.a2.validate
+101
View File
@@ -0,0 +1,101 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 94%]
........................................................................ [ 97%]
..................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 205 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4076 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__02_gate_tests.a1.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__02_gate_tests.a1.md
+19
View File
@@ -0,0 +1,19 @@
All green. Final verification complete.
**Task 02 (gate tests) — complete**
Implemented:
- Unit quadrants in `tests/unit/test_chat_gate.py`: added the code-defaults quadrant (fts>0 + cosine 0.50 ≥ floor 0.35 → HIGH) and pinned the new done-frame contract on the endpoint (deflected → `done["sources"] == []`, `query_log.sources` still records retrieval)
- E2E `test_honest_deflection.py`: new `test_out_of_kb_question_deflects_without_citations` — "Who composed Rhapsody in Blue?" (LLM-known, absent from fixture KB; fts_hits≥1, cosine 0.124 < floor 0.15 — the fts>0/cosine<floor quadrant pinned end-to-end) → `deflected: true`, `sources: []`, 2–3 suggestions
- Server: deflected turns' `done` frame now sends `sources=[]` (no false citations — the L2 acceptance; UI chips nothing); `query_log.sources` + per-turn log line keep recording the retrieval (observability unchanged, phase-113 A3 precedent)
- Updated the phase-04 integration test that pinned the old contract; reworded `test_retrieval_quality`'s kafkabridge question ("How does kafkabridge handle DNS?") — the old one (cos 0.134) now correctly falls in the new deflection quadrant
Results:
- `uv run pytest` → 2374 passed; `uv run pytest --cov=app --cov-report=term-missing` → 99% on `app/` (>90%)
- `uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov` → 4 passed (in isolation); `test_chat_rag.py` → 3 passed; `test_retrieval_quality.py` → 4 passed
- At-risk audit: scanned every E2E question literal for gate flips (only the intended one + branch-independent/empty-KB suites); ran 13 gate-sensitive E2E suites (38 tests) → all passed
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
Decisions: the task's example question scores 0.303 (> E2E threshold 0.30) under the mock, so it would ground — replaced with Rhapsody in Blue, which also exercises the new quadrant; deflected-done-`sources=[]` is a deliberate contract change required by the task's acceptance (matches phase 113's assumed end state).
Next pending task: `112_honesty_gate_weak_hits/03_prompt_contract_documentation.md`
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__02_gate_tests.a1.validate
+101
View File
@@ -0,0 +1,101 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 94%]
........................................................................ [ 97%]
...................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 208 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4079 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__03_prompt_contract_documentation.a1.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__03_prompt_contract_documentation.a1.md
+10
View File
@@ -0,0 +1,10 @@
All green. Task 03 complete.
**Report**
- **prompts.py docstring**: dated entry recording the 2026-09-15 interactive-test finding (stochastic "Ulaanbaatar" disclosed answer vs. clean deflection on re-run, Mongolia case) and owner decision (iii, 2026-09-14): disclosed answers acceptable + documented; gate (A8 revised) is the deterministic lever; options (i)/(ii) left open. Prompt strings untouched (diff is docstring-only).
- **Prompt-lock pin**: new `tests/unit/test_prompt_lock.py` (4 tests) — sha256 + exact prefix/suffix + total length pinning `PERSONA` (706 B), `TOOLS_SECTION` (2273 B), the inline LOW `DEFLECT_MODE` body (259 B), and both `_base` lengths; any byte change fails.
- **README**: added the rare disclosed-answer sentence to the Honest-deflection bullet (work item 2). Defect fix (task 01 leftovers): refreshed the now-stale A8 gate copy in the retrieval paragraph, env table (added `BOR_LEXICAL_SUPPORT_FLOOR` row), tuning bullet, and two spots in `app/rag/retriever.py`.
- **Results**: `uv run pytest --cov=app --cov-report=term-missing` → 2378 passed, **99%** on `app/` (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors; `uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov` → 4 passed in isolation.
- **Decisions**: kept the quoted *"I haven't done anything like that"* openings intact — task 04 rewrites those; my gate-copy fixes there will fold into it.
- **Next pending task**: `04_readme_deflection_copy.md`.
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__03_prompt_contract_documentation.a1.validate
+102
View File
@@ -0,0 +1,102 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 93%]
........................................................................ [ 96%]
........................................................................ [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 208 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4079 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__04_readme_deflection_copy.a1.err
View File
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__04_readme_deflection_copy.a1.md
+12
View File
@@ -0,0 +1,12 @@
**Task 04 complete — README deflection copy refreshed (docs-only, no code changes).**
- Rewrote README intro (L11): now "it admits it — and offers 2-3 concrete alternative questions about things it **does** have notes on" (no exact-copy promise; the mandated opening was removed in the 2026-08-22 prompt revision).
- Rewrote Troubleshooting "Honest deflection" entry (L579): amber-bubble reference no longer quotes the removed opening; kept the accurate amber-banner + "Maybe try" chips description.
- Third stale occurrence (L588, the "reply opens with…" clause) found via the required README grep — updated to admit + 2-3 concrete alternatives. `grep -rn "haven't done anything" README.md` → no hits (exit 1).
- Tests grep: all `haven't done anything` hits in `tests/` are mock-LLM/stub-LLM *fixture* copies (the mock's own reply, which already matches the new contract; `DEFLECT_PHRASE` is documented as "the mock's deflection answer must match this") — none assert the app/prompt copy, so no test changes were needed.
- `uv run pytest` → 2378 passed.
- `uv run pytest --cov=app` → TOTAL 99% (>90% gate).
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors.
- `uv run pytest tests/e2e/test_honest_deflection.py -v --no-cov` → 4 passed (phase E2E green in isolation; last task of phase 112).
- No git add/commit (left in working tree for the harness phase commit).
- Next pending task: `.agents/phases/todo/113_source_chip_quality/01_usefulness_bar_sources.md`
.agents/reports/112_honesty_gate_weak_hits/112_honesty_gate_weak_hits__04_readme_deflection_copy.a1.validate
+102
View File
@@ -0,0 +1,102 @@
........................................................................ [ 3%]
........................................................................ [ 6%]
........................................................................ [ 9%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 18%]
........................................................................ [ 21%]
........................................................................ [ 24%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 39%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 54%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 63%]
........................................................................ [ 66%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 78%]
........................................................................ [ 81%]
........................................................................ [ 84%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 93%]
........................................................................ [ 96%]
........................................................................ [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain-of-reese/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.7-final-0 ________________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 208 1 99%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 94 0 100%
app/api/docs.py 156 1 99%
app/api/git_sources.py 232 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 139 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 186 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 66 0 100%
app/models.py 128 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 317 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 206 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 123 0 100%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 215 3 99%
app/rag/llm.py 243 1 99%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 88 0 100%
app/rag/retriever.py 172 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 327 0 100%
--------------------------------------------------
TOTAL 4079 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK