phase: 160_tool_budget_telemetry
All green. Final report: **Phase 160 completion pass — verified (final verification; all 5 tasks already in `complete/`)** - **Verified, no code defects found:** budget trailers (`TOOL_BUDGET_REMAINING/LAST/SPENT`), cap-path rejection + `ROUND_CAP_FALLBACK_ANSWER` + `ROUND_CAP_WRAPUP_INSTRUCTION`, `done.budget_exhausted` (ABSENT-when-false), `rounds=/cap_hit=` log fields, `agent_max_rounds=10` default, frontend Continue button + restore + shared-page exclusion, mock markers. - **Tests/lint:** `uv run pytest` 153/153 green (EXIT 0); `uv run pytest --cov=app --cov-report=term-missing` TOTAL **99%** (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors/0 warnings. - **Criteria:** ① decreasing trailer pinned by unit tests (9→…→spent) ✅ ② cap path ends readable, no tool lines after cap (E2E) ✅ ③ `budget_exhausted` true-only-on-cap, absent otherwise (integration) ✅ ④ Continue button → one fresh `continue` turn, fresh budget, record intact (E2E) ✅ ⑤ flag survives refresh; shared chat never renders it (`shared.js` has no `budget_exhausted` path) ✅ ⑥ `max_rounds=0` byte-identical single `tools=None` request; default still 10 ✅ ⑦ full suite/coverage/lint green ✅ ⑧ `test_tool_budget_telemetry.py` **7/7** in isolation; all 6 regression E2E suites green in isolation ✅ ⑨ commit + phase move = harness (I make no git history/index changes). - **Docs:** README.md — fixed: the "Continue researching" button and cap behavior were undocumented (added a Chat-section paragraph + an Agent-tools-section paragraph, incl. the phase-71 recorded-revision note); env-var table, commands, and architecture all verified consistent otherwise. AGENTS.md — verified consistent, no changes needed. - **Decisions:** no code changes required — phase was fully implemented; only README docs added. No live-infrastructure touched. - **Next pending phase:** `161_reasoning_bubble`.
This commit is contained in:
+8
-3
@@ -139,8 +139,12 @@ POST /api/chat {message, history?} (auth: require_user)
|
||||
SSE frames (`data: <json>\n\n`): `thinking` (before first delta) →
|
||||
`tool` (grounded turns, one per model call) → `retry` (pre-first-frame
|
||||
restarts) → `delta` (answer tokens) → `done` `{deflected, sources[],
|
||||
suggestions[]}` (terminal). Failure: `error` (terminal — no `done`, no
|
||||
`query_log` row; a pre-stream DB outage is a plain 503 JSON).
|
||||
suggestions[], budget_exhausted?}` (terminal). Failure: `error` (terminal — no `done`, no
|
||||
`query_log` row; a pre-stream DB outage is a plain 503 JSON). A
|
||||
`tools=None` request never emits a `tool` frame; a round-capped turn
|
||||
whose final reply carries no visible content answers with the budget
|
||||
fallback instead of erroring (phase 160, recorded revision of the
|
||||
phase-71 cap-path terminal, owner-confirmed 2026-10-05).
|
||||
`BOR_STREAM_THINKING=0` suppresses `thinking` frames server-side (chars
|
||||
still counted). LLM retries (A15 extension): retry count and delay are
|
||||
configurable (defaults: 3 retries, 5 s flat delay), **only before a
|
||||
@@ -268,7 +272,8 @@ scripts.…`, and tests all honor it.
|
||||
```
|
||||
question=… embed_ms=… top_score=… fts_hits=… summary_hits=… tuning=N
|
||||
kb_chars=N history_msgs=N threshold=… deflected=… sources=…
|
||||
thinking_chars=… tool_calls=N total_ms=… retries=N scaffold_stripped=N
|
||||
thinking_chars=… tool_calls=N rounds=N cap_hit=0|1 total_ms=…
|
||||
retries=N scaffold_stripped=N
|
||||
```
|
||||
(`sources=` = retrieval + agent-read docs, deduped; `tool_calls=`
|
||||
executed only; a cancelled turn writes neither line nor `query_log`
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
All green. Final report:
|
||||
|
||||
**Phase 160 completion pass — verified (final verification; all 5 tasks already in `complete/`)**
|
||||
|
||||
- **Verified, no code defects found:** budget trailers (`TOOL_BUDGET_REMAINING/LAST/SPENT`), cap-path rejection + `ROUND_CAP_FALLBACK_ANSWER` + `ROUND_CAP_WRAPUP_INSTRUCTION`, `done.budget_exhausted` (ABSENT-when-false), `rounds=/cap_hit=` log fields, `agent_max_rounds=10` default, frontend Continue button + restore + shared-page exclusion, mock markers.
|
||||
- **Tests/lint:** `uv run pytest` 153/153 green (EXIT 0); `uv run pytest --cov=app --cov-report=term-missing` TOTAL **99%** (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors/0 warnings.
|
||||
- **Criteria:** ① decreasing trailer pinned by unit tests (9→…→spent) ✅ ② cap path ends readable, no tool lines after cap (E2E) ✅ ③ `budget_exhausted` true-only-on-cap, absent otherwise (integration) ✅ ④ Continue button → one fresh `continue` turn, fresh budget, record intact (E2E) ✅ ⑤ flag survives refresh; shared chat never renders it (`shared.js` has no `budget_exhausted` path) ✅ ⑥ `max_rounds=0` byte-identical single `tools=None` request; default still 10 ✅ ⑦ full suite/coverage/lint green ✅ ⑧ `test_tool_budget_telemetry.py` **7/7** in isolation; all 6 regression E2E suites green in isolation ✅ ⑨ commit + phase move = harness (I make no git history/index changes).
|
||||
- **Docs:** README.md — fixed: the "Continue researching" button and cap behavior were undocumented (added a Chat-section paragraph + an Agent-tools-section paragraph, incl. the phase-71 recorded-revision note); env-var table, commands, and architecture all verified consistent otherwise. AGENTS.md — verified consistent, no changes needed.
|
||||
- **Decisions:** no code changes required — phase was fully implemented; only README docs added. No live-infrastructure touched.
|
||||
- **Next pending phase:** `161_reasoning_bubble`.
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 470 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5776 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
All gates green. Task 01 complete.
|
||||
|
||||
**Report**
|
||||
|
||||
- **Implemented (task 01 — budget telemetry on every tool result):**
|
||||
- `app/rag/agent.py`: `TOOL_BUDGET_REMAINING` / `TOOL_BUDGET_LAST` / `TOOL_BUDGET_SPENT` constants + pure `_budget_note(remaining)` helper, placed beside the teaching-string family before `CORRECTION_INSTRUCTION`; trailer appended in `run_agent` after `rounds += 1` as `result + _budget_note(max_rounds - rounds)` — `_execute_tool` stays the pure `→ str` seam
|
||||
- Module docstring loop-contract point 4 extended (trailer's three forms + "count = what's LEFT after the call it is reading"); `.env.example` `BOR_AGENT_MAX_ROUNDS` comment extended
|
||||
- Unit tests: `_budget_note` boundary table (2/3/1/0/−1), 3-round trailer decreasing 2→1→spent (pinned exact in the round-cap cluster), `agent_max_rounds=1` → spent line, kill-switch pin extended (no `role:"tool"` message at all), rejected-call refusal + note, n=9/n=8 pins
|
||||
- Kept ~130 existing tool-message assertions green: raw-result pins via `startswith`/`removesuffix(_budget_note(9))` (trailer is appended content, not a new message), cap cluster pinned exact `raw + note`
|
||||
- **Gates:** `uv run pytest` → **3665 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (gate >90%); `uv run ruff check .` clean; `uv run pyright` 0 errors; `uv run pytest tests/integration/test_chat_api.py -v --no-cov` → 59 passed (no SSE/`done`/`tool_result`/`tool_calls=` change)
|
||||
- **Docs:** README.md — both `BOR_AGENT_MAX_ROUNDS` table rows note the per-result remaining-count telemetry; AGENTS.md — no doc impact
|
||||
- **Decisions:** no new setting, no default change (LOCKED held); `agent_max_rounds=0` kill switch byte-identical; truncation `ToolResultPiece`/`read_coverage` untouched
|
||||
- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md`
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
................................................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 270 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 440 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 466 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5751 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
All gates re-verified green. The previous attempt's work is intact in the working tree; nothing was lost, and no additional code changes were needed this pass.
|
||||
|
||||
**Report**
|
||||
|
||||
- **Task 01 state (verified this pass):** `TOOL_BUDGET_REMAINING`/`TOOL_BUDGET_LAST`/`TOOL_BUDGET_SPENT` + pure `_budget_note()` in `app/rag/agent.py`; trailer appended in `run_agent` after `rounds += 1` (`result + _budget_note(max_rounds - rounds)`), `_execute_tool` untouched; docstring point 4 + `.env.example` comment updated
|
||||
- **Tests:** new unit tests (boundary table 2/3/1/0/−1, 3-round decreasing 2→1→spent pinned exact, cap-1 spent line, kill-switch no-tool-message assert, rejection + note, n=9/n=8 pins); ~130 existing tool-message assertions kept green via raw-result `startswith`/`removesuffix` pins
|
||||
- **Gates (this pass):** `uv run pytest` → **3665 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90%); `uv run ruff check .` → clean; `uv run pyright` → 0 errors; `uv run pytest tests/integration/test_chat_api.py -v --no-cov` → 59 passed (SSE/`done`/`tool_result`/`tool_calls=` unchanged); budget cluster `-k budget|round_cap|zero_max_rounds|rejected` → 9 passed
|
||||
- **Docs:** README.md both `BOR_AGENT_MAX_ROUNDS` rows note the per-result telemetry; AGENTS.md no impact (not the phase's final pass)
|
||||
- **Decisions:** no new setting, default 10 unchanged (LOCKED held); kill switch byte-identical; truncation/`read_coverage` paths untouched
|
||||
- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md`
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
................................................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 270 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 440 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 466 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5751 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
Everything is verified. The previous run had already implemented task 02's code and tests; I audited every work item against the spec, fixed one lint defect (an over-long signature line in the new integration test), and ran the full gates.
|
||||
|
||||
**Report**
|
||||
|
||||
- **Audited/verified (implemented by prior run, all spec-conformant):** `ROUND_CAP_WRAPUP_INSTRUCTION` + `ROUND_CAP_FALLBACK_ANSWER` constants; `AgentHolder.rejected_calls` (+ docstring); `run_agent` forced-final block (coached `messages_capped`, in-loop `ToolCallPiece` rejection → `rejected_calls`, fallback yield when `final_content == 0`, `MalformedReplyError` deleted on cap path only, both warnings); `chat.py::_pump` `tools_offered` (deflected + recovery `False`, grounded `True`) with `dropped_calls` local + debug line; module/run_agent docstrings updated
|
||||
- **Fixed:** `tests/integration/test_chat_api.py:1521` E501 — split `test_cap_path_rejects_late_calls_and_fallbacks` signature across lines
|
||||
- **Gates (all green):** `uv run pytest --cov=app --cov-report=term-missing` → **3672 passed, 0 failed, TOTAL 99%** (>90%); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings, 0 informations
|
||||
- **Spec pins intact:** phase-45 kill-switch single `tools=None` request, `round cap reached (rounds=N)` warning, non-cap `MalformedReplyError` (grounded + deflected twice-scaffolding tests green), trailer on executed results (task 01)
|
||||
- **Docs:** no doc impact from this task (no new env/setting/API shape; `budget_exhausted` + Continue button are tasks 03/04; task 01's README/.env.example notes already in tree)
|
||||
- **No live-infrastructure changes** (DB container was already up for tests)
|
||||
- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md`
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
........................................................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 449 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 466 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5767 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
**Task 03 complete — report**
|
||||
|
||||
**Implemented:**
|
||||
- `AgentHolder.rounds` / `AgentHolder.cap_reached` (`app/rag/agent.py`), documented field style; `holder.rounds` mirrored beside `rounds += 1` and once before the forced final; `cap_reached` set exactly in the round-cap block
|
||||
- `ChatDoneEvent.budget_exhausted: bool = False` (`app/schemas.py`) — serializer pops it when False (ABSENT, never false; byte-identical off, `answer_image` precedent)
|
||||
- `app/api/chat.py`: `budget_exhausted=holder.cap_reached` on the done frame (deflected turns correct by construction); per-turn log line gains `rounds=%d cap_hit=%d` immediately after `tool_calls=%d`, all existing fields unchanged; module docstring paragraph
|
||||
- `.agents/PLAN.md`: §4 `done {…, budget_exhausted?}` + the recorded phase-71 cap-path revision sentence; §9 `rounds=N cap_hit=0|1` after `tool_calls=N` — nothing else changed
|
||||
- Tests: 4 new `ChatDoneEvent` serialization pins (absent/true/independent/back-compat), 4 new `AgentHolder` state tests (refusals count as rounds, in-cap `cap_reached=False`, full count at cap, kill-switch defaults), 2 new integration tests (cap turn → `budget_exhausted: true` + `rounds=1 cap_hit=1`; in-cap/deflected turns → key absent + `cap_hit=0`), field-order pin updated in `test_chat_gate.py`
|
||||
|
||||
**Gates (all green):** `uv run pytest` → 3682 passed; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL 99% (schemas 100%); `uv run ruff check .` + `uv run pyright` → clean. E2E regressions in isolation (DB up): smoke, agent_unlimited_tools, agent_document_tools, harness_aligned_tools, grep_regex_teaching, chat_rag — all green.
|
||||
|
||||
**Docs:** PLAN.md §4/§9 per the task's explicit work item 5 (recorded revision, owner-confirmed 2026-10-05); no README/AGENTS.md impact (they don't document the log line or done keys).
|
||||
|
||||
**Decisions:** no migration, no new setting; `agent_max_rounds` default 10 untouched; the single `ChatDoneEvent` construction site updated (deflected turns run no agent, so absent is correct by construction).
|
||||
|
||||
**Next pending task:** `04_continue_researching_button.md` (UI: the Continue button, persisted marker, restore path, CSS).
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
.......... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 469 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5775 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
All complete and verified. Final report:
|
||||
|
||||
**Phase 160 · Task 04 — "Continue researching" button: complete, 100% green**
|
||||
|
||||
- **UI (`app.js`)**: `CONTINUE_ICON` (inline aria-hidden SVG, RETRY_ICON stroke family), `CONTINUE_QUESTION = "continue"`, `appendContinueButton` (getActionsRow pattern, one-per-bubble guard, listitem ARIA, click → `continueTurn`); `continueTurn` (in-flight no-op → clear every `.continue-btn` → plain `runTurn(CONTINUE_QUESTION)`, full history); done settle persists `budget_exhausted` (`undefined` drops the key) + clears stale buttons + flag-gated append; restore branch keys on `m.budget_exhausted`; `startNewChat` untouched (list reset drops buttons)
|
||||
- **Schema**: `ChatMessage.budget_exhausted: bool | None = None` (phase-120/48 style; no serializer/migration — `chats.py` unchanged); shared page never shows the control (no `app.js` in shared.html; `shared.js` zero interactive controls) — verified + pinned
|
||||
- **CSS**: `.continue-btn` in the `.retry-btn` family (global `:focus-visible` ring — no per-class extension needed; no animation → reduced-motion parity needs nothing) + mobile-block 44px lockstep
|
||||
- **Tests**: new `tests/unit/test_frontend_tool_budget.py` (18 pins incl. byte-unchanged `retryLastTurn`/`markLastRetryable`/`rememberBrainTurn`); `test_chats_api.py` +4 byte-identical round-trip tests (POST/GET/PUT/shared); updated golden pins (`test_schemas`, `test_chat_message_failed`, `test_chat_image_questions`, `PINNED_RUN_TURN` sanctioned addition, `test_navbar_refresh` key set)
|
||||
- **Gates**: `uv run pytest` 3704 passed · `--cov=app` TOTAL 99% (>90%) · `ruff check .` clean · `pyright` 0 errors · e2e isolated: `test_navbar_refresh` 7/7, `test_chat_rag` 3/3, `test_smoke` 3/3 · `node --check app.js` OK
|
||||
- **Docs**: no README/AGENTS.md impact (no new env/commands/config; README doesn't enumerate bubble actions; the phase-160 env-table line landed in task 01)
|
||||
- **Decisions**: (1) the done-settle append sits after `markLastEditable()` (still after `markLastRetryable()`) to keep the phase-159 paired-settle pin green — row order remains Tune → Retry → Continue → Save-as-doc; (2) added the done-settle remove-first clear to satisfy "earlier answers' buttons gone once a newer turn lands"; (3) restore shows the button on every flagged record (per spec — no restore re-targeting)
|
||||
|
||||
Next pending task: `05_budget_tests.md` (mock round-cap marker + the isolated `tests/e2e/test_tool_budget_telemetry.py`).
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 470 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5776 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
All completion criteria are met. The stray `image.png` (Oct 5, pre-dates this session) is left untouched. Final report:
|
||||
|
||||
**Task 05 — Budget tests: complete, 100% green**
|
||||
|
||||
- **Implemented**
|
||||
- `tests/e2e/mock_llm.py`: deterministic `ROUND_CAP_TRIGGER` ("run out of tool calls", last-user-message-only) flow — tools-offered requests stream `ls` calls with incrementing `call_cap_N` ids (guaranteed cap hit); the forced final (no tools) streams the dumped `read` call (`call_cap_dump`, seed path, no content); the `continue` question (trigger in an earlier user message) streams the fixed `ROUND_CAP_CONTINUE_ANSWER`; documented in the module docstring's marker table
|
||||
- `tests/e2e/test_tool_budget_telemetry.py` (new, mock-only, 7 tests, module app with `BOR_AGENT_MAX_ROUNDS=3`, TRUNCATE-then-seed kubernetes.md, autouse `saved_chats` truncate): trailer countdown via the mock's request log, zero tool lines past the cap + fallback answer + no error frame, `done.budget_exhausted` true/absent, Continue button fresh turn + record shape, double-click single-shot (exact `/api/chat` POST count), flag survives refresh, deflected-turn negative
|
||||
- Unit/integration pins (item 3) already covered by tasks 01–03's `test_agent.py` round-cap cluster (`_budget_note` boundaries, executed+refused trailers, `max_rounds=0/1`, wrapup-instruction, rejection/fallback) — no new file needed; item 4's `test_frontend_tool_budget.py` (18 pins, task 04) left as-is — nothing unpinned
|
||||
|
||||
- **Gates**: `uv run pytest` → 3704 passed · `--cov=app` → TOTAL 99% (>90%; the 2 missing lines in agent.py/chat.py are pre-existing, not phase-160) · `ruff check .` + `pyright` clean · `tests/e2e/test_tool_budget_telemetry.py` → 7/7 in isolation (two consecutive runs) · regressions isolated: agent_unlimited_tools 4, agent_document_tools 4, harness_aligned_tools 3, grep_regex_teaching 2, big_read_progress 4, read_range_tool_lines 3, chat_rag 3, tool_scaffolding_guardrails 3, smoke 3
|
||||
- **Docs**: no doc impact (test-only change; README env-table note landed in task 01)
|
||||
- **Decisions**: `ROUND_CAP_CONTINUE_ANSWER` is a fixed suite-constant naming the seed doc (the continuation turn deflects, so its prompt carries no `<documents>` block — the `SCOPED_SEARCH_PATH` convention); cap question verified ≈0.44 cosine against the mock gate
|
||||
- **Next pending task**: none — task 05 is the phase's last task (phase dir moves to `complete/` by the harness)
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 5%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 277 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 196 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 266 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 135 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 454 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 764 4 99%
|
||||
app/rag/llm.py 246 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 155 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/schemas.py 470 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5776 17 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+1
-1
@@ -66,7 +66,7 @@ BOR_RECENCY_HALF_LIFE_DAYS=365 # recency decay timescale, days (phase 106); t
|
||||
BOR_NAME_HIT_BONUS=0.005 # selection-time bonus a name-hit doc's best fused score gets (phase 119); 0 = off (pre-phase order byte-identical)
|
||||
|
||||
# --- Agent document tools (grounded turns may extend context: ls / read / grep) ---
|
||||
# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools)
|
||||
# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools); every tool result carries the model's remaining count (phase 160)
|
||||
# BOR_READ_MAX_CHARS=128000 # cap on a `read` result (chars); over-cap reads truncate + a notice that teaches the grep -> read(offset, limit) workflow (phase 95, rewritten phase 131)
|
||||
|
||||
# --- Import scope (A9 default; ANY well-formed extension is allowed) ---
|
||||
|
||||
@@ -114,6 +114,24 @@ long — trim it and re-ask" (or a mistyped question) is fixed by
|
||||
trimming it in the editor instead of retyping it. Earlier bubbles
|
||||
never carry the button (last-question-only scope, owner-locked).
|
||||
|
||||
**When a long answer spends its research budget** (phase 160): the
|
||||
answer's meta row carries a **Continue researching** button (row order
|
||||
Tune → Retry → Continue → Save-as-doc; not admin-gated — chat is
|
||||
public, so any visitor can keep a long research session going). Each
|
||||
tool result tells the model how many tool calls remain; calls
|
||||
attempted after the budget is spent are **rejected** (not executed,
|
||||
not shown) instead of being dumped into the chat, and the turn ends
|
||||
with a final no-tools reply — the model is coached to answer from the
|
||||
results it already has and to tell you it can keep going, and a
|
||||
deterministic "budget spent" fallback line stands in if that reply
|
||||
carries no visible content. Clicking the button sends one plain
|
||||
`continue` turn — the budget is per-turn, so the new turn has a whole
|
||||
new round budget, and nothing server-side knows or cares that this is
|
||||
a continuation (A10, stateless). The button is inert while a turn is
|
||||
streaming and single-shot (a stale button is removed when the new
|
||||
turn starts); the flag is persisted on the saved record, so the button
|
||||
restores on a refresh — but a shared chat never shows it.
|
||||
|
||||
### RAG / Knowledge base (`/sources.html`)
|
||||
|
||||
The indexed document list. The *Path* column opens each document in the same
|
||||
@@ -497,7 +515,18 @@ The model may call tools as many times as needed, bounded by a round cap:
|
||||
|
||||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `BOR_AGENT_MAX_ROUNDS` | `10` | hard cap on agent tool rounds per grounded turn (0 = no tools, the kill switch) |
|
||||
| `BOR_AGENT_MAX_ROUNDS` | `10` | hard cap on agent tool rounds per grounded turn (0 = no tools, the kill switch); every tool result carries the model's remaining count (phase 160) |
|
||||
|
||||
At the cap, further tool calls are **rejected** (never executed, never
|
||||
shown) and the turn ends with a final no-tools reply: the model is
|
||||
coached to answer from the results it already has and to tell the user
|
||||
they can say "continue" to keep researching; a reply with no visible
|
||||
content answers with a deterministic "budget spent" fallback line
|
||||
instead of an error frame (phase 160, the recorded revision of the
|
||||
phase-71 cap-path terminal). The `done` frame carries an additive
|
||||
`budget_exhausted` key (ABSENT on every non-cap turn) that drives the
|
||||
**Continue researching** button on the answer's meta row — see the
|
||||
[Chat (`/`)](#chat--) section.
|
||||
|
||||
---
|
||||
|
||||
@@ -898,7 +927,7 @@ to `.env` and adjust.
|
||||
|
||||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools) |
|
||||
| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools); every tool result carries the model's remaining count (phase 160) |
|
||||
| `BOR_READ_MAX_CHARS` | `128000` | Cap on a `read` tool result (chars); over-cap reads truncate + a notice that teaches the grep → `read(offset, limit)` workflow (phase 95, rewritten phase 131) |
|
||||
|
||||
### Import & sources
|
||||
|
||||
@@ -1 +1,4 @@
|
||||
# TODO
|
||||
|
||||
- [ ] Need word wrap for image explanations and other long text rather than scrolling horizontally
|
||||
- [ ] There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly. 
|
||||
+70
-4
@@ -122,6 +122,29 @@ the LOW prompt never carries tools, and with ``agent_max_rounds`` at
|
||||
**0** ``run_agent`` makes exactly one ``tools=None`` request,
|
||||
reproducing the pre-phase behavior (the kill switch).
|
||||
|
||||
No ``tool`` frame without tools (phase 160, task 02): a request made
|
||||
WITHOUT tools never produces a ``tool`` frame — a structured ``tool_calls``
|
||||
reply to a ``tools=None`` request (the documented ``lite`` behavior, the
|
||||
phase-71 incident class) is rejected at the transport: ``_pump``
|
||||
(``tools_offered=False`` on the deflected turn and its recovery) drops the
|
||||
``ToolCallPiece`` (counted in a local, named by one debug line) instead of
|
||||
rendering it as a tool line, and the grounded cap path's rejected calls are
|
||||
suppressed inside ``run_agent`` (``holder.rejected_calls``). The ``tool``
|
||||
event shape (A15) is unchanged — a frame is simply no longer emitted for a
|
||||
call that cannot run.
|
||||
|
||||
Budget-exhausted report (phase 160, task 03, TODO L4): a turn that
|
||||
hits ``BOR_AGENT_MAX_ROUNDS`` reports itself — the done frame carries
|
||||
one additive key, ``budget_exhausted: true`` (driven by
|
||||
``AgentHolder.cap_reached``; ABSENT — never ``false`` — on every other
|
||||
turn, so non-cap frames stay byte-identical to pre-phase; the UI's
|
||||
"Continue researching" affordance keys on it, task 04). The per-turn
|
||||
log line records ``rounds=N`` and ``cap_hit=0|1`` after
|
||||
``tool_calls=N`` (PLAN §9): ``N`` is the rounds CONSUMED (an executed
|
||||
call or a rejection; a cap-path rejected call is not a round) and
|
||||
``cap_hit=1`` only on a cap-reached turn — ``rounds=0 cap_hit=0`` on
|
||||
deflected turns (the agent never runs) and kill-switch turns.
|
||||
|
||||
LLM retries (phase 67, ``BOR_LLM_RETRIES`` / ``BOR_LLM_RETRY_DELAY``,
|
||||
owner-locked 2026-09-01): when the aipi endpoint dies before a request
|
||||
has streamed its first output frame (locked A2), the request is
|
||||
@@ -890,11 +913,21 @@ async def chat(
|
||||
thinking_chars = 0
|
||||
content_chars = 0 # phase 71: the turn's visible (clean) content
|
||||
scaffold_stripped = 0 # phase 71: sum across the turn's requests
|
||||
# Phase 160 (task 02): the transport half of the cap path — a
|
||||
# request made WITHOUT tools (the deflected turn, A8, and its
|
||||
# recovery) never produces a ``tool`` frame: a structured
|
||||
# ``tool_calls`` reply is rejected at ``_pump`` (dropped,
|
||||
# counted in a local, named by one debug line). The grounded
|
||||
# first pass keeps its current behavior (``run_agent``
|
||||
# suppresses the cap-path calls internally, and the kill
|
||||
# switch is byte-identical — unchanged, phase 45).
|
||||
tools_offered = not plan.deflected
|
||||
|
||||
async def _pump(
|
||||
pieces: AsyncIterator[
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
|
||||
],
|
||||
tools_offered: bool = True,
|
||||
) -> AsyncIterator[str]:
|
||||
"""One request's piece loop (phase 71 extraction): the
|
||||
thinking/tool/retry/tool_result/delta handling shared by
|
||||
@@ -903,10 +936,27 @@ async def chat(
|
||||
(pinned by the existing integration suite). Phase 95:
|
||||
the ``ToolResultPiece`` branch emits the additive
|
||||
``tool_result`` SSE frame (the seventh, optional event
|
||||
type — the A15 extension)."""
|
||||
type — the A15 extension). Phase 160 (task 02):
|
||||
``tools_offered=False`` (a request made WITHOUT tools —
|
||||
the deflected turn and its recovery) drops any
|
||||
``ToolCallPiece`` — a ``tools=None`` request cannot run a
|
||||
call, so it is rejected at the transport (no ``tool``
|
||||
frame, no visible markup) and counted in a local so one
|
||||
debug line can name it. The grounded first pass keeps its
|
||||
current behavior (``tools_offered=True``; ``run_agent``
|
||||
suppresses the cap-path calls internally)."""
|
||||
nonlocal thinking_chars, content_chars, retries_used
|
||||
dropped_calls = 0 # phase 160: tools=None requests only
|
||||
async for piece in pieces: # StreamPiece | ToolCallPiece | RetryPiece
|
||||
if isinstance(piece, ToolCallPiece):
|
||||
if not tools_offered:
|
||||
# Phase 160 (task 02): a tools=None request
|
||||
# cannot run a call — reject it (no ``tool``
|
||||
# frame, no visible markup); count it so one
|
||||
# debug line can name it. Never executed,
|
||||
# never rendered.
|
||||
dropped_calls += 1
|
||||
continue
|
||||
# Phase 37 (PLAN §4 extension; phase 70): one SSE
|
||||
# ``tool`` frame per model-requested call.
|
||||
# ``argument`` is the single string the model
|
||||
@@ -1013,9 +1063,15 @@ async def chat(
|
||||
else:
|
||||
content_chars += len(piece.text)
|
||||
yield sse_event({"type": "delta", "text": piece.text})
|
||||
if dropped_calls:
|
||||
logger.debug(
|
||||
"chat: %d tool call(s) from a tools=None request "
|
||||
"rejected, not shown (phase 160)",
|
||||
dropped_calls,
|
||||
)
|
||||
|
||||
try:
|
||||
async for frame in _pump(answer_stream):
|
||||
async for frame in _pump(answer_stream, tools_offered=tools_offered):
|
||||
yield frame
|
||||
if plan.deflected and deflected_filter is not None:
|
||||
scaffold_stripped = deflected_filter.stripped_chars
|
||||
@@ -1052,7 +1108,7 @@ async def chat(
|
||||
delay=settings.llm_retry_delay,
|
||||
scaffolding=recovery_filter,
|
||||
)
|
||||
async for frame in _pump(recovery_stream):
|
||||
async for frame in _pump(recovery_stream, tools_offered=False):
|
||||
yield frame
|
||||
scaffold_stripped += recovery_filter.stripped_chars
|
||||
if content_chars == 0:
|
||||
@@ -1197,7 +1253,8 @@ async def chat(
|
||||
"question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d "
|
||||
"suggested=%d tuning=%d "
|
||||
"kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r "
|
||||
"thinking_chars=%d tool_calls=%d total_ms=%d retries=%d scaffold_stripped=%d",
|
||||
"thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d "
|
||||
"retries=%d scaffold_stripped=%d",
|
||||
request.message,
|
||||
embed_ms,
|
||||
plan.top_score,
|
||||
@@ -1212,6 +1269,8 @@ async def chat(
|
||||
source_paths,
|
||||
thinking_chars,
|
||||
holder.tool_calls,
|
||||
holder.rounds,
|
||||
int(holder.cap_reached),
|
||||
total_ms,
|
||||
retries_used,
|
||||
scaffold_stripped,
|
||||
@@ -1269,6 +1328,13 @@ async def chat(
|
||||
related=related_refs,
|
||||
suggestions=plan.suggestions,
|
||||
answer_image=answer_ref,
|
||||
# Phase 160 (task 03): the turn reports the cap
|
||||
# hit — ``True`` only when the agent's round
|
||||
# cap forced the final no-tools answer.
|
||||
# Deflected turns run no agent, so the default
|
||||
# (``False`` → ABSENT on serialization) is
|
||||
# correct by construction.
|
||||
budget_exhausted=holder.cap_reached,
|
||||
).model_dump()
|
||||
)
|
||||
finally:
|
||||
|
||||
+200
-15
@@ -211,12 +211,34 @@ task 04):
|
||||
that keeps emitting rejected calls is bounded by the cap (point 4).
|
||||
4. Every call the model emits is appended back to the message history as
|
||||
the assistant tool-call message + the tool result (refusals included),
|
||||
consumes one round, and the model is called again. At the round cap —
|
||||
consumes one round, and the model is called again. Every tool result
|
||||
that goes on the wire carries the budget-telemetry trailer (phase 160,
|
||||
task 01) — :data:`TOOL_BUDGET_REMAINING` (≥ 2 left),
|
||||
:data:`TOOL_BUDGET_LAST` (1 left) or :data:`TOOL_BUDGET_SPENT`
|
||||
(0 left) via :func:`_budget_note` — the count is what the model has
|
||||
LEFT after the call it is reading (so the first result of a 10-round
|
||||
budget reads 9, and the boundary call's result reads the spent line).
|
||||
At the round cap —
|
||||
``max_rounds = settings.agent_max_rounds`` (``BOR_AGENT_MAX_ROUNDS``,
|
||||
default 10) — the loop forces one final retried no-tools request
|
||||
(``chat_stream_retried`` with ``tools=None``) and returns: the cap is
|
||||
the **only** forced exit (besides "the stream carried no calls"), and
|
||||
it bounds pathological rejected-call streams.
|
||||
it bounds pathological rejected-call streams. Phase 160 (task 02):
|
||||
that forced final's single system message carries
|
||||
:data:`ROUND_CAP_WRAPUP_INSTRUCTION` folded in (the phase-71
|
||||
:data:`CORRECTION_INSTRUCTION` pattern), coaching the model that no
|
||||
tool calls remain — do not emit any (they now have no effect), answer
|
||||
from the results already in the conversation, and, if incomplete,
|
||||
tell the user they can say "continue" to keep researching. A tool
|
||||
call the model attempts on that ``tools=None`` final is **rejected**
|
||||
— never executed, never yielded (no ``tool`` frame, no visible
|
||||
markup), and counted in ``holder.rejected_calls`` (``tool_calls``
|
||||
stays executed-only). If that forced final streams no visible content
|
||||
(only calls, only scaffolding, or nothing), the turn settles with the
|
||||
deterministic :data:`ROUND_CAP_FALLBACK_ANSWER` — an honest
|
||||
"budget spent" line — instead of a terminal
|
||||
``MalformedReplyError`` (the recorded revision of the phase-71
|
||||
cap-path terminal; the non-cap recovery path's error is unchanged).
|
||||
5. A rare stream that carries both content and a tool call keeps the
|
||||
content (it was already emitted) **and** still runs the tool.
|
||||
6. *holder* (an :class:`AgentHolder`) records the read documents and the
|
||||
@@ -724,6 +746,45 @@ LS_MAX_FILE_LINES = 50
|
||||
#: round (no silent argument normalization).
|
||||
NOT_A_FOLDER = "'{arg}' is not a folder — {parent} has: {subfolders}"
|
||||
|
||||
#: The tool-call budget telemetry (phase 160, task 01; TODO L4 — "the
|
||||
#: LLM needs to know how many tool calls it has left"): appended to
|
||||
#: EVERY tool result the model receives — the raw result plus the note
|
||||
#: — so the model plans its last reads and wraps up on its own instead
|
||||
#: of hitting the cap mid-research. Appended in :func:`run_agent`
|
||||
#: (after ``rounds += 1``), NEVER inside :func:`_execute_tool` (the
|
||||
#: pure seam — its raw return stays byte-identical, the phase-95
|
||||
#: ``ToolResultPiece`` reads ``holder.read_truncations`` untouched, and
|
||||
#: a rejection's refusal text carries the note too — a refusal
|
||||
#: consumes a round, so the count is exactly as true there).
|
||||
#: **The count is what the model has LEFT after the call it is
|
||||
#: reading** — the first result of a 10-round budget reads 9, the
|
||||
#: boundary call's result reads the spent line (0). The three forms
|
||||
#: (``n ≥ 2`` / ``n == 1`` / ``n ≤ 0``) live in the one pure helper
|
||||
#: :func:`_budget_note` so the wording sits in exactly one place.
|
||||
TOOL_BUDGET_REMAINING: str = "\n\n[tool calls remaining: {n}]"
|
||||
TOOL_BUDGET_LAST: str = "\n\n[1 tool call remaining — make it count, then answer]"
|
||||
TOOL_BUDGET_SPENT: str = (
|
||||
'\n\n[No tool calls remain — answer now from what you have, or '
|
||||
'tell the user to say "continue" to keep researching.]'
|
||||
)
|
||||
|
||||
|
||||
def _budget_note(remaining: int) -> str:
|
||||
"""The phase-160 budget-telemetry trailer for a tool result.
|
||||
|
||||
*remaining* is ``max_rounds - rounds`` after the call's own round
|
||||
was consumed — what the model has LEFT: ``≥ 2`` → the formatted
|
||||
:data:`TOOL_BUDGET_REMAINING` line, ``1`` → :data:`TOOL_BUDGET_LAST`,
|
||||
``≤ 0`` → :data:`TOOL_BUDGET_SPENT`. Pure (the unit tests hit it
|
||||
directly); the wording lives in the three constants above.
|
||||
"""
|
||||
if remaining >= 2:
|
||||
return TOOL_BUDGET_REMAINING.format(n=remaining)
|
||||
if remaining == 1:
|
||||
return TOOL_BUDGET_LAST
|
||||
return TOOL_BUDGET_SPENT
|
||||
|
||||
|
||||
#: The harness-owned recovery line (phase 71, task 03) — folded into the
|
||||
#: ORIGINAL single system message of the one bounded recovery request
|
||||
#: (``system_prompt + "\n" + CORRECTION_INSTRUCTION``; provider-safe,
|
||||
@@ -735,6 +796,40 @@ CORRECTION_INSTRUCTION: str = (
|
||||
"text — no tool syntax."
|
||||
)
|
||||
|
||||
#: The round-cap wrap-up coaching (phase 160, task 02) — folded into the
|
||||
#: ORIGINAL single system message of the one forced final ``tools=None``
|
||||
#: request (``system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION``, the
|
||||
#: phase-71 :data:`CORRECTION_INSTRUCTION` pattern — the user message and
|
||||
#: the tool history stay in place). In the :data:`CORRECTION_INSTRUCTION`
|
||||
#: voice it says exactly four things: no tool calls remain; do NOT emit a
|
||||
#: tool call, tool-call JSON, or any ``<|tool_call…|>``-style markup (a
|
||||
#: tool call now has no effect at all); answer from the results already in
|
||||
#: the conversation, stating plainly what is still unchecked; and, if the
|
||||
#: answer is incomplete, end by telling the user they can say "continue"
|
||||
#: to let the model keep researching.
|
||||
ROUND_CAP_WRAPUP_INSTRUCTION: str = (
|
||||
"You have no tool calls left — the research budget for this turn is "
|
||||
"spent. Do NOT emit a tool call, tool-call JSON, or any "
|
||||
"<|tool_call…|>-style markup: a tool call now has no effect at all. "
|
||||
"Answer from the results already in this conversation, stating "
|
||||
"plainly what you checked and what is still unchecked. If your "
|
||||
'answer is incomplete, end by telling the user they can say "continue"'
|
||||
" to let you keep researching."
|
||||
)
|
||||
|
||||
#: The round-cap fallback answer (phase 160, task 02) — the deterministic
|
||||
#: backstop :func:`run_agent` streams when the forced final carries NO
|
||||
#: visible content (only rejected calls, only scaffolding, or nothing).
|
||||
#: One short sentence in the brand voice: the budget ran out before an
|
||||
#: answer came together, what was gathered above still stands, say
|
||||
#: "continue" to keep digging. Free of markdown decoration (the UI renders
|
||||
#: it through the normal markdown path).
|
||||
ROUND_CAP_FALLBACK_ANSWER: str = (
|
||||
"I've run out of research budget for this turn before the answer came "
|
||||
"together, but what I gathered above still stands — say \"continue\" "
|
||||
"to let me keep digging."
|
||||
)
|
||||
|
||||
|
||||
class MalformedReplyError(LLMError):
|
||||
"""The model kept replying in raw tool-scaffolding (phase 71).
|
||||
@@ -1488,6 +1583,32 @@ class AgentHolder:
|
||||
rejected calls (unknown tool, unknown/missing arguments or document,
|
||||
already-in-context) do not count. Drives the per-turn log line's
|
||||
``tool_calls=N`` field (task 04).
|
||||
``rejected_calls`` (phase 160, task 02): how many tool calls the
|
||||
model attempted AFTER the round cap — on the forced final
|
||||
``tools=None`` request a model (the documented ``lite`` behavior) can
|
||||
still emit structured ``tool_calls``; those are REJECTED (never
|
||||
executed, never shown — no ``tool`` frame, no visible markup) and
|
||||
counted HERE only. They count in ``tool_calls`` in nothing (the
|
||||
executed-only counter stays clean), the refusal style: "counts in
|
||||
nothing". Drives the cap-path rejection warning log.
|
||||
``rounds`` (phase 160, task 03): the number of rounds CONSUMED by
|
||||
the agent loop — one per call the model emitted on a tools-
|
||||
offered round (an executed call or a rejection; both consume the
|
||||
round — a call rejected past the cap on the forced final is NOT
|
||||
a round, it counts in ``rejected_calls``). Mirrored from the
|
||||
loop's local counter (``holder.rounds = rounds`` beside
|
||||
``rounds += 1``, and once before the forced final, so a turn that
|
||||
ends at the cap reports the full count) — the API layer reads
|
||||
it, never re-derives it. Drives the per-turn log line's
|
||||
``rounds=N`` field.
|
||||
``cap_reached`` (phase 160, task 03): ``True`` exactly where the
|
||||
round cap forces the final no-tools answer (the ``run_agent``
|
||||
round-cap block); ``False`` on every turn that answered inside
|
||||
the cap (and every ``agent_max_rounds=0`` kill-switch turn — the
|
||||
loop never runs). Drives the per-turn log line's
|
||||
``cap_hit=0|1`` field and the done frame's ``budget_exhausted``
|
||||
key (the UI's Continue-researching affordance keys on it, task
|
||||
04).
|
||||
``scaffold_stripped``: how many chars of tool-scaffolding the
|
||||
turn's filters removed across the turn's requests (rounds + the
|
||||
forced final + any recovery, phase 71) — drives the per-turn log
|
||||
@@ -1538,6 +1659,9 @@ class AgentHolder:
|
||||
|
||||
read_docs: list[Document] = field(default_factory=list)
|
||||
tool_calls: int = 0
|
||||
rejected_calls: int = 0
|
||||
rounds: int = 0
|
||||
cap_reached: bool = False
|
||||
scaffold_stripped: int = 0
|
||||
read_truncations: list[tuple[str, int, int]] = field(default_factory=list)
|
||||
shown_images: list[Document] = field(default_factory=list)
|
||||
@@ -2230,6 +2354,7 @@ async def run_agent(
|
||||
with db_factory() as tool_db:
|
||||
result = _execute_tool(tool_db, call, seed_docs, holder, settings)
|
||||
rounds += 1 # every call the model emits consumes a round
|
||||
holder.rounds = rounds # phase 160 (task 03): mirror for the API layer
|
||||
logger.info(
|
||||
"agent tool=%s args=%s round=%d/%d",
|
||||
call.name,
|
||||
@@ -2253,7 +2378,20 @@ async def run_agent(
|
||||
],
|
||||
}
|
||||
)
|
||||
messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
|
||||
# Phase 160 (task 01): the budget-telemetry trailer — the model
|
||||
# sees how many tool rounds it has LEFT after this call (appended
|
||||
# here, never inside :func:`_execute_tool`, which stays the pure
|
||||
# ``→ str`` seam: the raw result, the phase-95 ToolResultPiece and
|
||||
# the read_coverage bookkeeping are untouched; a rejected call's
|
||||
# refusal carries the note too — it consumed a round, so the
|
||||
# count is exactly as true there).
|
||||
messages.append(
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": call.id,
|
||||
"content": result + _budget_note(max_rounds - rounds),
|
||||
}
|
||||
)
|
||||
# Phase 95: surface this round's truncation(s) to the API layer.
|
||||
# The piece lands AFTER the round's ``tool`` frame (the matching
|
||||
# ``ToolCallPiece`` was already streamed, above) and BEFORE the
|
||||
@@ -2273,17 +2411,41 @@ async def run_agent(
|
||||
"no-tools answer",
|
||||
rounds,
|
||||
)
|
||||
# Phase 160 (task 03): report the cap — ``cap_reached``
|
||||
# is set exactly here (the forced no-tools final is about
|
||||
# to run) and ``rounds`` is mirrored once more before it,
|
||||
# so a turn that ends at the cap reports the FULL count
|
||||
# (the API layer reads the holder; it never re-derives).
|
||||
holder.cap_reached = True
|
||||
holder.rounds = rounds
|
||||
# Phase 48: the forced final answer gets the same explicit
|
||||
# teardown as the loop rounds (consumer abandon mid-final
|
||||
# answer must still close the model's stream). Phase 67: the
|
||||
# forced call retries under the same locked-A2 rule as the
|
||||
# loop rounds. Phase 71: the forced final runs through a
|
||||
# fresh filter too — raw scaffolding can never reach the
|
||||
# user from ANY grounded request.
|
||||
# user from ANY grounded request. Phase 160 (task 02): the
|
||||
# single system message carries the wrap-up coaching
|
||||
# (``system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION``,
|
||||
# the phase-71 :data:`CORRECTION_INSTRUCTION` pattern — the
|
||||
# user message and the tool history stay in place) so the
|
||||
# model is told the budget is spent and to answer or hand
|
||||
# off; a ``tools=None`` request can still come back with
|
||||
# structured ``tool_calls`` (the documented ``lite``
|
||||
# behavior), and those are REJECTED in the loop below —
|
||||
# never executed, never yielded (no ``tool`` frame, no
|
||||
# visible markup), counted in ``holder.rejected_calls``.
|
||||
messages_capped = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION,
|
||||
},
|
||||
*messages[1:],
|
||||
]
|
||||
final_filter = ScaffoldingFilter()
|
||||
final = chat_stream_retried(
|
||||
llm,
|
||||
messages,
|
||||
messages_capped,
|
||||
tools=None,
|
||||
retries=settings.llm_retries,
|
||||
delay=settings.llm_retry_delay,
|
||||
@@ -2292,15 +2454,22 @@ async def run_agent(
|
||||
final_content = 0
|
||||
try:
|
||||
async for piece in final:
|
||||
if isinstance(piece, ToolCallPiece):
|
||||
# Phase 160 (task 02): the budget is spent — a
|
||||
# call the model attempts now has no effect.
|
||||
# Reject it: never executed, never yielded (no
|
||||
# ``tool`` frame, no visible markup), counted
|
||||
# here only (``tool_calls`` stays executed-only).
|
||||
holder.rejected_calls += 1
|
||||
continue
|
||||
if isinstance(piece, StreamPiece) and piece.kind == "content":
|
||||
final_content += len(piece.text)
|
||||
yield piece
|
||||
finally:
|
||||
await final.aclose()
|
||||
# Phase 71: the same capture log + turn total; a
|
||||
# scaffolding-only forced final (this turn used no recovery,
|
||||
# so nothing is doubled up) settles with the same terminal
|
||||
# malformed-reply error rather than a silently empty answer.
|
||||
# Phase 71: the same capture log + turn total (the per-span
|
||||
# strip warning is the capture mechanism for new registry
|
||||
# entries — unchanged).
|
||||
holder.scaffold_stripped += final_filter.stripped_chars
|
||||
for span in final_filter.stripped_spans:
|
||||
logger.warning(
|
||||
@@ -2310,13 +2479,29 @@ async def run_agent(
|
||||
rounds + 1,
|
||||
span[:200],
|
||||
)
|
||||
if final_content == 0 and final_filter.stripped_chars > 0:
|
||||
if holder.rejected_calls:
|
||||
logger.warning(
|
||||
"agent: the forced final answer was pure tool-scaffolding "
|
||||
"— settling with a malformed-reply error"
|
||||
"agent: %d tool call(s) attempted after the round cap — "
|
||||
"rejected, not shown",
|
||||
holder.rejected_calls,
|
||||
)
|
||||
raise MalformedReplyError(
|
||||
"the forced final answer was raw tool-scaffolding — no "
|
||||
"clean answer to stream"
|
||||
# Phase 160 (task 02 — recorded revision of the phase-71
|
||||
# cap-path terminal, AGENTS.md rule 3, owner-confirmed
|
||||
# 2026-10-05): a forced final with NO visible content (only
|
||||
# rejected calls, only scaffolding, or nothing) settles with
|
||||
# the deterministic budget-fallback answer — an honest
|
||||
# "budget spent" line — instead of a terminal
|
||||
# ``MalformedReplyError`` (an error frame is the wrong
|
||||
# signal for the exact symptom this TODO reports). The
|
||||
# NON-cap recovery path's ``MalformedReplyError`` is
|
||||
# unchanged.
|
||||
if final_content == 0:
|
||||
logger.warning(
|
||||
"agent: round cap hit with no visible reply "
|
||||
"(rejected_calls=%d, scaffold_stripped=%d) — "
|
||||
"answering with the budget fallback",
|
||||
holder.rejected_calls,
|
||||
holder.scaffold_stripped,
|
||||
)
|
||||
yield StreamPiece(kind="content", text=ROUND_CAP_FALLBACK_ANSWER)
|
||||
return
|
||||
|
||||
+39
-1
@@ -312,6 +312,18 @@ class ChatDoneEvent(BaseModel):
|
||||
serialization — ABSENT, never ``null`` — so those frames stay
|
||||
byte-identical to pre-phase (the byte-identical-off contract, task
|
||||
01; the :class:`SourceRef.image_url` omission precedent).
|
||||
|
||||
Phase 160 (task 03, TODO L4): ``budget_exhausted`` — the turn
|
||||
reports that it hit ``BOR_AGENT_MAX_ROUNDS`` (the agent's round
|
||||
cap forced the final no-tools answer; driven by
|
||||
``AgentHolder.cap_reached``). ``True`` ONLY on a cap-reached turn;
|
||||
the UI's "Continue researching" affordance keys on it (task 04).
|
||||
``False`` — every deflected turn (they run no agent), every
|
||||
in-cap grounded turn, and the whole pre-phase — is DROPPED on
|
||||
serialization — ABSENT, never ``false`` — so those frames stay
|
||||
byte-identical to pre-phase (the byte-identical-off contract,
|
||||
PLAN §13; the :attr:`answer_image` / :class:`SourceRef.image_url`
|
||||
omission precedent).
|
||||
"""
|
||||
|
||||
type: str = "done"
|
||||
@@ -323,6 +335,10 @@ class ChatDoneEvent(BaseModel):
|
||||
#: :class:`SourceRef` (no list — LOCKED A3: one image per answer);
|
||||
#: the ref's own phase-83 field caps bind.
|
||||
answer_image: SourceRef | None = None
|
||||
#: Phase 160 (task 03, TODO L4) — see the class docstring. True
|
||||
#: ONLY on the cap-reached turn; ``False`` (every other turn)
|
||||
#: serializes WITHOUT the key — ABSENT, never ``false``.
|
||||
budget_exhausted: bool = False
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
@@ -333,10 +349,15 @@ class ChatDoneEvent(BaseModel):
|
||||
:class:`SourceRef.image_url` / phase-123 ``image`` omission
|
||||
precedents). A shown ref keeps all its fields (its own
|
||||
serializer drops a ``None`` ``image_url`` — a shown doc is an
|
||||
image doc, so in practice it always carries it)."""
|
||||
image doc, so in practice it always carries it). Phase 160
|
||||
(task 03): ``budget_exhausted`` ``False`` (every non-cap turn)
|
||||
serializes WITHOUT the key too — ABSENT, never ``false`` — the
|
||||
same byte-identical-off contract."""
|
||||
data = handler(self)
|
||||
if self.answer_image is None:
|
||||
data.pop("answer_image", None)
|
||||
if not self.budget_exhausted:
|
||||
data.pop("budget_exhausted", None)
|
||||
return data
|
||||
|
||||
|
||||
@@ -1325,6 +1346,23 @@ class ChatMessage(BaseModel):
|
||||
# serializer below), so pre-phase and image-less chats round-trip
|
||||
# byte-identically (the phase-50 contract).
|
||||
answer_image: SourceRef | None = None
|
||||
# Phase 160 (task 04, TODO L4): the BUDGET-EXHAUSTED marker — the
|
||||
# persisted answer that ran out of tool calls (a turn that hit
|
||||
# ``BOR_AGENT_MAX_ROUNDS``, task 03's additive ``done`` key). The
|
||||
# phase-120 ``failed`` / phase-48 ``stopped`` style: a marker on
|
||||
# the brain record itself, no separate error table, no new API.
|
||||
# It is the CLIENT'S RESTORE HINT, never a server behavior: the
|
||||
# done-time auto-save (``rememberBrainTurn``) persists the done
|
||||
# frame's ``budget_exhausted`` into this key, and the restore
|
||||
# path re-renders the "Continue researching" button from it (the
|
||||
# ``answer_image`` precedent — the field must exist so the
|
||||
# ``extra="forbid"`` boundary does NOT 422 those saves; no
|
||||
# migration: ``saved_chats.messages`` is raw JSONB, phase 50, and
|
||||
# ``app/api/chats.py``'s logic is unchanged). ``None`` (every
|
||||
# non-cap turn, every pre-phase record) flows as absent/null
|
||||
# exactly like ``stopped`` — the phase-50 byte-identical
|
||||
# round-trip contract covers the new key automatically.
|
||||
budget_exhausted: bool | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _drop_image_when_absent(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
|
||||
@@ -618,6 +618,15 @@ let lastUserWrap = null;
|
||||
const RETRY_ICON =
|
||||
'<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12a9 9 0 1 1-9-9c2.52 0 4.93 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/></svg>';
|
||||
|
||||
/* Phase 160 (task 04, TODO L4): the "Continue researching" button's
|
||||
* forward glyph (aria-hidden decoration — the visible "Continue
|
||||
* researching" label carries the accessible name). Same stroke family
|
||||
* as RETRY_ICON: fill="none", currentColor stroke (the CSS themes it
|
||||
* — ink-soft → ink on hover, the phase-08 palette), 24×24 viewBox,
|
||||
* stroke-width 1.7. No icon font, no CDN (A11). */
|
||||
const CONTINUE_ICON =
|
||||
'<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="m5 6 6 6-6 6"/><path d="m13 6 6 6-6 6"/></svg>';
|
||||
|
||||
/* Phase 159 (task 01, TODO.md L3): the Edit button's pencil glyph
|
||||
* (aria-hidden decoration — the visible "Edit" label carries the
|
||||
* accessible name). Same stroke family as RETRY_ICON: fill="none",
|
||||
@@ -648,6 +657,34 @@ function appendRetryButton(wrap) {
|
||||
meta.appendChild(btn);
|
||||
}
|
||||
|
||||
/* Phase 160 (task 04, TODO L4): the "Continue researching" button —
|
||||
* the one-click continuation of a turn that spent its whole tool
|
||||
* budget (the `done` frame's additive `budget_exhausted` key, task
|
||||
* 03 — ABSENT on every non-cap turn, so the button never appears
|
||||
* elsewhere). The house pattern through getActionsRow (phase 130
|
||||
* task 02 — the role=list row joins the button as a listitem so ARIA
|
||||
* stays valid); one button per bubble. NOT admin-gated, like Retry —
|
||||
* chat is public, so every visitor can keep a long research session
|
||||
* going. On a capped answer the row order is Tune → Retry → Continue
|
||||
* → Save-as-doc (the insertion order phase 130 relies on). Click →
|
||||
* continueTurn() — a plain next turn: the budget is per-turn, so the
|
||||
* new turn has a whole new round budget, and nothing server-side
|
||||
* knows or cares that this is a continuation (A10). */
|
||||
const CONTINUE_QUESTION = "continue";
|
||||
|
||||
function appendContinueButton(wrap) {
|
||||
const meta = getActionsRow(wrap);
|
||||
if (!meta) return;
|
||||
if (meta.querySelector(".continue-btn")) return; // one per bubble
|
||||
const btn = document.createElement("button");
|
||||
btn.type = "button";
|
||||
btn.className = "continue-btn";
|
||||
if (meta.getAttribute("role") === "list") btn.setAttribute("role", "listitem");
|
||||
btn.innerHTML = CONTINUE_ICON + "<span>Continue researching</span>";
|
||||
btn.addEventListener("click", () => continueTurn());
|
||||
meta.appendChild(btn);
|
||||
}
|
||||
|
||||
/* "Edit" button in the last USER bubble's actions row (phase 159,
|
||||
* task 01, TODO.md L3 — the trim-and-re-ask remedy: a "Question too
|
||||
* long — trim it and re-ask" turn, or any mistyped / half-formed
|
||||
@@ -2688,6 +2725,14 @@ function renderStoredMessage(m) {
|
||||
// precedes the failed record, unchanged; a mid-conversation failed
|
||||
// bubble gets no button — the phase-49 last-bubble-only rule).
|
||||
if (m.failed && m.error) appendFailedNote(wrap, m.error);
|
||||
// Phase 160 (task 04, TODO L4): a cap-reached answer keeps its
|
||||
// Continue affordance on a page refresh — a restored `history` is
|
||||
// exactly what the continuation turn needs (runTurn's plain send
|
||||
// carries conversation.slice(0, -1)). The marker is a client
|
||||
// restore hint only (the schema's budget_exhausted key, never a
|
||||
// server behavior), so pre-phase / non-cap records restore
|
||||
// byte-identically (no key → no button).
|
||||
if (m.budget_exhausted) appendContinueButton(wrap);
|
||||
lastBrainWrap = wrap; // phase 49: the LAST restored brain bubble wins
|
||||
}
|
||||
|
||||
@@ -3304,6 +3349,30 @@ function retryLastTurn(wrap) {
|
||||
return runTurn(text, { reask: true });
|
||||
}
|
||||
|
||||
/* Phase 160 (task 04, TODO L4): the "Continue researching" click —
|
||||
* the retryLastTurn discipline, one turn at a time. No-op while a
|
||||
* turn is in flight (thinking / streaming — the runTurn guard; the
|
||||
* button is never double-fired into two turns), then clear EVERY
|
||||
* rendered .continue-btn FIRST (the markLastRetryable remove-first
|
||||
* pattern: the moment the new turn's answer lands, a stale button on
|
||||
* any earlier bubble is gone by construction; the one-per-bubble
|
||||
* guard + the clear make a double-send impossible), then a PLAIN send
|
||||
* (runTurn's `reask` defaults to false): the "continue" user bubble
|
||||
* appears, and the record is pushed + saved through the ordinary save
|
||||
* points (conversation.push + saveConversation + persistConversation).
|
||||
* The request carries the FULL history (conversation.slice(0, -1)) —
|
||||
* the model resumes with everything it already read and gets a fresh
|
||||
* agent_max_rounds. ASSUMPTION: the continuation question is the
|
||||
* literal word "continue" — what the model is taught to ask the user
|
||||
* for (ROUND_CAP_WRAPUP_INSTRUCTION / the tool-budget-spent trailer,
|
||||
* phase 160 tasks 01–02), so the visible bubble, the typed
|
||||
* equivalent, and the model's own suggestion all agree. */
|
||||
function continueTurn() {
|
||||
if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;
|
||||
messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());
|
||||
return runTurn(CONTINUE_QUESTION);
|
||||
}
|
||||
|
||||
/* Phase 123 (task 02, TODO L6): the send flow's UPLOAD STEP (locked
|
||||
* A8) — the attached File goes to POST /api/chat-images as multipart
|
||||
* (the session cookie rides the browser; the endpoint is user-gated
|
||||
@@ -3933,6 +4002,12 @@ async function runTurn(text, { reask = false, image = null } = {}) {
|
||||
// row exactly as it looked live.
|
||||
related: ev.related?.length ? ev.related : undefined,
|
||||
suggestions: ev.suggestions,
|
||||
// Phase 160 (task 04, TODO L4): the cap-reached marker
|
||||
// persists with the turn — `undefined` drops the key from
|
||||
// the JSON (the house optional-meta pattern, the
|
||||
// answer_image line above is the precedent), so every
|
||||
// non-cap record stays byte-identical to pre-phase.
|
||||
budget_exhausted: ev.budget_exhausted || undefined,
|
||||
}, leavePartialIndex);
|
||||
lastBrainWrap = wrap; // this bubble is now the last brain answer
|
||||
// Phase 130 (task 02): the row order is the DOM order — the
|
||||
@@ -3943,6 +4018,20 @@ async function runTurn(text, { reask = false, image = null } = {}) {
|
||||
appendTuneButton(wrap); // every completed brain bubble is tunable
|
||||
markLastRetryable(); // phase 49: the Retry button is last-bubble-only
|
||||
markLastEditable(); // phase 159: the done turn's question is editable (the in-flight guard no-ops here — the runTurn idle settle lands it)
|
||||
// Phase 160 (task 04, TODO L4): a cap-reached answer offers
|
||||
// the one-click continuation — first, every earlier bubble's
|
||||
// continue button is cleared (the markLastRetryable
|
||||
// remove-first pattern: the moment a newer turn lands, an old
|
||||
// bubble's stale affordance is gone — its "continue" would
|
||||
// append to a history the conversation already moved past),
|
||||
// then the button is appended ONLY when the done frame is
|
||||
// flagged (ABSENT on every other turn, so the button never
|
||||
// appears elsewhere); the row order stays Tune → Retry →
|
||||
// Continue → Save-as-doc (the insertion order phase 130
|
||||
// relies on — markLastEditable above acts on the user
|
||||
// bubble, so the brain bubble's row order is unchanged).
|
||||
messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());
|
||||
if (ev.budget_exhausted) appendContinueButton(wrap);
|
||||
// Phase 59: the RAW persisted markdown (never the rendered
|
||||
// HTML) — exactly the string rememberBrainTurn stores above,
|
||||
// so a reload (the restore path) offers the identical draft.
|
||||
|
||||
@@ -1027,6 +1027,39 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
|
||||
.retry-btn svg { width: 14px; height: 14px; display: block; }
|
||||
.retry-btn:hover { background: var(--brand-soft); color: var(--ink); }
|
||||
|
||||
/* Phase 160 (task 04, TODO L4): the "Continue researching" button —
|
||||
the one-click continuation in the actions row
|
||||
(.msg-meta.msg-actions — phase 130) of a cap-reached brain bubble
|
||||
(the done frame's budget_exhausted, task 03). The exact visual
|
||||
family of .retry-btn (same pill size/spacing, the global
|
||||
:focus-visible ring, >=44px via min-height) so the meta actions
|
||||
read as a set — but neutral: ink-soft -> ink on hover (Tune keeps
|
||||
the brand pair). The forward glyph rides currentColor. No
|
||||
per-button auto margin (phase 130 — the row's justify right-aligns
|
||||
the cluster). No animation of its own, so the
|
||||
prefers-reduced-motion blocks need nothing for it (the family's
|
||||
parity — .retry-btn / .edit-btn have none either). Contrast: ink-soft on --bg ~8.6:1, ink
|
||||
~15:1, hover ink on --brand-soft ~15.7:1 — all AA. */
|
||||
.continue-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 0.35rem;
|
||||
min-height: 44px;
|
||||
padding: 0.35rem 0.8rem;
|
||||
border-radius: 999px;
|
||||
border: 1px solid var(--line);
|
||||
background: transparent;
|
||||
color: var(--ink-soft);
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
font-size: 0.82rem;
|
||||
white-space: nowrap;
|
||||
cursor: pointer;
|
||||
}
|
||||
.continue-btn svg { width: 14px; height: 14px; display: block; }
|
||||
.continue-btn:hover { background: var(--brand-soft); color: var(--ink); }
|
||||
|
||||
/* Phase 159 (task 01, TODO L3): the "Edit" button — the
|
||||
* trim-and-re-ask action in the .msg-meta.msg-actions row (phase 130)
|
||||
* of the NEWEST user bubble (markLastEditable — the phase-49
|
||||
@@ -5817,6 +5850,7 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
|
||||
.tune-btn { min-height: 44px; }
|
||||
.retry-btn { min-height: 44px; }
|
||||
.edit-btn { min-height: 44px; }
|
||||
.continue-btn { min-height: 44px; }
|
||||
/* Phase 27: the tuning page squeezes like the other cards — the row
|
||||
padding tightens; the icon+label actions keep their 44px floor and
|
||||
the note text ellipsizes (min-width: 0). */
|
||||
|
||||
@@ -842,6 +842,57 @@ Implements just enough of the aipi surface:
|
||||
deflected turns never hit it); verified 2026-10-01: no
|
||||
existing E2E question or fixture file contains the phrase, so
|
||||
every other suite is unaffected.
|
||||
- user message containing ``run out of tool calls``
|
||||
(``ROUND_CAP_TRIGGER``, phase 160 task 05 — the
|
||||
tool-call-budget telemetry's dedicated suite
|
||||
``tests/e2e/test_tool_budget_telemetry.py``) — detected on
|
||||
the LAST user message only (the mock's existing invariant:
|
||||
the last ``user`` message is the current question, the
|
||||
earlier non-system messages are prior turns):
|
||||
* while ``tools`` are offered (the request carries
|
||||
``tools`` — the app issues such a request only while the
|
||||
round budget is unspent, i.e. no tool result answers the
|
||||
last emitted call yet) and the flow is active: stream
|
||||
ONLY ``tool_calls`` deltas — ``ls`` with no arguments,
|
||||
incrementing synthetic ids ``call_cap_1``,
|
||||
``call_cap_2``, … (the next id is derived from the
|
||||
``call_cap_N`` ids already in the messages' ``assistant``
|
||||
tool calls), ``finish_reason: "tool_calls"``, no content —
|
||||
the model NEVER stops asking, so the turn is guaranteed
|
||||
to hit ``BOR_AGENT_MAX_ROUNDS``;
|
||||
* on the request WITHOUT ``tools`` (the forced final — the
|
||||
round-cap path; the request still carries the full
|
||||
history, so the trigger stays the last user message):
|
||||
stream a ``tool_calls`` delta ANYWAY — ``read`` on the
|
||||
SEED path (the system prompt's ``<documents>`` section's
|
||||
FIRST ``<document>`` block, the joined ``source/path`` —
|
||||
the ``read`` tool's combined identity, phase 70), id
|
||||
``call_cap_dump``, NO content — the EXACT dumped-call
|
||||
shape the phase kills (and the shape ``SCAFFOLD_TRIGGER``
|
||||
does not cover — that one emits textual markup, this one
|
||||
is structured). The app REJECTS it (never executed, never
|
||||
shown, counted in ``AgentHolder.rejected_calls``);
|
||||
* the ``continue`` question (the last user message is
|
||||
EXACTLY ``continue`` — the UI's ``CONTINUE_QUESTION`` —
|
||||
and an EARLIER user message carries the trigger; the
|
||||
client history rides in the stateless A10 request): the
|
||||
stable ``ROUND_CAP_CONTINUE_ANSWER`` (a fixed sentence
|
||||
naming the flow's seed document — the
|
||||
``SCOPED_SEARCH_PATH`` suite-constant convention: the
|
||||
literal "continue" shares no tokens with the KB, so the
|
||||
continuation turn deflects and its prompt carries no
|
||||
``<documents>`` block; the sentence names the document
|
||||
directly), so a suite asserts a real continuation answer
|
||||
rather than a tool call.
|
||||
Streaming only (the app's chat path is the only streaming
|
||||
consumer; a non-streaming marker request never issues and
|
||||
falls through to the regular answer, like the other marker
|
||||
flows). Checked BEFORE the plain ``TOOLS_TRIGGER`` flow
|
||||
(disjoint trigger phrases — the phase-71/72/94/118/131/
|
||||
139/141/145 ordering convention); verified 2026-10-05: no
|
||||
existing E2E question or fixture file contains the phrase
|
||||
(or sends a ``continue`` question with the trigger in the
|
||||
history), so every other suite is unaffected.
|
||||
|
||||
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
|
||||
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
|
||||
@@ -1193,6 +1244,42 @@ HISTORY_TRIGGER = "echo my history"
|
||||
#: phrase, so every other suite is unaffected.
|
||||
FOLDER_MAP_TRIGGER = "repeat your folder map"
|
||||
|
||||
#: Phase 160 (task 05, tool-call budget telemetry, TODO.md L4 —
|
||||
#: the dedicated suite
|
||||
#: ``tests/e2e/test_tool_budget_telemetry.py``): a user message
|
||||
#: containing this substring (case-insensitive), detected on the
|
||||
#: LAST user message only (the mock's existing invariant — the
|
||||
#: last ``user`` message is the current question, the earlier
|
||||
#: non-system messages are prior turns), drives the deterministic
|
||||
#: ROUND-CAP flow: while ``tools`` are offered the mock NEVER
|
||||
#: answers (``ls`` calls, incrementing ``call_cap_N`` ids — the
|
||||
#: turn is guaranteed to hit ``BOR_AGENT_MAX_ROUNDS``), and the
|
||||
#: forced final (no ``tools``) streams a structured ``read`` call
|
||||
#: anyway (id ``call_cap_dump``, the seed path, NO content) — the
|
||||
#: exact dumped-call shape the phase kills. The ``continue``
|
||||
#: question (exactly ``continue``, the trigger in an EARLIER user
|
||||
#: message) gets :data:`ROUND_CAP_CONTINUE_ANSWER`. Disjoint from
|
||||
#: every other trigger phrase; verified 2026-10-05: no existing
|
||||
#: E2E question or fixture file contains the phrase, so every
|
||||
#: other suite is unaffected.
|
||||
ROUND_CAP_TRIGGER = "run out of tool calls"
|
||||
|
||||
#: Phase 160 (task 05): the ROUND-CAP flow's CONTINUATION answer
|
||||
#: — the fixed sentence the ``continue`` question gets. The
|
||||
#: literal "continue" shares no tokens with the KB, so the
|
||||
#: continuation turn DEFLECTS (A8 LOW) and its prompt carries no
|
||||
#: ``<documents>`` block — the sentence therefore names the
|
||||
#: flow's seed document DIRECTLY (the
|
||||
#: :data:`SCOPED_SEARCH_PATH` suite-constant convention; the
|
||||
#: suite seeds exactly that document, so the name IS the
|
||||
#: seed's). Byte-stable on purpose: the suite asserts on it, so
|
||||
#: it must not drift.
|
||||
ROUND_CAP_CONTINUE_ANSWER = (
|
||||
"Picking up where the tool budget ran out — still digging into "
|
||||
"docs/homelab/kubernetes.md: it says the cluster runs Talos "
|
||||
"Linux on three nodes, and Cilium handles networking."
|
||||
)
|
||||
|
||||
#: Phase 122 (image documents, LOCKED A3): the fixed description the
|
||||
#: mock's vision (CHAT) model returns for an ``IMAGE_DESCRIPTION_MODE``
|
||||
#: request — the multimodal user message
|
||||
@@ -3079,6 +3166,106 @@ def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
return ("call", target, "call_0")
|
||||
|
||||
|
||||
#: Phase 160 (task 05): the round-cap flow's incrementing call ids
|
||||
#: — ``call_cap_1``, ``call_cap_2``, … The NEXT id is derived from
|
||||
#: the ids already in the messages' ``assistant`` tool calls (the
|
||||
#: flow is stateless over the message history — a retry that
|
||||
#: re-issues the same request re-derives the same next id).
|
||||
_CAP_CALL_ID_RE = re.compile(r"^call_cap_(\d+)$")
|
||||
|
||||
|
||||
def _round_cap_next_call_id(body: dict[str, Any]) -> str:
|
||||
"""The next incrementing ``call_cap_N`` id (stateless).
|
||||
|
||||
The mock's ``ls`` calls in the round-cap flow carry
|
||||
incrementing synthetic ids — the NEXT id is one past the
|
||||
highest ``call_cap_N`` id already in the messages'
|
||||
``assistant`` tool calls (``call_cap_1`` on the first
|
||||
request — no ids yet).
|
||||
"""
|
||||
n = 0
|
||||
for m in _messages(body):
|
||||
if m.get("role") != "assistant":
|
||||
continue
|
||||
for call in m.get("tool_calls") or []:
|
||||
if not isinstance(call, dict):
|
||||
continue
|
||||
match = _CAP_CALL_ID_RE.match(str(call.get("id") or ""))
|
||||
if match:
|
||||
n = max(n, int(match.group(1)))
|
||||
return f"call_cap_{n + 1}"
|
||||
|
||||
|
||||
def _round_cap_seed_path(body: dict[str, Any]) -> str:
|
||||
"""The seed document's combined ``source/path`` — or the
|
||||
sentinel (the premise broke: no ``<documents>`` block; the
|
||||
suite fails loudly on the sentinel in the dump call).
|
||||
|
||||
Phase 160 (task 05): the FORCED FINAL's request (the
|
||||
round-cap path) still carries the grounded system prompt —
|
||||
the ``<documents>`` section's FIRST ``<document>`` block is
|
||||
the seed document (the suite seeds exactly one). The mock
|
||||
joins the labeled ``source`` / ``path`` fields (the
|
||||
``read`` tool's combined identity, phase 70).
|
||||
"""
|
||||
block = _DOCUMENT_BLOCK_RE.search(_system(body))
|
||||
if not block:
|
||||
return "seed/unknown.md"
|
||||
return f"{block.group('source')}/{block.group('path')}"
|
||||
|
||||
|
||||
def _round_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
"""Phase 160 (task 05): the ROUND-CAP marker flow — or ``None``.
|
||||
|
||||
Discriminated on the LAST user message (the mock's existing
|
||||
invariant: the last ``user`` message is the current
|
||||
question, the earlier non-system messages are prior turns —
|
||||
the ``_history_echo`` ``last_user`` scan convention):
|
||||
|
||||
* the last user message carries :data:`ROUND_CAP_TRIGGER`:
|
||||
- the request offers ``tools`` (the app issues such a
|
||||
request only while the round budget is unspent — no
|
||||
tool result answers the last emitted call yet):
|
||||
``("call", call_cap_N)`` — the model NEVER stops
|
||||
asking (``ls``, no arguments, the next incrementing
|
||||
id) — the turn is guaranteed to hit
|
||||
``BOR_AGENT_MAX_ROUNDS``;
|
||||
- the request carries NO ``tools`` (the forced final —
|
||||
the round-cap path; the trigger is still the last user
|
||||
message): ``("dump", <seed path>)`` — a ``read`` on
|
||||
the seed path (id ``call_cap_dump``, NO content) — the
|
||||
exact dumped-call shape the phase kills;
|
||||
* the last user message is EXACTLY ``continue`` (the UI's
|
||||
``CONTINUE_QUESTION``) and an EARLIER user message
|
||||
carries the trigger (the client history rides in the
|
||||
stateless A10 request): ``("continue",)`` — the stable
|
||||
:data:`ROUND_CAP_CONTINUE_ANSWER` (a fixed sentence
|
||||
naming the flow's seed document — the
|
||||
:data:`SCOPED_SEARCH_PATH` suite-constant convention: the
|
||||
literal "continue" shares no tokens with the KB, so the
|
||||
continuation turn deflects and its prompt carries no
|
||||
``<documents>`` block).
|
||||
"""
|
||||
users = [
|
||||
_content_text(m.get("content"))
|
||||
for m in _messages(body)
|
||||
if m.get("role") == "user"
|
||||
]
|
||||
if not users:
|
||||
return None
|
||||
last_user = users[-1].strip().lower()
|
||||
if ROUND_CAP_TRIGGER in last_user:
|
||||
if body.get("tools"):
|
||||
return ("call", _round_cap_next_call_id(body))
|
||||
return ("dump", _round_cap_seed_path(body))
|
||||
if (
|
||||
last_user == "continue"
|
||||
and any(ROUND_CAP_TRIGGER in u.lower() for u in users[:-1])
|
||||
):
|
||||
return ("continue",)
|
||||
return None
|
||||
|
||||
|
||||
def long_answer() -> str:
|
||||
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
|
||||
a unique final line that must survive the stream untruncated."""
|
||||
@@ -4563,6 +4750,42 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 160 (task 05, tool-call budget telemetry,
|
||||
# TODO.md L4): the deterministic ROUND-CAP marker flow
|
||||
# (the budget-telemetry suite's dedicated flow — see the
|
||||
# module docstring). Checked BEFORE the plain
|
||||
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
|
||||
# phase-71/72/94/118/131/139/141/145 ordering
|
||||
# convention).
|
||||
round_cap = _round_cap_flow(body)
|
||||
if round_cap is not None:
|
||||
if round_cap[0] == "call":
|
||||
# The model never stops asking: ``ls`` with no
|
||||
# arguments, the next incrementing id, NO content
|
||||
# — the turn is guaranteed to hit
|
||||
# ``BOR_AGENT_MAX_ROUNDS``.
|
||||
stream = _tool_call_stream("ls", {}, round_cap[1])
|
||||
elif round_cap[0] == "dump":
|
||||
# The forced final (no ``tools``) — the model
|
||||
# dumps a structured ``read`` call anyway (the
|
||||
# seed path, id ``call_cap_dump``, NO content) —
|
||||
# the exact shape the phase kills (the app
|
||||
# REJECTS it — never executed, never shown).
|
||||
stream = _tool_call_stream(
|
||||
"read", {"path": round_cap[1]}, "call_cap_dump"
|
||||
)
|
||||
else: # "continue" — the stable continuation answer
|
||||
stream = _sse_stream(
|
||||
_apply_max_tokens(
|
||||
ROUND_CAP_CONTINUE_ANSWER, body.get("max_tokens")
|
||||
),
|
||||
0.0,
|
||||
)
|
||||
return StreamingResponse(
|
||||
stream,
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 124 (task 07, image chips and answers, TODO L3): the
|
||||
# scripted SHOW_IMAGE flow — the mock shows the seeded image
|
||||
# document (the ``<documents>`` block with the image-
|
||||
|
||||
@@ -628,7 +628,8 @@ def test_api_created_chats_carry_the_bor_chat_v1_shape(
|
||||
# brain records (accepted by the ChatMessage schema — its earlier
|
||||
# absence 422'd the done-time auto-save, phase-118 verification fix).
|
||||
allowed = {"who", "text", "sources", "related", "deflected", "suggestions",
|
||||
"thinking", "tools", "stopped", "failed", "error"}
|
||||
"thinking", "tools", "stopped", "failed", "error",
|
||||
"budget_exhausted"}
|
||||
assert all({"who", "text"} <= set(m) <= allowed for m in body["messages"])
|
||||
# The History row renders it (the Open link's text IS the title).
|
||||
page.click("#nav-history")
|
||||
|
||||
@@ -0,0 +1,816 @@
|
||||
"""Phase 160 E2E (Playwright, mock-only): the agent knows its
|
||||
tool-call budget (and never dumps tool calls into the answer).
|
||||
|
||||
Source: ``TODO.md`` L4 — "The LLM need to know how many tool calls
|
||||
it has left. During long research sessions it will sometimes end the
|
||||
conversation with tool calls that just get dumped into chat because
|
||||
it ran out of turns. …"
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_tool_budget_telemetry.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the contract
|
||||
is the deterministic ROUND-CAP marker flow in ``tests/e2e/mock_llm.py``
|
||||
(``ROUND_CAP_TRIGGER`` = ``run out of tool calls``, detected on the
|
||||
LAST user message only — the mock's existing invariant: the last
|
||||
``user`` message is the current question, the earlier non-system
|
||||
messages are prior turns):
|
||||
|
||||
* while ``tools`` are offered (the app issues such a request only
|
||||
while the round budget is unspent — i.e. no tool result answers
|
||||
the last emitted call yet) the mock NEVER answers — it streams
|
||||
ONLY ``tool_calls`` deltas calling ``ls`` with no arguments,
|
||||
incrementing ids ``call_cap_1``, ``call_cap_2``, … (the next id
|
||||
is derived from the ids already in the messages' ``assistant``
|
||||
tool calls), ``finish_reason: "tool_calls"``, no content — the
|
||||
turn is guaranteed to hit ``BOR_AGENT_MAX_ROUNDS``;
|
||||
* on the request WITHOUT ``tools`` (the forced final) the mock
|
||||
streams a ``tool_calls`` delta ANYWAY — ``read`` on the seed
|
||||
path (id ``call_cap_dump``, NO content) — the exact dumped-call
|
||||
shape the phase kills (the app REJECTS it: never executed, never
|
||||
shown, counted in ``AgentHolder.rejected_calls``);
|
||||
* the ``continue`` question (exactly ``continue`` — the UI's
|
||||
``CONTINUE_QUESTION`` — with the trigger in an EARLIER user
|
||||
message; the client history rides in the stateless A10 request)
|
||||
gets the stable ``ROUND_CAP_CONTINUE_ANSWER`` (a fixed sentence
|
||||
naming the seed document — the ``SCOPED_SEARCH_PATH``
|
||||
suite-constant convention: the literal "continue" shares no tokens
|
||||
with the KB, so the continuation turn deflects and its prompt
|
||||
carries no ``<documents>`` block; the sentence names the document
|
||||
directly).
|
||||
|
||||
The e2e app server boots with ``BOR_AGENT_MAX_ROUNDS=3`` (the
|
||||
module-scoped ``app_server`` override — the conftest env shape with
|
||||
one swap): the cap flow burns exactly three executed rounds, and
|
||||
the tool results carry ``[tool calls remaining: 2]``, ``[1 tool
|
||||
call remaining — make it count, then answer]`` and the spent line
|
||||
(``[No tool calls remain — answer now from what you have, or tell
|
||||
the user to say "continue" to keep researching.]`` — the
|
||||
``app.rag.agent`` constants, imported so the pin cannot drift).
|
||||
|
||||
KB fixture (the ``test_failed_turn_retry.py`` TRUNCATE-then-seed
|
||||
pattern): ONE fixture document (``homelab/kubernetes.md``) — the
|
||||
cap question's tokens clear the mock-calibrated gate (≈0.44 against
|
||||
the mock's token-overlap embeddings, past the 0.30 threshold —
|
||||
HIGH without any FTS help; the E2E app pins the mock-calibrated
|
||||
values in conftest).
|
||||
|
||||
Test → contract mapping (``00_phase.md`` §Completion criteria):
|
||||
|
||||
1. ``test_tool_results_carry_the_remaining_count`` — server-side
|
||||
pin at the wire level: the mock's recorded request log (the
|
||||
``/v1/e2e/captured`` pair — reset per turn) shows each
|
||||
successive ``tool``-role message ending with the decreasing
|
||||
trailer (``[tool calls remaining: 2]``, the last-call line, the
|
||||
spent line), the incrementing ``call_cap_N`` ids in the
|
||||
``assistant`` messages, and the final request carrying the spent
|
||||
tool result + the coached system message (no ``tools`` key);
|
||||
2. ``test_capped_turn_never_shows_tool_lines_after_the_cap`` —
|
||||
with ``BOR_AGENT_MAX_ROUNDS=3``: the chat shows EXACTLY three
|
||||
tool lines (one per executed round), the dump call produces NO
|
||||
tool line (no ``Reading <seed path>`` line anywhere), the answer
|
||||
bubble holds the non-empty ``ROUND_CAP_FALLBACK_ANSWER`` (the
|
||||
deterministic backstop — the mock streams no content on the
|
||||
forced final), the turn settles with ``done`` (never the error
|
||||
banner) and the SSE capture has no ``error`` frame;
|
||||
3. ``test_done_flags_budget_exhausted`` — the captured SSE
|
||||
stream's ``done`` frame carries ``budget_exhausted: true`` on
|
||||
the capped turn; a normal grounded turn's ``done`` carries NO
|
||||
``budget_exhausted`` key at all (absent, never false — the
|
||||
byte-identical control case in the same file); the capped
|
||||
answer's Continue button is gone once the newer turn lands;
|
||||
4. ``test_continue_button_runs_a_fresh_turn`` — the capped answer
|
||||
bubble shows **Continue researching**; clicking it appends
|
||||
exactly ONE user bubble ``continue``, the mock answers with
|
||||
``ROUND_CAP_CONTINUE_ANSWER`` (a real continuation answer — a
|
||||
fixed sentence naming the seed document, not a tool call), the
|
||||
earlier tool lines do not duplicate (still exactly three — the
|
||||
continuation turn streams no tool frames), and ``bor.chat.v1``
|
||||
holds ``[user, brain(flagged), user("continue"), brain]`` (the
|
||||
flagged record carries ``budget_exhausted: true``);
|
||||
5. ``test_continue_is_inert_while_streaming_and_single_shot`` —
|
||||
double-clicking Continue (``page.dblclick`` — the second click
|
||||
lands after the first handler removed every ``.continue-btn``
|
||||
and started the turn; the in-flight no-op guard + the
|
||||
remove-first clear make a double-send impossible) starts ONE
|
||||
turn (counted via ``page.on("request")`` — exactly one
|
||||
``/api/chat`` POST; the single ``continue`` user bubble + the
|
||||
single continuation answer pin it);
|
||||
6. ``test_flag_survives_a_refresh`` — ``page.reload()`` after a
|
||||
capped turn: the flagged answer restores WITH the Continue
|
||||
button (the restore path keys on ``m.budget_exhausted``) and
|
||||
clicking it still works (the mock's continuation answer
|
||||
streams);
|
||||
7. negative: ``test_deflected_turn_gets_no_continue_button`` — a
|
||||
deflected question (A8 LOW mode — token-disjoint from the KB:
|
||||
``Who composed Rhapsody in Blue?`` ≈0.05, zero FTS hits)
|
||||
settles deflected with NO ``budget_exhausted`` key, NO ``tool``
|
||||
frame and NO Continue button.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from playwright.sync_api import Page, expect
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings as _Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document
|
||||
from app.rag.agent import (
|
||||
ROUND_CAP_FALLBACK_ANSWER,
|
||||
ROUND_CAP_WRAPUP_INSTRUCTION,
|
||||
TOOL_BUDGET_LAST,
|
||||
TOOL_BUDGET_REMAINING,
|
||||
TOOL_BUDGET_SPENT,
|
||||
)
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import (
|
||||
ADMIN_PASSWORD,
|
||||
SESSION_SECRET,
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from tests.e2e.mock_llm import ROUND_CAP_CONTINUE_ANSWER, embed_text
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
#: This module app binds its OWN port (the conftest session app
|
||||
#: owns its port in a combined run — a same-port second uvicorn
|
||||
#: dies on bind and would drive the wrong server; the
|
||||
#: test_big_read_progress.py convention, env-overridable).
|
||||
APP_PORT = int(os.environ.get("E2E_APP_PORT_TOOLBUDGET", "8129"))
|
||||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||||
|
||||
#: The cap the whole suite runs under (the module app's env
|
||||
#: override): the cap flow burns exactly three executed rounds —
|
||||
#: the trailer math runs over a 3-round budget (2 → 1 → spent).
|
||||
MAX_ROUNDS = 3
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Seed + questions (see the module docstring for the gate notes)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
SEED_SOURCE = "docs"
|
||||
SEED_PATH = "homelab/kubernetes.md"
|
||||
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
|
||||
KUB_CONTENT = (REPO / "tests" / "fixtures" / "docs" / SEED_PATH).read_text()
|
||||
|
||||
#: The cap-flow question: the tokens clear the mock-calibrated
|
||||
#: gate (≈0.44 against the mock's token-overlap embeddings, past
|
||||
#: the 0.30 threshold — HIGH without any FTS help), and it carries
|
||||
#: ``ROUND_CAP_TRIGGER``.
|
||||
CAP_QUESTION = (
|
||||
"Explain my kubernetes cluster: talos linux nodes, cilium "
|
||||
"networking, local-path-provisioner and longhorn storage. "
|
||||
"run out of tool calls"
|
||||
)
|
||||
assert "run out of tool calls" in CAP_QUESTION.lower()
|
||||
|
||||
#: The normal grounded control (test 3 — the byte-identical
|
||||
#: ``done`` case): token overlap with the seed (≈0.21, past the
|
||||
#: 0.15 lexical floor) + FTS hits — grounded, no trigger, no tool
|
||||
#: calls (the mock answers plain).
|
||||
CONTROL_QUESTION = "How is my Kubernetes cluster set up?"
|
||||
|
||||
#: The negative (test 7 — A8 LOW): token-disjoint from the KB
|
||||
#: (≈0.05, zero FTS hits — deflected; the phase-112 calibrated
|
||||
#: question).
|
||||
DEFLECT_QUESTION = "Who composed Rhapsody in Blue?"
|
||||
|
||||
STORAGE_KEY = "bor.chat.v1"
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# The module app (conftest env shape + the BOR_AGENT_MAX_ROUNDS=3
|
||||
# swap)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_server(mock_llm: int) -> Iterator[str]:
|
||||
"""The app under test — the conftest's session-app env shape
|
||||
(including the phase-61/62 leak guards) with one swap:
|
||||
``BOR_AGENT_MAX_ROUNDS=3`` (the cap flow's denominator — the
|
||||
trailer math runs over a 3-round budget)."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||
env["BOR_LLM_BASE_URL"] = (
|
||||
"https://aipi.reeseapps.com/v1"
|
||||
if USE_REAL_LLM
|
||||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||||
)
|
||||
# The E2E mock's token-overlap embeddings have their own score
|
||||
# distribution (phase 09) — the mock-calibrated values (the
|
||||
# conftest pattern).
|
||||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||||
# Phase 67: instant retry waits; the real retry budget (the
|
||||
# conftest pattern).
|
||||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||||
env.setdefault(
|
||||
"BOR_DATABASE_URL",
|
||||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||
)
|
||||
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||
# Phase 61/62 leak guards (the conftest pattern): force the code
|
||||
# defaults so an operator's local (gitignored) ``.env`` cannot
|
||||
# leak into the app under test.
|
||||
env["BOR_DOCS_REPO"] = ""
|
||||
env["BOR_SUGGESTIONS"] = json.dumps(
|
||||
_Settings.model_fields["suggestions"].default
|
||||
)
|
||||
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
|
||||
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
|
||||
# The suite's swap: the cap flow burns exactly three executed
|
||||
# rounds (the trailer math: 2 → 1 → spent).
|
||||
env["BOR_AGENT_MAX_ROUNDS"] = str(MAX_ROUNDS)
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||||
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
||||
cwd=REPO,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
_wait_http(f"{APP_URL}/api/health")
|
||||
yield APP_URL
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url(app_server: str) -> str:
|
||||
return app_server
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# DB seeding (TRUNCATE-then-seed, cf. test_failed_turn_retry.py)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed(db: Session) -> None:
|
||||
"""The single fixture document (see the module docstring)."""
|
||||
md = Document(
|
||||
source=SEED_SOURCE,
|
||||
path=SEED_PATH,
|
||||
full_path=f"/tmp/{SEED_PATH}",
|
||||
title="Kubernetes",
|
||||
content=KUB_CONTENT,
|
||||
content_hash=hashlib.sha256(KUB_CONTENT.encode()).hexdigest(),
|
||||
indexed_at=datetime.now(UTC),
|
||||
)
|
||||
db.add(md)
|
||||
db.flush()
|
||||
# One chunk carrying the mock's own embedding → genuine token
|
||||
# overlap for the grounded questions (the FTS path carries them
|
||||
# to HIGH).
|
||||
db.add(
|
||||
Chunk(
|
||||
document_id=md.id,
|
||||
position=0,
|
||||
content=KUB_CONTENT,
|
||||
embedding=embed_text(KUB_CONTENT),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _reset_db() -> None:
|
||||
"""Truncate the KB (plus the prompt-shaping tables), then
|
||||
re-seed.
|
||||
|
||||
``steering_notes`` / ``kb_overview`` are truncated too, so the
|
||||
prompts are byte-stable regardless of leftovers from other
|
||||
suites.
|
||||
"""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
_seed(db)
|
||||
db.commit()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_chats() -> Iterator[None]:
|
||||
"""The send auto-saves a ``saved_chats`` row per turn (phase 55) —
|
||||
truncate around every test so the suite starts from (and leaves)
|
||||
an empty deployment (the phase-80/103 isolation pattern)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
yield
|
||||
# The auto-save is fire-and-forget (the browser never awaits the
|
||||
# PUT): a short margin lets the last turn's PUT land server-side
|
||||
# before the TRUNCATE, so the teardown never 500s an in-flight
|
||||
# upsert (the "Could not refresh instance" race). This runs AFTER
|
||||
# the page closes (LIFO finalization), so the PUT is either done
|
||||
# or aborted — never in flight.
|
||||
time.sleep(0.75)
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Page helpers (the SSE capture — the phase-37/67 pattern) + the
|
||||
# mock's request-log capture
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
SSE_HOOK = """
|
||||
() => {
|
||||
if (window.__sseInstalled) return;
|
||||
window.__sseInstalled = true;
|
||||
window.__sseFrames = [];
|
||||
const origFetch = window.fetch;
|
||||
window.fetch = async function (...args) {
|
||||
const res = await origFetch.apply(this, args);
|
||||
try {
|
||||
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
|
||||
if (url.includes('/api/chat')) {
|
||||
res.clone().text().then((bodyText) => {
|
||||
for (const block of bodyText.split('\\n\\n')) {
|
||||
const line = block.trim();
|
||||
if (line.startsWith('data: ')) {
|
||||
window.__sseFrames.push(line.slice(6));
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (e) { /* non-clonable responses: ignored */ }
|
||||
return res;
|
||||
};
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _install_sse_hook(page: Page) -> None:
|
||||
page.evaluate(SSE_HOOK)
|
||||
|
||||
|
||||
def _frames(page: Page, terminal: str = "done") -> list[dict]:
|
||||
"""The captured SSE frames of the CURRENT turn, once the
|
||||
*terminal* frame lands."""
|
||||
deadline = time.monotonic() + 30.0
|
||||
while True:
|
||||
raw = page.evaluate("() => window.__sseFrames || []")
|
||||
parsed = [json.loads(line) for line in raw if line]
|
||||
if any(f.get("type") == terminal for f in parsed):
|
||||
return parsed
|
||||
if time.monotonic() > deadline:
|
||||
raise AssertionError(
|
||||
f"SSE hook captured no `{terminal}` frame (frames so far: "
|
||||
f"{len(parsed)}) — hook install failed?"
|
||||
)
|
||||
time.sleep(0.05)
|
||||
|
||||
|
||||
def _stored(page: Page) -> dict:
|
||||
"""The persisted ``bor.chat.v1`` conversation (localStorage — the
|
||||
phase-14 store the restore reads on refresh)."""
|
||||
raw = page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
|
||||
assert raw is not None, "the conversation key must exist in localStorage"
|
||||
return json.loads(raw)
|
||||
|
||||
|
||||
def _submit(page: Page, question: str) -> None:
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
# The user bubble lands synchronously with the submit handler.
|
||||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||||
|
||||
|
||||
def _wait_settled(page: Page) -> None:
|
||||
"""The turn is complete: answer text in the bubble, button
|
||||
recovered (the phase-48 settle convention)."""
|
||||
expect(page.locator(".msg.brain .bubble").last).not_to_have_text(
|
||||
"", timeout=30_000
|
||||
)
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
|
||||
def _mock_capture_reset(mock_port: int) -> None:
|
||||
"""Clear the mock's request capture so a suite asserts on
|
||||
EXACTLY the requests of the turn it is about to drive (the
|
||||
phase-123 capture pair)."""
|
||||
r = httpx.post(
|
||||
f"http://127.0.0.1:{mock_port}/v1/e2e/captured/reset", timeout=5
|
||||
)
|
||||
assert r.status_code == 200
|
||||
|
||||
|
||||
def _mock_captured(mock_port: int) -> list[dict]:
|
||||
"""The mock's recorded chat-completions request bodies (oldest
|
||||
first) — the wire-level pin surface for test 1."""
|
||||
r = httpx.get(
|
||||
f"http://127.0.0.1:{mock_port}/v1/e2e/captured", timeout=5
|
||||
)
|
||||
assert r.status_code == 200
|
||||
return r.json()
|
||||
|
||||
|
||||
def _tool_msgs(messages: list[dict]) -> list[str]:
|
||||
"""The ``tool``-role result contents of one request, in order."""
|
||||
return [
|
||||
str(m.get("content") or "")
|
||||
for m in messages
|
||||
if m.get("role") == "tool"
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 1. The budget trailer on every executed tool result (wire level)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_tool_results_carry_the_remaining_count(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_mock_capture_reset(mock_llm)
|
||||
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
|
||||
# Wire level (the mock's recorded request log — the app's
|
||||
# EXACT requests): exactly four chat requests — three executed
|
||||
# tool rounds + the forced final.
|
||||
bodies = _mock_captured(mock_llm)
|
||||
assert len(bodies) == 4, len(bodies)
|
||||
|
||||
# Request 1: no tool results yet; the tools are offered.
|
||||
assert bodies[0].get("tools"), "request 1 must offer the tools"
|
||||
assert _tool_msgs(bodies[0]["messages"]) == []
|
||||
|
||||
# Round 1's result (in request 2): the decreasing trailer —
|
||||
# ``[tool calls remaining: 2]`` (two of three left after the
|
||||
# call it is reading).
|
||||
msgs2 = _tool_msgs(bodies[1]["messages"])
|
||||
assert len(msgs2) == 1, msgs2
|
||||
assert msgs2[0].endswith(TOOL_BUDGET_REMAINING.format(n=2)), msgs2[0]
|
||||
|
||||
# Round 2's result (in request 3): the last-call line (one left
|
||||
# — "make it count, then answer").
|
||||
msgs3 = _tool_msgs(bodies[2]["messages"])
|
||||
assert len(msgs3) == 2, msgs3
|
||||
assert msgs3[0].endswith(TOOL_BUDGET_REMAINING.format(n=2)), msgs3[0]
|
||||
assert msgs3[1].endswith(TOOL_BUDGET_LAST), msgs3[1]
|
||||
|
||||
# The boundary call's result (in request 4 — the forced final):
|
||||
# the SPENT line (the budget is gone; say "continue").
|
||||
msgs4 = _tool_msgs(bodies[3]["messages"])
|
||||
assert len(msgs4) == 3, msgs4
|
||||
assert msgs4[2].endswith(TOOL_BUDGET_SPENT), msgs4[2]
|
||||
|
||||
# The mock's incrementing ids (the flow's stateless
|
||||
# derivation): call_cap_1 → call_cap_2 → call_cap_3 in the
|
||||
# assistant messages.
|
||||
asst_ids = [
|
||||
bodies[i + 1]["messages"][2 * i + 2]["tool_calls"][0]["id"]
|
||||
for i in range(3)
|
||||
]
|
||||
assert asst_ids == ["call_cap_1", "call_cap_2", "call_cap_3"], asst_ids
|
||||
|
||||
# The final request is the no-tools one, coached: no ``tools``
|
||||
# key (or null), the system message carries the wrap-up
|
||||
# instruction, and the user question + the tool history stay in
|
||||
# place.
|
||||
assert bodies[3].get("tools") is None
|
||||
systems = [
|
||||
str(m.get("content") or "")
|
||||
for m in bodies[3]["messages"]
|
||||
if m.get("role") == "system"
|
||||
]
|
||||
assert any(ROUND_CAP_WRAPUP_INSTRUCTION in s for s in systems), systems
|
||||
users = [
|
||||
str(m.get("content") or "")
|
||||
for m in bodies[3]["messages"]
|
||||
if m.get("role") == "user"
|
||||
]
|
||||
assert CAP_QUESTION in users, users
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 2. Past the cap: zero tool lines after the cap, the fallback
|
||||
# answer, no error
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_capped_turn_never_shows_tool_lines_after_the_cap(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
|
||||
frames = _frames(page)
|
||||
|
||||
# The turn settles with ``done`` — never the terminal ``error``
|
||||
# frame (the pre-phase-160 cap path's MalformedReplyError class
|
||||
# is gone; the budget fallback is the terminal).
|
||||
assert not [f for f in frames if f.get("type") == "error"], frames
|
||||
assert frames[-1]["type"] == "done"
|
||||
|
||||
# Wire: EXACTLY three ``tool`` frames — one per executed round
|
||||
# (the cap). The dump call (the forced final's structured
|
||||
# ``read`` on the seed path) produces NO frame — it was
|
||||
# rejected (never executed, never shown).
|
||||
tool_frames = [f for f in frames if f.get("type") == "tool"]
|
||||
assert len(tool_frames) == MAX_ROUNDS, tool_frames
|
||||
assert all(f["name"] == "ls" for f in tool_frames), tool_frames
|
||||
assert not [f for f in tool_frames if f.get("argument") == SEED_SP]
|
||||
|
||||
# UI: exactly three tool lines (all the no-arg top-level
|
||||
# listings); the dump call's ``Reading <seed path>`` line is
|
||||
# NOWHERE — the rejected call never renders.
|
||||
lines = page.locator(".msg.brain .tool-call")
|
||||
expect(lines).to_have_count(3)
|
||||
for i in range(3):
|
||||
expect(lines.nth(i)).to_contain_text("Listing documents")
|
||||
msg_text = page.locator(".msg.brain").last.text_content() or ""
|
||||
assert "Reading " not in msg_text, msg_text
|
||||
|
||||
# The answer: the deterministic backstop line (the mock streams
|
||||
# no content on the forced final — only the rejected dump
|
||||
# call) — non-empty and readable.
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(ROUND_CAP_FALLBACK_ANSWER)
|
||||
|
||||
# No error banner (the turn settled, never the failure path).
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 3. done.budget_exhausted: true exactly on the capped turn, ABSENT
|
||||
# (never false) on the normal grounded control
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_done_flags_budget_exhausted(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
# The capped turn: ``done`` carries ``budget_exhausted: true``.
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
frames = _frames(page)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["budget_exhausted"] is True, done
|
||||
assert done["deflected"] is False
|
||||
# The flagged answer offers the Continue affordance (task 04).
|
||||
expect(page.locator(".continue-btn")).to_have_count(1)
|
||||
|
||||
# The normal grounded control: the ``done`` frame carries NO
|
||||
# ``budget_exhausted`` key at all (absent, never false — the
|
||||
# byte-identical off rule, PLAN §13).
|
||||
page.evaluate("() => { window.__sseFrames = []; }")
|
||||
_submit(page, CONTROL_QUESTION)
|
||||
_wait_settled(page)
|
||||
frames = _frames(page)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert "budget_exhausted" not in done, done
|
||||
assert done["deflected"] is False
|
||||
|
||||
# The newer turn's landing cleared the capped answer's Continue
|
||||
# button (the remove-first pattern — a stale "continue" would
|
||||
# append to a history the conversation moved past).
|
||||
expect(page.locator(".continue-btn")).to_have_count(0)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 4. Continue researching: one fresh turn, the real continuation
|
||||
# answer, the record shape
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_continue_button_runs_a_fresh_turn(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
||||
ROUND_CAP_FALLBACK_ANSWER
|
||||
)
|
||||
|
||||
# The flagged answer offers **Continue researching**.
|
||||
btn = page.locator(".continue-btn")
|
||||
expect(btn).to_have_count(1)
|
||||
expect(btn).to_contain_text("Continue researching")
|
||||
|
||||
# Fresh capture window: the continuation turn's frames only
|
||||
# (the cap turn's frames are still in the hook's buffer).
|
||||
page.evaluate("() => { window.__sseFrames = []; }")
|
||||
btn.click()
|
||||
|
||||
# Exactly ONE ``continue`` user bubble (the plain send).
|
||||
expect(page.locator(".msg.user .bubble")).to_have_count(2)
|
||||
expect(page.locator(".msg.user .bubble").last).to_have_text("continue")
|
||||
|
||||
# … and the mock's REAL continuation answer (a fixed sentence
|
||||
# naming the seed document — not a tool call).
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(ROUND_CAP_CONTINUE_ANSWER, timeout=30_000)
|
||||
_wait_settled(page)
|
||||
|
||||
# The continuation turn streams NO tool frames — the earlier
|
||||
# tool lines do not duplicate (still exactly three).
|
||||
frames = _frames(page)
|
||||
assert [f for f in frames if f.get("type") == "tool"] == []
|
||||
expect(page.locator(".msg.brain .tool-call")).to_have_count(3)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
# A FRESH turn: the budget is whole again — the flag is absent
|
||||
# (the continuation turn deflects — the literal "continue"
|
||||
# shares no tokens with the KB, so the A8 gate is LOW).
|
||||
assert "budget_exhausted" not in done, done
|
||||
assert done["deflected"] is True
|
||||
|
||||
# The record: [user, brain(flagged), user("continue"), brain].
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == [
|
||||
"user",
|
||||
"brain",
|
||||
"user",
|
||||
"brain",
|
||||
]
|
||||
assert stored["messages"][1]["budget_exhausted"] is True
|
||||
assert stored["messages"][1]["text"] == ROUND_CAP_FALLBACK_ANSWER
|
||||
assert stored["messages"][2]["text"] == "continue"
|
||||
assert ROUND_CAP_CONTINUE_ANSWER in stored["messages"][3]["text"]
|
||||
assert not stored["messages"][3].get("budget_exhausted")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 5. Single-shot: double-clicking Continue starts ONE turn
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_continue_is_inert_while_streaming_and_single_shot(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
expect(page.locator(".continue-btn")).to_have_count(1)
|
||||
|
||||
# Count the ``/api/chat`` POSTs (the single-shot pin) — the
|
||||
# exact path (the auto-save's ``PUT /api/chats/…`` must not
|
||||
# match: ``/api/chats`` is not ``/api/chat``).
|
||||
posts: list[int] = []
|
||||
page.on(
|
||||
"request",
|
||||
lambda req: (
|
||||
posts.append(1)
|
||||
if req.method == "POST"
|
||||
and req.url.split("?", 1)[0].endswith("/api/chat")
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
# The double-click: the FIRST click runs the handler (removes
|
||||
# every ``.continue-btn`` and starts the turn); the SECOND
|
||||
# click lands after the removal — no second target (and the
|
||||
# in-flight no-op guard would swallow it anyway) — ONE turn.
|
||||
page.locator(".continue-btn").dblclick()
|
||||
_wait_settled(page)
|
||||
assert len(posts) == 1, posts
|
||||
|
||||
# ONE ``continue`` user bubble + ONE continuation answer (no
|
||||
# double-send: a second turn would add a second user bubble +
|
||||
# a second brain answer).
|
||||
expect(page.locator(".msg.user .bubble")).to_have_count(2)
|
||||
expect(page.locator(".msg.brain")).to_have_count(2)
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(ROUND_CAP_CONTINUE_ANSWER, timeout=30_000)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 6. The flag survives a refresh (the restore path)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_flag_survives_a_refresh(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, CAP_QUESTION)
|
||||
_wait_settled(page)
|
||||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
||||
ROUND_CAP_FALLBACK_ANSWER
|
||||
)
|
||||
expect(page.locator(".continue-btn")).to_have_count(1)
|
||||
|
||||
# The refresh: the flagged answer restores WITH the Continue
|
||||
# button (the restore path keys on ``m.budget_exhausted``).
|
||||
page.reload()
|
||||
expect(page.locator("#messages > .msg")).to_have_count(2)
|
||||
expect(page.locator(".msg.user .bubble")).to_have_text(CAP_QUESTION)
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(ROUND_CAP_FALLBACK_ANSWER, timeout=30_000)
|
||||
btn = page.locator(".continue-btn")
|
||||
expect(btn).to_have_count(1)
|
||||
expect(btn).to_contain_text("Continue researching")
|
||||
|
||||
# The restored button WORKS: clicking it runs the continuation
|
||||
# turn (the mock's continuation answer streams).
|
||||
btn.click()
|
||||
expect(page.locator(".msg.user .bubble")).to_have_count(2)
|
||||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
||||
ROUND_CAP_CONTINUE_ANSWER, timeout=30_000
|
||||
)
|
||||
_wait_settled(page)
|
||||
|
||||
# The record: [user, brain(flagged), user("continue"), brain].
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == [
|
||||
"user",
|
||||
"brain",
|
||||
"user",
|
||||
"brain",
|
||||
]
|
||||
assert stored["messages"][1]["budget_exhausted"] is True
|
||||
assert stored["messages"][2]["text"] == "continue"
|
||||
assert ROUND_CAP_CONTINUE_ANSWER in stored["messages"][3]["text"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 7. Negative: a deflected turn — no flag, no Continue button
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_deflected_turn_gets_no_continue_button(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, DEFLECT_QUESTION)
|
||||
_wait_settled(page)
|
||||
|
||||
frames = _frames(page)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
# Settled DEFLECTED (A8 LOW), with NO ``budget_exhausted`` key
|
||||
# (the flag is cap-reached turns only) and NO ``tool`` frame
|
||||
# (a tools=None request never emits one).
|
||||
assert done["deflected"] is True, done
|
||||
assert "budget_exhausted" not in done, done
|
||||
assert [f for f in frames if f.get("type") == "tool"] == []
|
||||
|
||||
# The deflected bubble — the honest copy, and NO Continue
|
||||
# button (the affordance is cap-reached answers only).
|
||||
last = page.locator(".msg.brain").last
|
||||
expect(last).to_have_class(re.compile(r"is-deflected"))
|
||||
expect(page.locator(".msg.brain .tool-call")).to_have_count(0)
|
||||
expect(page.locator(".continue-btn")).to_have_count(0)
|
||||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
||||
"I haven't done anything like that"
|
||||
)
|
||||
@@ -320,13 +320,13 @@ def test_ls_top_level_lists_registered_sources_through_run_agent(
|
||||
holder, llm = _run_call(db, "ls", {})
|
||||
|
||||
assert llm.requests[0][1] == AGENT_TOOLS
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"2 sources:\n"
|
||||
"\n"
|
||||
"Deployments — 1 documents\n"
|
||||
"Homelab — 2 documents\n"
|
||||
" Home lab notes."
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1
|
||||
assert holder.read_docs == []
|
||||
|
||||
@@ -352,7 +352,7 @@ def test_ls_top_level_empty_registry_through_run_agent(
|
||||
db.commit()
|
||||
|
||||
holder, llm = _run_call(db, "ls", {})
|
||||
assert llm.requests[1][0][3]["content"] == "0 sources:"
|
||||
assert llm.requests[1][0][3]["content"].startswith("0 sources:")
|
||||
assert holder.tool_calls == 1
|
||||
|
||||
|
||||
@@ -375,20 +375,20 @@ def test_ls_source_scope_lists_root_folder_through_run_agent(kb, registry, db) -
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Homelab"})
|
||||
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"Homelab — 1 documents, 2 folders:\n"
|
||||
"\n"
|
||||
" backups/ — 2 documents: Backup notes.\n"
|
||||
" networking/ — 1 documents\n"
|
||||
"\n"
|
||||
"source: Homelab | path: readme.md | title: Readme | date: 2024-06-15"
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1
|
||||
assert holder.read_docs == []
|
||||
|
||||
# A registered source with no documents: the header line alone.
|
||||
holder0, llm0 = _run_call(db, "ls", {"path": "Deployments"})
|
||||
assert llm0.requests[1][0][3]["content"] == "Deployments — 0 documents, 0 folders:"
|
||||
assert llm0.requests[1][0][3]["content"].startswith("Deployments — 0 documents, 0 folders:")
|
||||
assert holder0.tool_calls == 1
|
||||
|
||||
|
||||
@@ -405,23 +405,23 @@ def test_ls_nested_folder_scope_drills_one_level_through_run_agent(
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Homelab/networking"})
|
||||
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"Homelab/networking — 0 documents, 2 folders:\n"
|
||||
"\n"
|
||||
" networking/lan/ — 2 documents\n"
|
||||
" networking/vpn/ — 1 documents"
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1
|
||||
assert holder.read_docs == []
|
||||
|
||||
# One level deeper.
|
||||
holder2, llm2 = _run_call(db, "ls", {"path": "Homelab/networking/lan"})
|
||||
assert llm2.requests[1][0][3]["content"] == (
|
||||
assert llm2.requests[1][0][3]["content"].startswith(
|
||||
"Homelab/networking/lan — 2 documents, 0 folders:\n"
|
||||
"\n"
|
||||
"source: Homelab | path: networking/lan/a.md | title: A | date: 2024-06-15\n"
|
||||
"source: Homelab | path: networking/lan/b.md | title: B | date: 2024-06-15"
|
||||
)
|
||||
)
|
||||
assert holder2.tool_calls == 1
|
||||
|
||||
|
||||
@@ -434,7 +434,9 @@ def test_ls_folder_file_cap_through_run_agent(kb, registry, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Homelab/big"})
|
||||
|
||||
content = llm.requests[1][0][3]["content"]
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the
|
||||
# listing — the raw listing is the RAW part (the trailer stripped).
|
||||
content = llm.requests[1][0][3]["content"].removesuffix(agent._budget_note(9))
|
||||
lines = content.splitlines()
|
||||
assert lines[0] == "Homelab/big — 51 documents, 0 folders:"
|
||||
assert lines[2] == (
|
||||
@@ -460,9 +462,8 @@ def test_ls_scoped_unknown_source_refused_through_run_agent(kb, src, db) -> None
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Ghost"})
|
||||
|
||||
assert (
|
||||
llm.requests[1][0][3]["content"]
|
||||
== agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost")
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost")
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
|
||||
@@ -477,9 +478,8 @@ def test_ls_path_like_scope_unknown_source_gets_no_source_refusal(kb, src, db) -
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "app/rag/importer.py"})
|
||||
|
||||
assert (
|
||||
llm.requests[1][0][3]["content"]
|
||||
== agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app")
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app")
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
|
||||
@@ -500,10 +500,10 @@ def test_ls_unknown_folder_gets_not_a_folder_with_parents_subfolders(
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Homelab/netwoking"})
|
||||
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"'Homelab/netwoking' is not a folder — Homelab has: "
|
||||
"backups/ containers/ networking/"
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
|
||||
|
||||
@@ -523,11 +523,11 @@ def test_read_combined_path_through_run_agent(kb, db) -> None:
|
||||
|
||||
# Phase 106 (D5): the date rides every read — the SECOND line (the
|
||||
# first line stays the byte-identical header).
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"Document Alpha/deep/nested/doc.md:\n"
|
||||
"date: 2024-06-15\n"
|
||||
"FULL-TEXT"
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1
|
||||
# SEC-14-04: short-lived session loads fresh copies
|
||||
assert [d.id for d in holder.read_docs] == [created.id]
|
||||
@@ -542,8 +542,8 @@ def test_read_bare_source_name_refused_through_run_agent(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "read", {"path": "Alpha"})
|
||||
|
||||
assert (
|
||||
llm.requests[1][0][3]["content"] == "No document at 'Alpha' — check the ls output."
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No document at 'Alpha' — check the ls output."
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
|
||||
@@ -558,9 +558,8 @@ def test_read_unknown_combined_path_refused_through_run_agent(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "read", {"path": "Alpha/nope/deep.md"})
|
||||
|
||||
assert (
|
||||
llm.requests[1][0][3]["content"]
|
||||
== "No document at 'Alpha/nope/deep.md' — check the ls output."
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No document at 'Alpha/nope/deep.md' — check the ls output."
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
|
||||
@@ -592,19 +591,19 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) ->
|
||||
# Round 1: the bare path resolves to no combined identity, but it IS
|
||||
# the indexed document's path — the refusal names the one combined
|
||||
# identity to use (not counted, the tools stay offered).
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No document at 'deep/nested/doc.md' — "
|
||||
"did you mean 'Alpha/deep/nested/doc.md'?"
|
||||
)
|
||||
)
|
||||
assert llm.requests[1][1] == AGENT_TOOLS
|
||||
# Round 2: the corrected combined identity succeeds — the full
|
||||
# content (plus the phase-106 D5 date line), the holder records the
|
||||
# row, and it counts.
|
||||
assert llm.requests[2][0][5]["content"] == (
|
||||
assert llm.requests[2][0][5]["content"].startswith(
|
||||
"Document Alpha/deep/nested/doc.md:\n"
|
||||
"date: 2024-06-15\n"
|
||||
"FULL-TEXT"
|
||||
)
|
||||
)
|
||||
assert llm.requests[2][1] == AGENT_TOOLS
|
||||
# SEC-14-04: short-lived session loads fresh copies
|
||||
assert [d.id for d in holder.read_docs] == [created.id]
|
||||
@@ -633,13 +632,13 @@ def test_read_bare_path_two_sources_one_of_suggestion_then_corrected_read(
|
||||
holder = AgentHolder()
|
||||
asyncio.run(_consume(cast("LLMClient", llm), lambda: SessionLocal(), holder))
|
||||
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No document at 'shared/x.md' — did you mean one of: "
|
||||
"'Alpha/shared/x.md', 'Beta/shared/x.md'?"
|
||||
)
|
||||
assert llm.requests[2][0][5]["content"] == (
|
||||
)
|
||||
assert llm.requests[2][0][5]["content"].startswith(
|
||||
"Document Alpha/shared/x.md:\ndate: 2024-06-15\nA-TEXT"
|
||||
)
|
||||
)
|
||||
# SEC-14-04: short-lived session loads fresh copies
|
||||
assert [d.id for d in holder.read_docs] == [a.id]
|
||||
assert holder.tool_calls == 1 # only the corrected read executed
|
||||
@@ -673,10 +672,10 @@ def test_grep_whole_kb_through_run_agent(kb, db) -> None:
|
||||
# Offered: the first request carries AGENT_TOOLS (the 3-tool list).
|
||||
assert llm.requests[0][1] == AGENT_TOOLS
|
||||
# Executed against the real DB: catalog order, grep-style lines.
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"Alpha/a/one.md:2: needle in one\n"
|
||||
"Beta/b/two.md:2: NEEDLE in two"
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1
|
||||
assert holder.read_docs == [] # locked A5: grep adds no context
|
||||
|
||||
@@ -689,7 +688,7 @@ def test_grep_scoped_through_run_agent(kb, db) -> None:
|
||||
holder, llm = _run_call(db, "grep", {"pattern": "needle", "path": "Alpha/a/one.md"})
|
||||
|
||||
# Only the named document is searched — the other one's hit is absent.
|
||||
assert llm.requests[1][0][3]["content"] == "Alpha/a/one.md:2: Needle here"
|
||||
assert llm.requests[1][0][3]["content"].startswith("Alpha/a/one.md:2: Needle here")
|
||||
assert holder.tool_calls == 1
|
||||
assert holder.read_docs == []
|
||||
|
||||
@@ -703,9 +702,8 @@ def test_grep_scoped_missing_doc_refused_through_run_agent(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "grep", {"pattern": "needle", "path": "Alpha/ghost.md"})
|
||||
|
||||
assert (
|
||||
llm.requests[1][0][3]["content"]
|
||||
== "No document at 'Alpha/ghost.md' — check the ls output."
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No document at 'Alpha/ghost.md' — check the ls output."
|
||||
)
|
||||
assert holder.tool_calls == 0 and holder.read_docs == []
|
||||
|
||||
@@ -716,8 +714,8 @@ def test_grep_no_matches_through_run_agent(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "grep", {"pattern": "zebra"})
|
||||
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"No matches for 'zebra' in the knowledge base."
|
||||
)
|
||||
)
|
||||
assert holder.tool_calls == 1 # an executed grep with zero hits
|
||||
assert holder.read_docs == []
|
||||
|
||||
@@ -28,7 +28,7 @@ from sqlalchemy.orm import Session
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Document, GitSource
|
||||
from app.rag.agent import AgentHolder, run_agent
|
||||
from app.rag.agent import AgentHolder, _budget_note, run_agent
|
||||
from app.rag.llm import LLMClient, RetryPiece, StreamPiece, ToolCallPiece, ToolResultPiece
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -160,7 +160,9 @@ def test_read_result_second_line_is_stored_date(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "read", {"path": "Alpha/deep/nested/doc.md"})
|
||||
|
||||
content = llm.requests[1][0][3]["content"]
|
||||
# Phase 160 (task 01): the trailer rides the read result — the raw
|
||||
# result is the RAW part (the trailer stripped).
|
||||
content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9))
|
||||
lines = content.splitlines()
|
||||
assert lines[0] == "Document Alpha/deep/nested/doc.md:" # byte-identical header
|
||||
assert lines[1] == f"date: {D2_STR}" # the STORED date (row's UTC date part)
|
||||
@@ -178,13 +180,13 @@ def test_read_result_date_is_the_row_date_not_a_constant(kb, db) -> None:
|
||||
db.commit()
|
||||
|
||||
holder_a, llm_a = _run_call(db, "read", {"path": "Alpha/a.md"})
|
||||
assert llm_a.requests[1][0][3]["content"] == (
|
||||
assert llm_a.requests[1][0][3]["content"].startswith(
|
||||
f"Document Alpha/a.md:\ndate: {D1_STR}\nA-TEXT"
|
||||
)
|
||||
)
|
||||
holder_b, llm_b = _run_call(db, "read", {"path": "Alpha/b.md"})
|
||||
assert llm_b.requests[1][0][3]["content"] == (
|
||||
assert llm_b.requests[1][0][3]["content"].startswith(
|
||||
f"Document Alpha/b.md:\ndate: {D3_STR}\nB-TEXT"
|
||||
)
|
||||
)
|
||||
# SEC-14-04: short-lived sessions
|
||||
assert [d.id for d in holder_a.read_docs] == [a.id]
|
||||
assert [d.id for d in holder_b.read_docs] == [b.id]
|
||||
@@ -207,7 +209,9 @@ def test_ls_drill_file_lines_carry_their_dates(kb, src, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "ls", {"path": "Homelab"})
|
||||
|
||||
content = llm.requests[1][0][3]["content"]
|
||||
# Phase 160 (task 01): the trailer rides the listing — stripped for
|
||||
# the raw-part pins below.
|
||||
content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9))
|
||||
lines = content.splitlines()
|
||||
# The root level: one direct file (readme.md — D1) + the two
|
||||
# subfolder lines (date-free) + the date-free header.
|
||||
@@ -221,7 +225,7 @@ def test_ls_drill_file_lines_carry_their_dates(kb, src, db) -> None:
|
||||
|
||||
# Drill into backups: BOTH files list, each with its OWN date.
|
||||
holder2, llm2 = _run_call(db, "ls", {"path": "Homelab/backups"})
|
||||
lines2 = llm2.requests[1][0][3]["content"].splitlines()
|
||||
lines2 = llm2.requests[1][0][3]["content"].removesuffix(_budget_note(9)).splitlines()
|
||||
assert lines2[0] == "Homelab/backups — 2 documents, 0 folders:"
|
||||
assert lines2[2] == (
|
||||
f"source: Homelab | path: backups/cron.md | title: Cron | date: {D1_STR}"
|
||||
@@ -247,7 +251,7 @@ def test_ls_top_level_source_lines_carry_no_date(kb, db) -> None:
|
||||
|
||||
holder, llm = _run_call(db, "ls", {})
|
||||
|
||||
content = llm.requests[1][0][3]["content"]
|
||||
content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9))
|
||||
assert content == "1 sources:\n\nHomelab — 1 documents"
|
||||
assert "date" not in content
|
||||
assert holder.tool_calls == 1
|
||||
|
||||
@@ -82,6 +82,7 @@ class FakeRagLLM:
|
||||
stream_error: Exception | None = None,
|
||||
fail_mid_stream: bool = False,
|
||||
tool_script: list[list[StreamPiece | ToolCallPiece]] | None = None,
|
||||
no_tools_script: list[list[StreamPiece | ToolCallPiece]] | None = None,
|
||||
embed_fail_count: int = 0,
|
||||
stream_fail_count: int = 0,
|
||||
answer_sequence: list[str] | None = None,
|
||||
@@ -123,6 +124,18 @@ class FakeRagLLM:
|
||||
#: thinking + answer stream below, so a deflected turn through this
|
||||
#: fake is byte-identical to the plain fake's output.
|
||||
self.tool_script: list[list[StreamPiece | ToolCallPiece]] = list(tool_script or [])
|
||||
#: Phase 160 (task 02): canned piece lists for requests made
|
||||
#: WITHOUT tools (``tools=None``) — the deflected turn, the
|
||||
#: cap-forced final, the kill-switch single-request path, and the
|
||||
#: phase-71 recovery. Empty (the default) keeps the pre-phase
|
||||
#: behavior: a tools=None request yields the thinking + answer
|
||||
#: stream below, so every existing test is untouched. When set,
|
||||
#: the *i*-th tools=None request consumes ``no_tools_script[i]``
|
||||
#: (so a mock can make a tools=None request return structured
|
||||
#: ``ToolCallPiece``s — the documented ``lite`` behavior).
|
||||
self.no_tools_script: list[list[StreamPiece | ToolCallPiece]] = list(
|
||||
no_tools_script or []
|
||||
)
|
||||
|
||||
async def embed(
|
||||
self,
|
||||
@@ -197,6 +210,8 @@ class FakeRagLLM:
|
||||
raw: list[StreamPiece | ToolCallPiece]
|
||||
if tools is not None and self.tool_script:
|
||||
raw = self.tool_script.pop(0)
|
||||
elif tools is None and self.no_tools_script:
|
||||
raw = self.no_tools_script.pop(0)
|
||||
elif self.fail_mid_stream:
|
||||
raw = [StreamPiece("content", "partial ")]
|
||||
mid_stream_drop = True
|
||||
@@ -980,9 +995,10 @@ marker in the model's context."""
|
||||
m for r in scripted.seen_messages for m in r if m.get("role") == "tool"
|
||||
]
|
||||
assert tool_msgs, "the executed read must be appended as a tool message"
|
||||
assert (
|
||||
tool_msgs[-1]["content"]
|
||||
== "Document docs/fits.md:\ndate: 2024-06-15\n" + content
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the
|
||||
# read result — the raw result is the RAW part.
|
||||
assert tool_msgs[-1]["content"].startswith(
|
||||
"Document docs/fits.md:\ndate: 2024-06-15\n" + content
|
||||
)
|
||||
assert TRUNCATION_MARKER not in tool_msgs[-1]["content"]
|
||||
# Still a successful read.
|
||||
@@ -1165,9 +1181,10 @@ def test_untruncated_read_streams_no_tool_result_frame(
|
||||
m for r in scripted.seen_messages for m in r if m.get("role") == "tool"
|
||||
]
|
||||
assert tool_msgs
|
||||
assert (
|
||||
tool_msgs[-1]["content"]
|
||||
== "Document docs/big-read.md:\ndate: 2024-06-15\n" + content
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the
|
||||
# read result — the raw result is the RAW part.
|
||||
assert tool_msgs[-1]["content"].startswith(
|
||||
"Document docs/big-read.md:\ndate: 2024-06-15\n" + content
|
||||
)
|
||||
assert TRUNCATION_MARKER not in tool_msgs[-1]["content"]
|
||||
|
||||
@@ -1500,6 +1517,176 @@ def test_zero_max_rounds_reproduce_pre_phase_single_request(
|
||||
assert "scaffold_stripped=0" in lines[-1] # phase 71: uniform clean-turn field
|
||||
|
||||
|
||||
def test_cap_path_rejects_late_calls_and_fallbacks(
|
||||
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""A grounded turn with ``agent_max_rounds=1`` whose forced final
|
||||
(``tools=None``) returns a ``ToolCallPiece`` (the documented
|
||||
``lite`` behavior — a no-tools request that still comes back with
|
||||
structured calls): the executed round streams ``tool`` ×1, then
|
||||
NOTHING of the sort — no second ``tool`` frame, no ``error`` frame —
|
||||
and the turn settles with ``done`` whose delta text is the
|
||||
``ROUND_CAP_FALLBACK_ANSWER`` (the model gave no visible content on
|
||||
the forced final). The late call is never executed and never counted
|
||||
in ``tool_calls`` (the executed-only log field stays 1)."""
|
||||
scripted = FakeRagLLM(
|
||||
tool_script=[[ToolCallPiece(id="call_1", name="ls", arguments={})]],
|
||||
no_tools_script=[
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="call_f1",
|
||||
name="read",
|
||||
arguments={"path": "docs/homelab/kubernetes.md"},
|
||||
)
|
||||
]
|
||||
],
|
||||
)
|
||||
live = get_settings()
|
||||
monkeypatch.setattr(
|
||||
chat_api,
|
||||
"get_settings",
|
||||
lambda: Settings(
|
||||
_env_file=None, # pyright: ignore[reportCallIssue]
|
||||
relevance_threshold=live.relevance_threshold,
|
||||
agent_max_rounds=1,
|
||||
),
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
types = [f["type"] for f in frames]
|
||||
# Only the executed round 1 call streams as a tool frame; the forced
|
||||
# final's call is rejected — no second tool frame, no error frame.
|
||||
assert types.count("tool") == 1
|
||||
assert "error" not in types
|
||||
# The turn settles with done; the answer is the budget fallback.
|
||||
done = frames[-1]
|
||||
assert done["type"] == "done"
|
||||
assert done["deflected"] is False
|
||||
delta_text = "".join(f.get("text", "") for f in frames if f["type"] == "delta")
|
||||
assert delta_text == agent.ROUND_CAP_FALLBACK_ANSWER
|
||||
# The late ``read`` was never executed: ``tool_calls=1`` (the
|
||||
# executed round only) in the per-turn log line.
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "tool_calls=1" in lines[-1]
|
||||
|
||||
|
||||
def test_cap_reached_turn_reports_budget_exhausted_and_log_fields(
|
||||
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""A grounded turn with ``agent_max_rounds=1`` that hits the cap:
|
||||
the ``done`` frame carries ``"budget_exhausted": true`` (the
|
||||
additive key — the ``data:`` line the client sees) and the
|
||||
per-turn log line reads ``rounds=1 cap_hit=1`` (the cap hit,
|
||||
named — the owner can grep how often the cap bites)."""
|
||||
scripted = FakeRagLLM(
|
||||
tool_script=[[ToolCallPiece(id="call_1", name="ls", arguments={})]],
|
||||
no_tools_script=[[StreamPiece("content", "the wrap-up answer")]],
|
||||
)
|
||||
live = get_settings()
|
||||
monkeypatch.setattr(
|
||||
chat_api,
|
||||
"get_settings",
|
||||
lambda: Settings(
|
||||
_env_file=None, # pyright: ignore[reportCallIssue]
|
||||
relevance_threshold=live.relevance_threshold,
|
||||
agent_max_rounds=1,
|
||||
),
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
done = frames[-1]
|
||||
assert done["type"] == "done"
|
||||
assert done["deflected"] is False
|
||||
assert done["budget_exhausted"] is True # the cap-reached flag
|
||||
assert "error" not in [f["type"] for f in frames]
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "rounds=1 cap_hit=1" in lines[-1]
|
||||
assert "tool_calls=1" in lines[-1]
|
||||
|
||||
|
||||
def test_in_cap_and_deflected_turns_report_no_budget_exhausted(
|
||||
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""An ordinary grounded turn (answered in-cap) and a deflected
|
||||
turn: the ``done`` frames carry NO ``budget_exhausted`` key
|
||||
(ABSENT, never ``false`` — the byte-identical-off contract), and
|
||||
the per-turn log lines read ``cap_hit=0`` — the grounded one
|
||||
with ``rounds=`` the executed rounds (2), the deflected one
|
||||
``rounds=0 cap_hit=0`` (the agent never runs)."""
|
||||
scripted = FakeRagLLM(
|
||||
tool_script=[
|
||||
[ToolCallPiece(id="call_1", name="ls", arguments={})],
|
||||
[ToolCallPiece(id="call_2", name="ls", arguments={})],
|
||||
]
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
_, _, deflected_frames = _stream_chat(client, OFF_TOPIC)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
done = frames[-1]
|
||||
assert done["type"] == "done"
|
||||
assert done["deflected"] is False
|
||||
assert "budget_exhausted" not in done # ABSENT, never false
|
||||
d_done = deflected_frames[-1]
|
||||
assert d_done["type"] == "done"
|
||||
assert d_done["deflected"] is True
|
||||
assert "budget_exhausted" not in d_done
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert len(lines) == 2
|
||||
assert "tool_calls=2 rounds=2 cap_hit=0" in lines[0] # the grounded turn
|
||||
assert "rounds=0 cap_hit=0" in lines[1] # the deflected turn
|
||||
|
||||
|
||||
def test_deflected_turn_rejects_structured_calls(
|
||||
client, db, seeded_kb: FakeRagLLM
|
||||
) -> None:
|
||||
"""A deflected turn (``tools=None``, A8) whose model returns a
|
||||
structured ``tool_calls`` (the documented ``lite`` behavior) emits
|
||||
NO ``tool`` frame — the call is rejected at the transport
|
||||
(``_pump`` drops it, ``tools_offered=False``), never rendered, never
|
||||
executed. The turn still settles (``done``) with the model's
|
||||
content (the rejection only drops the call, not the answer)."""
|
||||
scripted = FakeRagLLM(
|
||||
no_tools_script=[
|
||||
[
|
||||
ToolCallPiece(id="call_1", name="ls", arguments={}),
|
||||
StreamPiece("content", "the deflected answer"),
|
||||
]
|
||||
],
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
_, _, frames = _stream_chat(client, OFF_TOPIC)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
types = [f["type"] for f in frames]
|
||||
assert not any(t == "tool" for t in types) # no tool frame
|
||||
assert "error" not in types
|
||||
done = frames[-1]
|
||||
assert done["type"] == "done"
|
||||
assert done["deflected"] is True
|
||||
# The model's content still streamed (the rejection only drops the
|
||||
# call, not the answer).
|
||||
delta_text = "".join(f.get("text", "") for f in frames if f["type"] == "delta")
|
||||
assert delta_text == "the deflected answer"
|
||||
# One request, no tools key (the deflected direct path, A8).
|
||||
assert len(scripted.seen_messages) == 1
|
||||
assert scripted.seen_tools == [None]
|
||||
|
||||
|
||||
def test_tool_execution_db_failure_yields_error_event(
|
||||
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
|
||||
@@ -232,6 +232,12 @@ def _expect(records: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
"stopped": m.get("stopped"),
|
||||
"failed": m.get("failed"),
|
||||
"error": m.get("error"),
|
||||
# Phase 160 (task 04): the budget-exhausted marker —
|
||||
# explicit null where the record does not carry it (the
|
||||
# bor.chat.v1 shape is explicit-null for every optional
|
||||
# key; a record WITH the key round-trips it byte-
|
||||
# identically — the pins below).
|
||||
"budget_exhausted": m.get("budget_exhausted"),
|
||||
}
|
||||
for m in records
|
||||
]
|
||||
@@ -464,8 +470,10 @@ def test_create_round_trips_full_brain_record(admin_client: TestClient) -> None:
|
||||
assert r.status_code == 201
|
||||
# The bor.chat.v1-shaped payload round-trips losslessly: every
|
||||
# optional key (sources/related/deflected/suggestions/thinking/
|
||||
# tools/stopped) survives identical.
|
||||
assert r.json()["messages"][1] == FULL_BRAIN
|
||||
# tools/stopped/failed/error/budget_exhausted — the phase-160
|
||||
# additive marker, explicit null where absent) survives identical
|
||||
# (the _expect full stored shape).
|
||||
assert r.json()["messages"][1] == _expect([_user(FIRST_QUESTION), FULL_BRAIN])[1]
|
||||
|
||||
|
||||
def test_create_rejects_empty_messages(admin_client: TestClient) -> None:
|
||||
@@ -1036,6 +1044,128 @@ def test_shared_serve_includes_the_brain_answer_image_ref(
|
||||
assert "answer_image" not in messages[0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 160 (task 04, TODO L4): the budget-exhausted marker — the
|
||||
# persisted answer that ran out of tool calls. The done-time
|
||||
# auto-save (``rememberBrainTurn``) persists the done frame's
|
||||
# ``budget_exhausted`` into the brain record's ``budget_exhausted``
|
||||
# key (the ``answer_image`` precedent — the field must exist so the
|
||||
# ``extra="forbid"`` boundary does NOT 422 those saves; the
|
||||
# phase-113 lesson, applied up front), and the restore path re-
|
||||
# renders the "Continue researching" button from it (task 04 — the
|
||||
# marker is a client restore hint only, never a server behavior; no
|
||||
# migration: ``saved_chats.messages`` is raw JSONB, phase 50, and
|
||||
# ``app/api/chats.py``'s logic is unchanged). These round-trips are
|
||||
# that regression pin.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_create_round_trips_the_budget_exhausted_marker(
|
||||
admin_client: TestClient,
|
||||
) -> None:
|
||||
"""The phase-113 regression pin (the A2 quiet-failure path): a
|
||||
brain record carrying ``budget_exhausted: true`` →
|
||||
``POST /api/chats`` is 201 (NOT a silent 422) and the admin GET
|
||||
round-trips the record BYTE-FOR-BYTE through Postgres (the
|
||||
``_expect`` full bor.chat.v1 shape — the restore path re-renders
|
||||
the Continue button from the marker). The user record carries the
|
||||
key as an explicit null (it never did, and the stored shape is
|
||||
explicit-null for every optional key)."""
|
||||
brain = {**FULL_BRAIN, "budget_exhausted": True}
|
||||
r = admin_client.post(
|
||||
"/api/chats",
|
||||
json={"messages": [_user(FIRST_QUESTION), brain]},
|
||||
)
|
||||
assert r.status_code == 201, f"expected 201, got {r.status_code}: {r.text}"
|
||||
body = r.json()
|
||||
assert body["messages"][1]["budget_exhausted"] is True
|
||||
assert body["messages"][0]["budget_exhausted"] is None
|
||||
|
||||
got = admin_client.get(f"/api/chats/{body['id']}")
|
||||
assert got.status_code == 200
|
||||
assert got.json()["messages"] == _expect(
|
||||
[_user(FIRST_QUESTION), {**FULL_BRAIN, "budget_exhausted": True}]
|
||||
)
|
||||
|
||||
|
||||
def test_create_without_budget_exhausted_marker_round_trips_unchanged(
|
||||
admin_client: TestClient,
|
||||
) -> None:
|
||||
"""A record WITHOUT the marker (every non-cap turn, every pre-
|
||||
phase record) round-trips with the key as an explicit null — the
|
||||
stored shape never gains a value (the byte-identical contract,
|
||||
the phase-50 precedent: the key is absent from the client's
|
||||
``bor.chat.v1`` record via the ``undefined``-drops-the-key
|
||||
pattern, and explicit-null here keeps the restore path null-safe).
|
||||
The ``FULL_BRAIN`` shape (no marker) is the pin: its stored and
|
||||
served form is byte-identical to pre-phase-160 apart from the
|
||||
one new null key."""
|
||||
r = admin_client.post(
|
||||
"/api/chats",
|
||||
json={"messages": [_user(FIRST_QUESTION), FULL_BRAIN]},
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
body = r.json()
|
||||
assert body["messages"][1]["budget_exhausted"] is None
|
||||
got = admin_client.get(f"/api/chats/{body['id']}")
|
||||
assert got.status_code == 200
|
||||
assert got.json()["messages"] == _expect([_user(FIRST_QUESTION), FULL_BRAIN])
|
||||
|
||||
|
||||
def test_shared_serve_includes_the_budget_exhausted_marker(
|
||||
admin_client: TestClient,
|
||||
) -> None:
|
||||
"""The shared view is faithful (the data is in the public
|
||||
snapshot), but the control is NOT: the shared page
|
||||
(``/shared/<token>``, rendered server-side by ``shared.js`` —
|
||||
never by ``app.js``, owner-locked zero interactive controls)
|
||||
IGNORES the key by construction — no Continue control renders for
|
||||
a flagged record (work item 8; pinned in
|
||||
``test_frontend_tool_budget.py::test_shared_page_never_shows_
|
||||
the_continue_control``). The user record stays with the explicit
|
||||
null."""
|
||||
brain = {**FULL_BRAIN, "budget_exhausted": True}
|
||||
r = admin_client.post(
|
||||
"/api/chats",
|
||||
json={"messages": [_user(FIRST_QUESTION), brain]},
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
share_url = _share(admin_client, r.json()["id"])["share_url"]
|
||||
|
||||
anon = TestClient(fastapi_app) # fresh jar: truly anonymous
|
||||
got = anon.get(f"/api{share_url}")
|
||||
assert got.status_code == 200
|
||||
messages = got.json()["messages"]
|
||||
assert messages[1]["budget_exhausted"] is True # faithful snapshot
|
||||
assert messages[0]["budget_exhausted"] is None
|
||||
|
||||
|
||||
def test_put_round_trips_the_budget_exhausted_marker(
|
||||
admin_client: TestClient,
|
||||
) -> None:
|
||||
"""The re-Save upsert (the auto-save path the client's restore
|
||||
relies on) round-trips the marker byte-identically too: a PUT
|
||||
with the flagged record replaces the stored JSONB and the 200
|
||||
body + the GET both carry it (a refreshed chat page restores the
|
||||
Continue affordance from the persisted record)."""
|
||||
created = admin_client.post(
|
||||
"/api/chats",
|
||||
json={"messages": [_user(FIRST_QUESTION), FULL_BRAIN]},
|
||||
).json()
|
||||
brain = {**FULL_BRAIN, "budget_exhausted": True}
|
||||
r = admin_client.put(
|
||||
f"/api/chats/{created['id']}",
|
||||
json={"messages": [_user(FIRST_QUESTION), brain]},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["messages"][1]["budget_exhausted"] is True
|
||||
got = admin_client.get(f"/api/chats/{created['id']}")
|
||||
assert got.status_code == 200
|
||||
assert got.json()["messages"] == _expect(
|
||||
[_user(FIRST_QUESTION), {**FULL_BRAIN, "budget_exhausted": True}]
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 159 (task 02): edit-and-resend — the SAVE shape. The client
|
||||
# edits the last question: the record is replaced in place, EVERY
|
||||
|
||||
@@ -443,7 +443,10 @@ def test_plain_read_truncates_then_range_read_answers_the_find_traps_question(
|
||||
assert isinstance(plain_result["content"], str)
|
||||
notice = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=len(doc.content))
|
||||
assert plain_result["content"].startswith(f"Document {IDENTITY}:\n")
|
||||
assert plain_result["content"].endswith(f"\n{TRUNCATION_MARKER}\n{notice}")
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the
|
||||
# truncated result — the marker + notice end the RAW part.
|
||||
raw = plain_result["content"].removesuffix(agent._budget_note(9))
|
||||
assert raw.endswith(f"\n{TRUNCATION_MARKER}\n{notice}")
|
||||
# The truncation surfaced to the API layer as a ToolResultPiece
|
||||
# (the SSE ``tool_result`` frame's wire value).
|
||||
truncations = [p for p in pieces if isinstance(p, ToolResultPiece)]
|
||||
|
||||
@@ -411,7 +411,7 @@ def test_two_ranges_of_one_doc_both_reach_the_model(two_range_doc: Document) ->
|
||||
# the byte-identical line is now true).
|
||||
reread_result = llm.requests[3][0][7]
|
||||
assert reread_result["role"] == "tool"
|
||||
assert reread_result["content"] == agent.ALREADY_IN_CONTEXT
|
||||
assert reread_result["content"].startswith(agent.ALREADY_IN_CONTEXT)
|
||||
|
||||
# Round 4: the answer echoes BOTH markers (the bug: pre-phase,
|
||||
# the echo could contain only marker A — range B's slice never
|
||||
@@ -485,7 +485,10 @@ def test_truncated_plain_read_then_range_b_proceeds_and_plain_reread_teaches(
|
||||
assert isinstance(plain_result["content"], str)
|
||||
notice = READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))
|
||||
assert plain_result["content"].startswith(f"Document {IDENTITY}:\n")
|
||||
assert plain_result["content"].endswith(f"\n{TRUNCATION_MARKER}\n{notice}")
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the
|
||||
# truncated result — the marker + notice end the RAW part.
|
||||
raw = plain_result["content"].removesuffix(agent._budget_note(9))
|
||||
assert raw.endswith(f"\n{TRUNCATION_MARKER}\n{notice}")
|
||||
assert MARKER_A in plain_result["content"]
|
||||
truncations = [p for p in pieces if isinstance(p, ToolResultPiece)]
|
||||
assert [(p.argument, p.chars_shown, p.chars_total) for p in truncations] == [
|
||||
@@ -512,12 +515,12 @@ def test_truncated_plain_read_then_range_b_proceeds_and_plain_reread_teaches(
|
||||
# resolved doc.
|
||||
reread_result = llm.requests[3][0][7]
|
||||
assert reread_result["role"] == "tool"
|
||||
assert reread_result["content"] == agent.REREAD_PARTIAL.format(
|
||||
assert reread_result["content"].startswith(agent.REREAD_PARTIAL.format(
|
||||
covered=f"1–{covered}, {marker_b_line - 10}–{marker_b_line - 10 + RANGE_B_LIMIT - 1}",
|
||||
source=SOURCE,
|
||||
path=DOC_PATH,
|
||||
total=total_lines,
|
||||
)
|
||||
))
|
||||
|
||||
# Round 4: the answer echoes both markers.
|
||||
answer = _answer(pieces)
|
||||
|
||||
+572
-235
File diff suppressed because it is too large
Load Diff
@@ -1578,6 +1578,8 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
|
||||
"sources=",
|
||||
"thinking_chars=",
|
||||
"tool_calls=",
|
||||
"rounds=",
|
||||
"cap_hit=",
|
||||
"total_ms=",
|
||||
"retries=",
|
||||
"scaffold_stripped=",
|
||||
|
||||
@@ -491,7 +491,9 @@ def test_chat_message_image_defaults_to_none_and_is_omitted() -> None:
|
||||
assert m.image is None
|
||||
dumped = m.model_dump()
|
||||
assert "image" not in dumped # ABSENT, never ``null`` (the phase-50 contract)
|
||||
# and every pre-phase key set is untouched
|
||||
# and every pre-phase key set is untouched (the phase-160 additive
|
||||
# ``budget_exhausted`` marker joins as an explicit null — the
|
||||
# phase-120 ``failed`` / phase-48 ``stopped`` style).
|
||||
assert set(dumped) == {
|
||||
"who",
|
||||
"text",
|
||||
@@ -504,6 +506,7 @@ def test_chat_message_image_defaults_to_none_and_is_omitted() -> None:
|
||||
"stopped",
|
||||
"failed",
|
||||
"error",
|
||||
"budget_exhausted",
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -173,10 +173,14 @@ def test_pre_phase_record_round_trips_byte_identical_on_existing_keys() -> None:
|
||||
# Every pre-phase key survives byte-identical…
|
||||
for key, value in pre_phase.items():
|
||||
assert dumped[key] == value, f"pre-phase key {key!r} changed: {dumped[key]!r}"
|
||||
# …and the ONLY additions are the two new keys as explicit nulls.
|
||||
assert set(dumped) == set(pre_phase) | {"failed", "error"}
|
||||
# …and the ONLY additions are the new keys as explicit nulls
|
||||
# (the phase-120/48 ``failed`` / ``error`` pair, and the phase-160
|
||||
# additive ``budget_exhausted`` marker — the client's restore
|
||||
# hint, explicit null where absent).
|
||||
assert set(dumped) == set(pre_phase) | {"failed", "error", "budget_exhausted"}
|
||||
assert dumped["failed"] is None
|
||||
assert dumped["error"] is None
|
||||
assert dumped["budget_exhausted"] is None
|
||||
# The stored shape re-validates (round-trip through the DB JSONB).
|
||||
assert ChatMessage.model_validate(dumped).model_dump() == dumped
|
||||
|
||||
@@ -197,4 +201,5 @@ def test_minimal_user_record_unchanged() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
}
|
||||
|
||||
@@ -796,9 +796,15 @@ def test_retry_last_turn_is_byte_unchanged() -> None:
|
||||
#: task-01 idle-settle ``markLastEditable()`` mark pass (work item 3
|
||||
#: call sites — the in-flight guard's at-rest landing), and the
|
||||
#: task-03 aborted-settle mark pair (the 300s guard's ``aborted``
|
||||
#: catch branch — a timed-out turn must not strand the question).
|
||||
#: The ``reask`` path is consumed as-is. A diff here is a contract
|
||||
#: violation for the phase's remaining tasks.
|
||||
#: catch branch — a timed-out turn must not strand the question);
|
||||
#: plus the phase-160 (task 04, TODO L4) sanctioned additions:
|
||||
#: the done settle's ``budget_exhausted`` meta key (the additive
|
||||
#: done-frame marker — ``undefined`` drops the key, so every non-cap
|
||||
#: record stays byte-identical) and the flag-gated
|
||||
#: ``appendContinueButton`` (the "Continue researching" affordance —
|
||||
#: ABSENT on every other turn). The ``reask`` path is consumed as-is.
|
||||
#: A diff here is a contract violation for the phase's remaining
|
||||
#: tasks.
|
||||
PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}) {
|
||||
if (!reask) {
|
||||
// Phase 123 (task 02, TODO L6): the attached image rides the user
|
||||
@@ -1181,6 +1187,12 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
// row exactly as it looked live.
|
||||
related: ev.related?.length ? ev.related : undefined,
|
||||
suggestions: ev.suggestions,
|
||||
// Phase 160 (task 04, TODO L4): the cap-reached marker
|
||||
// persists with the turn — `undefined` drops the key from
|
||||
// the JSON (the house optional-meta pattern, the
|
||||
// answer_image line above is the precedent), so every
|
||||
// non-cap record stays byte-identical to pre-phase.
|
||||
budget_exhausted: ev.budget_exhausted || undefined,
|
||||
}, leavePartialIndex);
|
||||
lastBrainWrap = wrap; // this bubble is now the last brain answer
|
||||
// Phase 130 (task 02): the row order is the DOM order — the
|
||||
@@ -1191,6 +1203,20 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
appendTuneButton(wrap); // every completed brain bubble is tunable
|
||||
markLastRetryable(); // phase 49: the Retry button is last-bubble-only
|
||||
markLastEditable(); // phase 159: the done turn's question is editable (the in-flight guard no-ops here — the runTurn idle settle lands it)
|
||||
// Phase 160 (task 04, TODO L4): a cap-reached answer offers
|
||||
// the one-click continuation — first, every earlier bubble's
|
||||
// continue button is cleared (the markLastRetryable
|
||||
// remove-first pattern: the moment a newer turn lands, an old
|
||||
// bubble's stale affordance is gone — its "continue" would
|
||||
// append to a history the conversation already moved past),
|
||||
// then the button is appended ONLY when the done frame is
|
||||
// flagged (ABSENT on every other turn, so the button never
|
||||
// appears elsewhere); the row order stays Tune → Retry →
|
||||
// Continue → Save-as-doc (the insertion order phase 130
|
||||
// relies on — markLastEditable above acts on the user
|
||||
// bubble, so the brain bubble's row order is unchanged).
|
||||
messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());
|
||||
if (ev.budget_exhausted) appendContinueButton(wrap);
|
||||
// Phase 59: the RAW persisted markdown (never the rendered
|
||||
// HTML) — exactly the string rememberBrainTurn stores above,
|
||||
// so a reload (the restore path) offers the identical draft.
|
||||
@@ -1397,13 +1423,16 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
|
||||
|
||||
def test_run_turn_is_pinned_at_phase_159_state() -> None:
|
||||
"""``runTurn`` is pinned at exactly the state phase 159 left it
|
||||
(task 02 consumes its ``reask`` path AS-IS — the phase's "NOT
|
||||
touched" contract): the full source must match the pin — the
|
||||
``reask`` path, the turn-local resets, the SSE contract, the
|
||||
save points, and the sanctioned additions (the task-01
|
||||
``lastUserWrap`` assignment + the idle-settle mark pass, and the
|
||||
task-03 aborted-settle mark pair) included."""
|
||||
"""``runTurn`` is pinned at exactly the state phase 159 left it,
|
||||
plus phase 160 (task 04)'s sanctioned additions (task 02 consumes
|
||||
its ``reask`` path AS-IS — the phase's "NOT touched" contract;
|
||||
the phase-160 additions are the done settle's additive
|
||||
``budget_exhausted`` meta key + the flag-gated Continue button):
|
||||
the full source must match the pin — the ``reask`` path, the
|
||||
turn-local resets, the SSE contract, the save points, and the
|
||||
sanctioned additions (the task-01 ``lastUserWrap`` assignment +
|
||||
the idle-settle mark pass, the task-03 aborted-settle mark pair,
|
||||
and the phase-160 done-settle pair) included."""
|
||||
assert _fn_source(_js(), "runTurn") == PINNED_RUN_TURN
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,546 @@
|
||||
"""Unit: the phase-160 (task 04, TODO L4) "Continue researching" button.
|
||||
|
||||
When a turn ends because its tool budget was spent (the ``done``
|
||||
frame's additive ``budget_exhausted`` key, task 03 — ABSENT on every
|
||||
non-cap turn), the answer bubble offers **Continue researching**: one
|
||||
click sends the next turn (a plain send — the budget is per-turn, so
|
||||
the fresh turn has a whole new round budget) and the marker persists
|
||||
with the record so a page refresh restores the affordance. The shared
|
||||
page never shows the button (read-only by construction).
|
||||
|
||||
Pins (the house source-pin pattern — no browser):
|
||||
|
||||
* ``CONTINUE_ICON`` / ``CONTINUE_QUESTION`` — the inline aria-hidden
|
||||
SVG in the RETRY_ICON stroke family (no icon font, no CDN — A11)
|
||||
and the literal continuation word (what the model is taught to ask
|
||||
the user for);
|
||||
* ``appendContinueButton`` — the house pattern through
|
||||
``getActionsRow`` (the role=list row joins the button as a
|
||||
listitem), the one-per-bubble guard, click → ``continueTurn()``;
|
||||
* ``continueTurn`` — the ``retryLastTurn`` discipline: no-op while a
|
||||
turn is in flight, every ``.continue-btn`` removed FIRST, then a
|
||||
PLAIN ``runTurn(CONTINUE_QUESTION)`` (no ``reask``);
|
||||
* the ``done`` settle — ``budget_exhausted`` rides
|
||||
``rememberBrainTurn``'s meta (``undefined`` drops the key, so every
|
||||
non-cap record stays byte-identical) and the button is appended
|
||||
ONLY when the frame is flagged, in the row order Tune → Retry →
|
||||
Continue → Save-as-doc (the insertion order phase 130 relies on);
|
||||
* the restore branch — keys on ``m.budget_exhausted``;
|
||||
* ``startNewChat`` needs nothing — its list reset drops the buttons;
|
||||
* ``retryLastTurn`` / ``markLastRetryable`` / ``rememberBrainTurn``
|
||||
are byte-unchanged (the phase-120 shared pin for the first);
|
||||
* the shared page (work item 8) — ``shared.html`` never loads
|
||||
``app.js`` and ``shared.js`` builds no buttons / actions rows, so a
|
||||
flagged record renders without any Continue control there.
|
||||
|
||||
The browser-level assertions (the live click running a fresh turn, the
|
||||
reload restoring the button, the shared page's read-only render) are
|
||||
E2E-gated by tests/e2e/test_tool_budget_telemetry.py (task 05).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from tests.unit.test_frontend_failed_turn import PINNED_RETRY_LAST_TURN
|
||||
|
||||
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||||
APP_JS = FRONTEND / "assets" / "app.js"
|
||||
STYLES_CSS = FRONTEND / "assets" / "styles.css"
|
||||
SHARED_HTML = FRONTEND / "shared.html"
|
||||
SHARED_JS = FRONTEND / "assets" / "shared.js"
|
||||
|
||||
|
||||
def _js() -> str:
|
||||
return APP_JS.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _css() -> str:
|
||||
return STYLES_CSS.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _fn_body(js: str, header: str) -> str:
|
||||
"""The full source of the function whose header is *header* —
|
||||
brace-matched from the first ``{`` OUTSIDE the parameter list
|
||||
(paren depth 0, the house convention), so destructured parameters
|
||||
are skipped (the pinned functions carry no string literal with a
|
||||
stray brace)."""
|
||||
start = js.index(header)
|
||||
i = start
|
||||
paren = 0
|
||||
while i < len(js):
|
||||
c = js[i]
|
||||
if c == "(":
|
||||
paren += 1
|
||||
elif c == ")":
|
||||
paren -= 1
|
||||
elif c == "{" and paren == 0:
|
||||
break
|
||||
i += 1
|
||||
depth = 0
|
||||
for j in range(i, len(js)):
|
||||
c = js[j]
|
||||
if c == "{":
|
||||
depth += 1
|
||||
elif c == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return js[start : j + 1]
|
||||
raise AssertionError(f"unbalanced braces in {header!r}")
|
||||
|
||||
|
||||
def _done_branch(js: str) -> str:
|
||||
"""The SSE ``done`` branch — from the branch test to the next
|
||||
``else if`` (flat block, the slice is exact)."""
|
||||
start = js.index('ev.type === "done"')
|
||||
return js[start : js.index('ev.type === "tool_result"', start)]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the glyph + the continuation word
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_continue_icon_is_an_inline_svg_in_the_stroke_family() -> None:
|
||||
"""``CONTINUE_ICON`` is the inline ``aria-hidden`` SVG in the
|
||||
RETRY_ICON stroke family (task 01: 24×24, ``fill="none"``,
|
||||
``stroke="currentColor"``, ``stroke-width="1.7"`` — the CSS themes
|
||||
the stroke) — no icon font, no CDN (A11)."""
|
||||
js = _js()
|
||||
start = js.index("const CONTINUE_ICON =")
|
||||
seg = js[start :]
|
||||
q1 = seg.index("'")
|
||||
q2 = seg.index("'", q1 + 1)
|
||||
literal = seg[q1 + 1 : q2] # the first string literal = the inline SVG
|
||||
for attr in (
|
||||
'aria-hidden="true"',
|
||||
'viewBox="0 0 24 24"',
|
||||
'fill="none"',
|
||||
'stroke="currentColor"',
|
||||
'stroke-width="1.7"',
|
||||
):
|
||||
assert attr in literal, f"the glyph keeps the shared family: {attr!r}"
|
||||
assert literal.startswith("<svg") and literal.endswith("</svg>")
|
||||
|
||||
|
||||
def test_continue_question_is_the_literal_word_continue() -> None:
|
||||
"""``CONTINUE_QUESTION`` is the literal word ``continue`` — the
|
||||
word the model is taught to ask the user for (task 02's
|
||||
ROUND_CAP_WRAPUP_INSTRUCTION / the spent-budget trailer), so the
|
||||
visible bubble, the typed equivalent, and the model's own
|
||||
suggestion all agree."""
|
||||
js = _js()
|
||||
assert 'const CONTINUE_QUESTION = "continue";' in js
|
||||
|
||||
|
||||
def test_continue_icon_is_defined() -> None:
|
||||
"""EXACTLY ONE ``CONTINUE_ICON`` constant (the glyph is shared by
|
||||
the button, never duplicated)."""
|
||||
js = _js()
|
||||
assert js.count("const CONTINUE_ICON =") == 1
|
||||
assert js.count("const CONTINUE_QUESTION =") == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# appendContinueButton — the house pattern
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_append_continue_button_joins_the_actions_row() -> None:
|
||||
"""``appendContinueButton(wrap)`` keeps the house pattern (the
|
||||
appendRetryButton shape, task 02): the getActionsRow lookup (the
|
||||
dedicated actions row, NEVER the chips row), the null no-op, the
|
||||
one-per-bubble guard (``.continue-btn``), the role=list →
|
||||
listitem ARIA join, the icon + the visible "Continue researching"
|
||||
label (the label carries the accessible name), and the click →
|
||||
``continueTurn()`` binding."""
|
||||
js = _js()
|
||||
body = _fn_body(js, "function appendContinueButton(wrap) {")
|
||||
assert "getActionsRow(wrap)" in body, "the shared get-or-create helper"
|
||||
assert "if (!meta) return" in body, "no-op without a .msg-body"
|
||||
assert 'querySelector(".continue-btn")' in body, (
|
||||
"the one-per-bubble guard"
|
||||
)
|
||||
assert 'meta.getAttribute("role") === "list"' in body, (
|
||||
"the role=list check (the row is always role=list)"
|
||||
)
|
||||
assert 'setAttribute("role", "listitem")' in body, "the listitem join"
|
||||
assert 'btn.className = "continue-btn"' in body
|
||||
assert 'btn.innerHTML = CONTINUE_ICON + "<span>Continue researching</span>"' in body, (
|
||||
"icon + the visible label (the accessible name)"
|
||||
)
|
||||
assert "btn.addEventListener(\"click\", () => continueTurn())" in body, (
|
||||
"the click handler — a plain continueTurn() (no wrap, no args)"
|
||||
)
|
||||
assert "meta.appendChild(btn)" in body
|
||||
# NOT admin-gated, like Retry (chat is public).
|
||||
assert "isAdmin" not in body
|
||||
|
||||
|
||||
def test_continue_button_lands_after_the_retry_button() -> None:
|
||||
"""``appendContinueButton`` is defined DIRECTLY AFTER
|
||||
``appendRetryButton`` (task 02's placement — the sibling in the
|
||||
actions-row family, before the Edit button's block)."""
|
||||
js = _js()
|
||||
retry = js.index("function appendRetryButton(wrap) {")
|
||||
cont = js.index("function appendContinueButton(wrap) {")
|
||||
edit = js.index("function appendEditButton(wrap) {")
|
||||
assert retry < cont < edit
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# continueTurn — the retryLastTurn discipline
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_continue_turn_bails_and_clears_and_sends_plain() -> None:
|
||||
"""``continueTurn()`` (task 03): (1) no-op while a turn is in
|
||||
flight (the ``retryLastTurn`` guard — one turn at a time, no
|
||||
double-send); (2) clears EVERY rendered ``.continue-btn`` FIRST
|
||||
(the markLastRetryable remove-first pattern — a stale button on an
|
||||
earlier bubble is gone the moment the new turn's answer lands);
|
||||
(3) a PLAIN send — ``runTurn(CONTINUE_QUESTION)`` with NO
|
||||
``reask`` argument (the default ``false``): the user bubble
|
||||
appears, the record is pushed + saved at the ordinary save
|
||||
points, and the request carries the full history."""
|
||||
js = _js()
|
||||
body = _fn_body(js, "function continueTurn() {")
|
||||
assert (
|
||||
"if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;"
|
||||
in body
|
||||
), "the in-flight no-op (the retryLastTurn guard)"
|
||||
assert (
|
||||
'messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());'
|
||||
in body
|
||||
), "every continue button is cleared before the send"
|
||||
assert "return runTurn(CONTINUE_QUESTION)" in body, (
|
||||
"a PLAIN send — the returned promise (the retryLastTurn "
|
||||
"convention)"
|
||||
)
|
||||
assert "reask" not in body, (
|
||||
"NO reask argument — a plain send (runTurn's default false)"
|
||||
)
|
||||
order = [
|
||||
body.index("UI_STATE.thinking"),
|
||||
body.index(".remove()"),
|
||||
body.index("runTurn(CONTINUE_QUESTION)"),
|
||||
]
|
||||
assert order == sorted(order), (
|
||||
"guard → clear-every-button → send, in that order"
|
||||
)
|
||||
|
||||
|
||||
def test_continue_turn_sends_one_plain_turn_with_full_history() -> None:
|
||||
"""The plain-send contract on the ``runTurn`` side (task 03's
|
||||
save-point list): with ``reask`` defaulting to false, the turn
|
||||
appends the user bubble, pushes ``{ who: "user", text }`` (no
|
||||
image — a text-only record), runs the ordinary save points
|
||||
(``saveConversation`` + ``persistConversation``), and the request's
|
||||
``history`` is the FULL ``conversation.slice(0, -1)`` — the model
|
||||
resumes with everything it already read."""
|
||||
js = _js()
|
||||
rt = _fn_body(js, "function runTurn(text, { reask = false, image = null } = {}) {")
|
||||
assert "if (!reask) {" in rt, "the default is a plain send"
|
||||
assert 'conversation.push(' in rt
|
||||
assert 'saveConversation();' in rt
|
||||
assert "persistConversation();" in rt
|
||||
assert "const history = conversation.slice(0, -1).map(" in rt, (
|
||||
"the request carries the full history (everything except the "
|
||||
"just-pushed question)"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the done settle — the flag rides the record + the button is gated
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_done_handler_persists_the_flag_and_appends_the_button() -> None:
|
||||
"""The ``done`` settle (task 04): ``budget_exhausted`` rides
|
||||
``rememberBrainTurn``'s meta as
|
||||
``ev.budget_exhausted || undefined`` (the ``answer_image``
|
||||
precedent — ``undefined`` drops the key from the JSON, so every
|
||||
non-cap record stays byte-identical), every earlier bubble's
|
||||
continue button is cleared FIRST (the completion criterion —
|
||||
"neither does any earlier answer once a newer turn lands" — the
|
||||
markLastRetryable remove-first pattern), and the button is
|
||||
appended ONLY when the frame is flagged."""
|
||||
js = _js()
|
||||
branch = _done_branch(js)
|
||||
assert "budget_exhausted: ev.budget_exhausted || undefined" in branch, (
|
||||
"the flag persists with the turn (undefined drops the key)"
|
||||
)
|
||||
assert (
|
||||
'messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());'
|
||||
in branch
|
||||
), "every earlier bubble's continue button is cleared on the settle"
|
||||
assert "if (ev.budget_exhausted) appendContinueButton(wrap);" in branch, (
|
||||
"the button is appended ONLY on a flagged done"
|
||||
)
|
||||
order = [
|
||||
branch.index('querySelectorAll(".continue-btn")'),
|
||||
branch.index("appendContinueButton(wrap)"),
|
||||
]
|
||||
assert order == sorted(order), "clear-every-button → gated append"
|
||||
|
||||
|
||||
def test_done_branch_row_order_is_tune_retry_continue_save() -> None:
|
||||
"""The row order on a capped answer (task 04): Tune → Retry (the
|
||||
markLastRetryable re-append) → Continue (flag-gated) → Save-as-doc
|
||||
— the INSERTION order phase 130 relies on (the DOM order inside
|
||||
the row is the visual order)."""
|
||||
js = _js()
|
||||
branch = _done_branch(js)
|
||||
order = [
|
||||
branch.index("appendTuneButton(wrap);"),
|
||||
branch.index("markLastRetryable();"),
|
||||
branch.index("appendContinueButton(wrap);"),
|
||||
branch.index('appendSaveAsDocButton(wrap, finalText || acc || "…");'),
|
||||
]
|
||||
assert order == sorted(order), (
|
||||
"done: Tune → Retry → Continue → Save-as-doc (the insertion "
|
||||
"order phase 130 relies on)"
|
||||
)
|
||||
|
||||
|
||||
def test_continue_button_is_not_added_on_the_other_settle_paths() -> None:
|
||||
"""The Continue button is a CAP-REACHED-DONE affordance only: the
|
||||
zero-frame fallback, the stop finalize, and the failed-turn funnel
|
||||
(none of them sees a ``budget_exhausted`` frame — a failed or
|
||||
stopped turn never hit the cap) append NO continue button, and the
|
||||
``done`` branch's flag-gate is the single gate."""
|
||||
js = _js()
|
||||
assert js.count("appendContinueButton(wrap);") == 2, (
|
||||
"exactly TWO call sites (with the terminating semicolon — the "
|
||||
"function header ends in a brace, not a semicolon): the done "
|
||||
"settle + the restore path"
|
||||
)
|
||||
zero = js.index("if (!aborted && !wrap) {")
|
||||
block = js[zero : js.index("} catch (err) {", zero)]
|
||||
assert "appendContinueButton" not in block, (
|
||||
"the zero-frame fallback (a failed turn) adds no continue button"
|
||||
)
|
||||
funnel = _fn_body(
|
||||
js,
|
||||
"function finalizeFailedTurn(\n"
|
||||
" detail,\n"
|
||||
" { acc, thinking, tools, segments, wrap, leavePartialIndex }\n"
|
||||
") {",
|
||||
)
|
||||
assert "appendContinueButton" not in funnel, (
|
||||
"the failed-turn funnel adds no continue button"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the restore path + startNewChat
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_restore_branch_keys_on_the_persisted_marker() -> None:
|
||||
"""The phase-14 restore (``renderStoredMessage``'s brain branch,
|
||||
task 05): the Continue affordance restores from the persisted
|
||||
``m.budget_exhausted`` marker — a reload keeps the button on a
|
||||
flagged answer (a restored ``history`` is exactly what the
|
||||
continuation turn needs), and a pre-phase / non-cap record (no
|
||||
key) restores byte-identically (no button). The append lands
|
||||
BEFORE the ``lastBrainWrap`` assignment, after the per-message
|
||||
appends (the actions row is the bubble's LAST row, as on the
|
||||
live done path)."""
|
||||
js = _js()
|
||||
body = _fn_body(js, "function renderStoredMessage(m) {")
|
||||
assert "if (m.budget_exhausted) appendContinueButton(wrap);" in body, (
|
||||
"the restore keys on the persisted marker"
|
||||
)
|
||||
order = [
|
||||
body.index("appendFailedNote(wrap, m.error);"),
|
||||
body.index("if (m.budget_exhausted) appendContinueButton(wrap);"),
|
||||
body.index("lastBrainWrap = wrap"),
|
||||
]
|
||||
assert order == sorted(order), (
|
||||
"restore: the per-message appends → the continue button → "
|
||||
"lastBrainWrap (the after-the-loop markLastRetryable lands the "
|
||||
"Retry in the same actions row, as on the live path)"
|
||||
)
|
||||
|
||||
|
||||
def test_start_new_chat_unchanged_drops_the_buttons_with_the_list() -> None:
|
||||
"""``startNewChat`` (task 05: needs nothing) — its list reset
|
||||
removes every ``.msg`` bubble, so the continue buttons die with
|
||||
the list (the phase-49 reasoning). The reset + the in-flight
|
||||
guard stay byte-pinned."""
|
||||
js = _js()
|
||||
body = _fn_body(js, "function startNewChat() {")
|
||||
assert (
|
||||
"if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;"
|
||||
in body
|
||||
), "the in-flight guard stays"
|
||||
assert 'messagesEl.querySelectorAll(".msg").forEach((el) => el.remove());' in body, (
|
||||
"the list reset drops the buttons along with the list"
|
||||
)
|
||||
assert "appendContinueButton" not in body, (
|
||||
"no continue-button wiring was added to startNewChat"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# NOT touched — the phase's explicit contract (byte-pinned sources)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The FULL source of ``markLastRetryable`` as of pre-phase-160 task
|
||||
#: 04 — the remove-then-append shape (remove every rendered
|
||||
#: .retry-btn FIRST, then re-append on the last brain bubble) is
|
||||
#: UNCHANGED: the continue button has its OWN clear pass in
|
||||
#: continueTurn, and markLastRetryable's targeting is untouched.
|
||||
PINNED_MARK_LAST_RETRYABLE = """function markLastRetryable() {
|
||||
messagesEl.querySelectorAll(".retry-btn").forEach((b) => b.remove());
|
||||
if (!lastBrainWrap) return;
|
||||
let lastIdx = -1;
|
||||
for (let i = conversation.length - 1; i >= 0; i -= 1) {
|
||||
if (conversation[i].who === "brain") {
|
||||
lastIdx = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const prev = lastIdx > 0 ? conversation[lastIdx - 1] : null;
|
||||
if (!prev || prev.who !== "user") return;
|
||||
appendRetryButton(lastBrainWrap);
|
||||
}"""
|
||||
|
||||
#: The FULL source of ``rememberBrainTurn`` as of pre-phase-160 task
|
||||
#: 04 — the ``...meta`` spread means the new ``budget_exhausted``
|
||||
#: optional key rides the record with ZERO change to this helper
|
||||
#: (``undefined`` drops the key from the JSON, the house
|
||||
#: optional-meta pattern).
|
||||
PINNED_REMEMBER_BRAIN_TURN = """function rememberBrainTurn(rawText, meta, replaceIndex = -1) {
|
||||
const rec = { who: "brain", text: rawText || "…", ...meta };
|
||||
if (replaceIndex >= 0 && conversation[replaceIndex]?.who === "brain") {
|
||||
conversation[replaceIndex] = rec;
|
||||
} else {
|
||||
conversation.push(rec);
|
||||
}
|
||||
saveConversation();
|
||||
// Phase 55 (A2): the auto-save rides the brain save point — the row
|
||||
// updates with the (possibly replaced) brain turn + metadata. The
|
||||
// pagehide partial reuses this helper, so it rides the same path (no
|
||||
// extra wiring).
|
||||
persistConversation();
|
||||
}"""
|
||||
|
||||
|
||||
def test_mark_last_retryable_is_byte_unchanged() -> None:
|
||||
"""``markLastRetryable`` is byte-unchanged by phase 160 (task 04
|
||||
— the redo-in-place targeting is REUSED, not extended): the full
|
||||
source must match the pin (a diff is a contract violation)."""
|
||||
assert _fn_body(_js(), "function markLastRetryable() {") == PINNED_MARK_LAST_RETRYABLE
|
||||
|
||||
|
||||
def test_remember_brain_turn_is_byte_unchanged() -> None:
|
||||
"""``rememberBrainTurn`` is byte-unchanged by phase 160 (task 04 —
|
||||
the ``budget_exhausted`` meta key rides the existing ``...meta``
|
||||
spread, so the save point is consumed as-is): the full source must
|
||||
match the pin."""
|
||||
assert _fn_body(_js(), "function rememberBrainTurn(rawText, meta, replaceIndex = -1) {") == (
|
||||
PINNED_REMEMBER_BRAIN_TURN
|
||||
)
|
||||
|
||||
|
||||
def test_retry_last_turn_is_byte_unchanged() -> None:
|
||||
"""``retryLastTurn`` is byte-unchanged by phase 160 (task 04 — the
|
||||
continue button REUSES its discipline, the function is consumed
|
||||
as-is): the full source must match the phase-120 pin (the shared
|
||||
constant — one pin, no drift)."""
|
||||
assert _fn_body(_js(), "function retryLastTurn(wrap) {") == PINNED_RETRY_LAST_TURN
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the shared page (work item 8) — the Continue control never renders there
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_shared_page_never_shows_the_continue_control() -> None:
|
||||
"""Work item 8 (no code change expected): the shared page
|
||||
(``/shared/<token>``, served server-side — ``shared.html``)
|
||||
renders the conversation through ``shared.js``'s OWN read-only
|
||||
renderer, never through ``app.js`` — so the ``budget_exhausted``
|
||||
key is IGNORED there by construction: the page loads no
|
||||
``app.js`` (no ``appendContinueButton`` in its script graph),
|
||||
and ``shared.js`` builds NO buttons, no forms, and no
|
||||
``.msg-actions`` row (owner-locked: zero interactive controls —
|
||||
the "no buttons, no forms, no links, no click handlers"
|
||||
contract), so a flagged record renders without any Continue
|
||||
affordance."""
|
||||
html = SHARED_HTML.read_text(encoding="utf-8")
|
||||
assert "app.js" not in html, (
|
||||
"the shared page never loads app.js — its action buttons "
|
||||
"(incl. Continue) are not in its script graph"
|
||||
)
|
||||
shared = SHARED_JS.read_text(encoding="utf-8")
|
||||
assert "continue-btn" not in shared
|
||||
assert "msg-actions" not in shared, (
|
||||
"shared.js never builds the actions row — zero controls"
|
||||
)
|
||||
assert 'createElement("button")' not in shared, (
|
||||
"owner-locked: NO buttons are ever created on the shared page"
|
||||
)
|
||||
assert 'addEventListener("click"' not in shared, (
|
||||
"owner-locked: NO click handlers on the shared page (the img "
|
||||
"load-failure degradation handler is not a control — the "
|
||||
"zero-interactive-controls contract)"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# styles.css — the .continue-btn family
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_continue_btn_css_is_the_retry_family() -> None:
|
||||
"""``.continue-btn`` is the exact visual family of ``.retry-btn``
|
||||
(task 07): the same pill size/spacing (``min-height: 44px``,
|
||||
``gap: 0.35rem``, ``padding: 0.35rem 0.8rem``,
|
||||
``border-radius: 999px``, ``border: 1px solid var(--line)``),
|
||||
the same neutral color (``--ink-soft`` → ``--ink`` on hover over
|
||||
``--brand-soft``), the 14px currentColor glyph, and the shared
|
||||
GLOBAL ``:focus-visible`` ring (styles.css L162 — the ring rule is
|
||||
not per-class, so no selector-list extension is needed). No
|
||||
animation of its own → the ``prefers-reduced-motion`` blocks need
|
||||
nothing for it (the family's parity — .retry-btn / .edit-btn have
|
||||
none either)."""
|
||||
css = _css()
|
||||
start = css.index(".continue-btn {")
|
||||
end = css.index("}", start)
|
||||
block = css[start : end + 1]
|
||||
for decl in (
|
||||
"display: inline-flex;",
|
||||
"gap: 0.35rem;",
|
||||
"min-height: 44px;",
|
||||
"padding: 0.35rem 0.8rem;",
|
||||
"border-radius: 999px;",
|
||||
"border: 1px solid var(--line);",
|
||||
"background: transparent;",
|
||||
"color: var(--ink-soft);",
|
||||
"font-weight: 600;",
|
||||
"font-size: 0.82rem;",
|
||||
"white-space: nowrap;",
|
||||
"cursor: pointer;",
|
||||
):
|
||||
assert decl in block, f"the pill family keeps {decl!r}"
|
||||
assert ".continue-btn svg { width: 14px; height: 14px; display: block; }" in css
|
||||
assert ".continue-btn:hover { background: var(--brand-soft); color: var(--ink); }" in css
|
||||
|
||||
|
||||
def test_continue_btn_mobile_block_min_height() -> None:
|
||||
"""The mobile block (task 07): ``.continue-btn`` joins the
|
||||
``.tune-btn`` / ``.retry-btn`` / ``.edit-btn`` 44px min-height
|
||||
lockstep next to ``.retry-btn`` — EXACTLY ONE such single-line
|
||||
rule (the base rule is a multi-line block, so the one-line form
|
||||
occurs only in the mobile media block)."""
|
||||
css = _css()
|
||||
assert css.count(".continue-btn { min-height: 44px; }") == 1
|
||||
idx = css.index(".continue-btn { min-height: 44px; }")
|
||||
# It sits beside the family's other lockstep lines (near them in
|
||||
# the same mobile block).
|
||||
retry = css.index(".retry-btn { min-height: 44px; }")
|
||||
assert abs(idx - retry) < 200, (
|
||||
"the continue line is beside .retry-btn in the mobile block"
|
||||
)
|
||||
@@ -565,6 +565,88 @@ def test_done_event_answer_image_ref_drops_its_own_null_image_url() -> None:
|
||||
assert "image_url" not in dump["answer_image"]
|
||||
|
||||
|
||||
# --- the ``budget_exhausted`` done key (phase 160, task 03) ---
|
||||
|
||||
|
||||
def test_done_event_without_budget_exhausted_drops_the_key() -> None:
|
||||
"""``budget_exhausted=False`` (the default — every deflected /
|
||||
in-cap / pre-phase turn) → ``model_dump()`` has NO
|
||||
``budget_exhausted`` key — ABSENT, never ``false`` — and a full
|
||||
done frame (sources + related + suggestions, no flag) serializes
|
||||
byte-identically to the pre-phase expectation (the
|
||||
byte-identical-off contract, PLAN §13)."""
|
||||
event = ChatDoneEvent(
|
||||
deflected=False,
|
||||
sources=[SourceRef(source="docs", path="a.md", title="A")],
|
||||
related=[SourceRef(source="docs", path="b.md", title="B")],
|
||||
suggestions=["Maybe try X?"],
|
||||
)
|
||||
dump = event.model_dump()
|
||||
assert "budget_exhausted" not in dump
|
||||
# The pre-phase-160 key set, exactly (answer_image absent too).
|
||||
assert set(dump) == {"type", "deflected", "sources", "related", "suggestions"}
|
||||
assert _json_dumps(dump, sort_keys=True) == _json_dumps(
|
||||
{
|
||||
"type": "done",
|
||||
"deflected": False,
|
||||
"sources": [{"source": "docs", "path": "a.md", "title": "A"}],
|
||||
"related": [{"source": "docs", "path": "b.md", "title": "B"}],
|
||||
"suggestions": ["Maybe try X?"],
|
||||
},
|
||||
sort_keys=True,
|
||||
)
|
||||
raw = _json_dumps(dump)
|
||||
assert "budget_exhausted" not in raw
|
||||
|
||||
|
||||
def test_done_event_with_budget_exhausted_carries_the_key() -> None:
|
||||
"""``budget_exhausted=True`` (a cap-reached turn) → the key is
|
||||
present and ``true``, the rest of the frame untouched; an
|
||||
explicit ``False`` behaves exactly like the default (absent)."""
|
||||
dump = ChatDoneEvent(deflected=False, sources=[], budget_exhausted=True).model_dump()
|
||||
assert dump["budget_exhausted"] is True
|
||||
assert dump["type"] == "done"
|
||||
assert dump["deflected"] is False
|
||||
assert dump["related"] == []
|
||||
assert dump["suggestions"] == []
|
||||
explicit_false = ChatDoneEvent(
|
||||
deflected=False, sources=[], budget_exhausted=False
|
||||
).model_dump()
|
||||
assert "budget_exhausted" not in explicit_false
|
||||
|
||||
|
||||
def test_done_event_budget_exhausted_and_answer_image_are_independent() -> None:
|
||||
"""Both optional keys shown on the same turn: the two omission
|
||||
rules are independent — a cap-reached turn with a shown answer
|
||||
image carries BOTH keys; the answer-image ref's own serializer
|
||||
still applies inside the frame."""
|
||||
dump = ChatDoneEvent(
|
||||
deflected=False,
|
||||
sources=[],
|
||||
answer_image=SourceRef(**_SHOWN_REF),
|
||||
budget_exhausted=True,
|
||||
).model_dump()
|
||||
assert dump["budget_exhausted"] is True
|
||||
assert dump["answer_image"] == _SHOWN_REF
|
||||
|
||||
|
||||
def test_done_event_without_budget_exhausted_parses_a_pre_phase_frame() -> None:
|
||||
"""Back-compat: a pre-phase-160 done frame (no
|
||||
``budget_exhausted`` key) parses with the default ``False`` and
|
||||
re-dumps byte-identically (PLAN §4 — the field is additive in
|
||||
both directions)."""
|
||||
old_payload = {
|
||||
"type": "done",
|
||||
"deflected": True,
|
||||
"sources": [],
|
||||
"related": [],
|
||||
"suggestions": [],
|
||||
}
|
||||
event = ChatDoneEvent(**old_payload)
|
||||
assert event.budget_exhausted is False
|
||||
assert event.model_dump() == old_payload
|
||||
|
||||
|
||||
# --- the ``ChatMessage`` boundary (the saved-chat record key) ---
|
||||
|
||||
|
||||
|
||||
@@ -259,7 +259,9 @@ def test_read_result_plain_carries_date_second_line(
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
content = llm.requests[1][0][3]["content"]
|
||||
# Phase 160 (task 01): the budget-telemetry trailer rides the read
|
||||
# result — the raw result is the RAW part (the trailer stripped).
|
||||
content = llm.requests[1][0][3]["content"].removesuffix(agent._budget_note(9))
|
||||
lines = content.splitlines()
|
||||
assert lines[0] == "Document S/a.md:" # byte-identical first line
|
||||
assert lines[1] == f"date: {DATE}" # the D5 second line
|
||||
@@ -291,7 +293,9 @@ def test_read_result_truncated_carries_date_then_marker_and_notice(
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content_body))
|
||||
)
|
||||
assert llm.requests[1][0][3]["content"] == expected
|
||||
# Phase 160 (task 01): the trailer rides the truncated result too —
|
||||
# the truncation contract (phase 95) is the RAW part.
|
||||
assert llm.requests[1][0][3]["content"].startswith(expected)
|
||||
# The truncation still records on the holder (the ToolResultPiece
|
||||
# signal) — the date line adds nothing to the counters.
|
||||
assert holder.read_truncations == [("S/big.md", cap, len(content_body))]
|
||||
|
||||
@@ -386,6 +386,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
},
|
||||
{
|
||||
"who": "brain",
|
||||
@@ -434,6 +435,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
},
|
||||
{
|
||||
"who": "user",
|
||||
@@ -447,6 +449,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
},
|
||||
{
|
||||
"who": "brain",
|
||||
@@ -460,6 +463,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
|
||||
"stopped": True, # the owner stopped the generation mid-answer
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
},
|
||||
]
|
||||
|
||||
@@ -468,7 +472,10 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
|
||||
dumped = payload.model_dump()
|
||||
# the stored shape is byte-identical: every message dict — None-keys
|
||||
# and all — round-trips unchanged (and the top-level record keeps its
|
||||
# ``share`` default).
|
||||
# ``share`` default). The phase-160 additive ``budget_exhausted``
|
||||
# marker joins as an explicit null (the phase-120 ``failed`` /
|
||||
# phase-48 ``stopped`` style — ``None`` flows as null, the
|
||||
# phase-50 byte-identical contract covers the new key).
|
||||
assert dumped["messages"] == messages
|
||||
assert dumped["share"] is False
|
||||
|
||||
@@ -488,8 +495,11 @@ def test_realistic_payload_round_trips_through_update_model() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
}
|
||||
payload = SavedChatUpdate.model_validate({"messages": [msg]})
|
||||
# Identical stored shape (the phase-160 additive ``budget_exhausted``
|
||||
# marker rides as an explicit null — the phase-120/48 style).
|
||||
assert payload.model_dump()["messages"] == [msg]
|
||||
|
||||
|
||||
@@ -708,8 +718,11 @@ def test_segments_absent_by_default_and_omitted_in_dump() -> None:
|
||||
dumped = msg.model_dump()
|
||||
assert "segments" not in dumped
|
||||
# Byte-identical to the pre-phase stored shape: every bor.chat.v1
|
||||
# optional key present (null), the three serializer-popped keys
|
||||
# (image / answer_image / segments) absent.
|
||||
# optional key present (null) — the phase-160 additive
|
||||
# ``budget_exhausted`` marker included (the phase-120 ``failed`` /
|
||||
# phase-48 ``stopped`` style — explicit null where absent) — and
|
||||
# the three serializer-popped keys (image / answer_image /
|
||||
# segments) absent.
|
||||
assert dumped == {
|
||||
"who": "brain",
|
||||
"text": "ok",
|
||||
@@ -722,6 +735,7 @@ def test_segments_absent_by_default_and_omitted_in_dump() -> None:
|
||||
"stopped": None,
|
||||
"failed": None,
|
||||
"error": None,
|
||||
"budget_exhausted": None,
|
||||
}
|
||||
# An explicit empty list is a different thing: [] is a value, so it
|
||||
# round-trips as [] (present, empty — the pop is None-gated only).
|
||||
|
||||
Reference in New Issue
Block a user