diff --git a/.agents/PLAN.md b/.agents/PLAN.md index fb2b543..8c7cefa 100644 --- a/.agents/PLAN.md +++ b/.agents/PLAN.md @@ -139,8 +139,12 @@ POST /api/chat {message, history?} (auth: require_user) SSE frames (`data: \n\n`): `thinking` (before first delta) → `tool` (grounded turns, one per model call) → `retry` (pre-first-frame restarts) → `delta` (answer tokens) → `done` `{deflected, sources[], -suggestions[]}` (terminal). Failure: `error` (terminal — no `done`, no -`query_log` row; a pre-stream DB outage is a plain 503 JSON). +suggestions[], budget_exhausted?}` (terminal). Failure: `error` (terminal — no `done`, no +`query_log` row; a pre-stream DB outage is a plain 503 JSON). A +`tools=None` request never emits a `tool` frame; a round-capped turn +whose final reply carries no visible content answers with the budget +fallback instead of erroring (phase 160, recorded revision of the +phase-71 cap-path terminal, owner-confirmed 2026-10-05). `BOR_STREAM_THINKING=0` suppresses `thinking` frames server-side (chars still counted). LLM retries (A15 extension): retry count and delay are configurable (defaults: 3 retries, 5 s flat delay), **only before a @@ -268,7 +272,8 @@ scripts.…`, and tests all honor it. ``` question=… embed_ms=… top_score=… fts_hits=… summary_hits=… tuning=N kb_chars=N history_msgs=N threshold=… deflected=… sources=… - thinking_chars=… tool_calls=N total_ms=… retries=N scaffold_stripped=N + thinking_chars=… tool_calls=N rounds=N cap_hit=0|1 total_ms=… + retries=N scaffold_stripped=N ``` (`sources=` = retrieval + agent-read docs, deduped; `tool_calls=` executed only; a cancelled turn writes neither line nor `query_log` diff --git a/.agents/phases/todo/160_tool_budget_telemetry/00_phase.md b/.agents/phases/complete/160_tool_budget_telemetry/00_phase.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/00_phase.md rename to .agents/phases/complete/160_tool_budget_telemetry/00_phase.md diff --git a/.agents/phases/todo/160_tool_budget_telemetry/01_remaining_in_tool_results.md b/.agents/phases/complete/160_tool_budget_telemetry/01_remaining_in_tool_results.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/01_remaining_in_tool_results.md rename to .agents/phases/complete/160_tool_budget_telemetry/01_remaining_in_tool_results.md diff --git a/.agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md b/.agents/phases/complete/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md rename to .agents/phases/complete/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md diff --git a/.agents/phases/todo/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md b/.agents/phases/complete/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md rename to .agents/phases/complete/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md diff --git a/.agents/phases/todo/160_tool_budget_telemetry/04_continue_researching_button.md b/.agents/phases/complete/160_tool_budget_telemetry/04_continue_researching_button.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/04_continue_researching_button.md rename to .agents/phases/complete/160_tool_budget_telemetry/04_continue_researching_button.md diff --git a/.agents/phases/todo/160_tool_budget_telemetry/05_budget_tests.md b/.agents/phases/complete/160_tool_budget_telemetry/05_budget_tests.md similarity index 100% rename from .agents/phases/todo/160_tool_budget_telemetry/05_budget_tests.md rename to .agents/phases/complete/160_tool_budget_telemetry/05_budget_tests.md diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.md new file mode 100644 index 0000000..f71cf54 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.md @@ -0,0 +1,10 @@ +All green. Final report: + +**Phase 160 completion pass — verified (final verification; all 5 tasks already in `complete/`)** + +- **Verified, no code defects found:** budget trailers (`TOOL_BUDGET_REMAINING/LAST/SPENT`), cap-path rejection + `ROUND_CAP_FALLBACK_ANSWER` + `ROUND_CAP_WRAPUP_INSTRUCTION`, `done.budget_exhausted` (ABSENT-when-false), `rounds=/cap_hit=` log fields, `agent_max_rounds=10` default, frontend Continue button + restore + shared-page exclusion, mock markers. +- **Tests/lint:** `uv run pytest` 153/153 green (EXIT 0); `uv run pytest --cov=app --cov-report=term-missing` TOTAL **99%** (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors/0 warnings. +- **Criteria:** ① decreasing trailer pinned by unit tests (9→…→spent) ✅ ② cap path ends readable, no tool lines after cap (E2E) ✅ ③ `budget_exhausted` true-only-on-cap, absent otherwise (integration) ✅ ④ Continue button → one fresh `continue` turn, fresh budget, record intact (E2E) ✅ ⑤ flag survives refresh; shared chat never renders it (`shared.js` has no `budget_exhausted` path) ✅ ⑥ `max_rounds=0` byte-identical single `tools=None` request; default still 10 ✅ ⑦ full suite/coverage/lint green ✅ ⑧ `test_tool_budget_telemetry.py` **7/7** in isolation; all 6 regression E2E suites green in isolation ✅ ⑨ commit + phase move = harness (I make no git history/index changes). +- **Docs:** README.md — fixed: the "Continue researching" button and cap behavior were undocumented (added a Chat-section paragraph + an Agent-tools-section paragraph, incl. the phase-71 recorded-revision note); env-var table, commands, and architecture all verified consistent otherwise. AGENTS.md — verified consistent, no changes needed. +- **Decisions:** no code changes required — phase was fully implemented; only README docs added. No live-infrastructure touched. +- **Next pending phase:** `161_reasoning_bubble`. diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.validate new file mode 100644 index 0000000..d02f75f --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__00_phase.a1.validate @@ -0,0 +1,123 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 34%] +........................................................................ [ 36%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 50%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 69%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 83%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 99%] +................................ [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 277 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 470 0 100% +-------------------------------------------------- +TOTAL 5776 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.md new file mode 100644 index 0000000..b7a6fc3 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.md @@ -0,0 +1,13 @@ +All gates green. Task 01 complete. + +**Report** + +- **Implemented (task 01 — budget telemetry on every tool result):** + - `app/rag/agent.py`: `TOOL_BUDGET_REMAINING` / `TOOL_BUDGET_LAST` / `TOOL_BUDGET_SPENT` constants + pure `_budget_note(remaining)` helper, placed beside the teaching-string family before `CORRECTION_INSTRUCTION`; trailer appended in `run_agent` after `rounds += 1` as `result + _budget_note(max_rounds - rounds)` — `_execute_tool` stays the pure `→ str` seam + - Module docstring loop-contract point 4 extended (trailer's three forms + "count = what's LEFT after the call it is reading"); `.env.example` `BOR_AGENT_MAX_ROUNDS` comment extended + - Unit tests: `_budget_note` boundary table (2/3/1/0/−1), 3-round trailer decreasing 2→1→spent (pinned exact in the round-cap cluster), `agent_max_rounds=1` → spent line, kill-switch pin extended (no `role:"tool"` message at all), rejected-call refusal + note, n=9/n=8 pins + - Kept ~130 existing tool-message assertions green: raw-result pins via `startswith`/`removesuffix(_budget_note(9))` (trailer is appended content, not a new message), cap cluster pinned exact `raw + note` +- **Gates:** `uv run pytest` → **3665 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (gate >90%); `uv run ruff check .` clean; `uv run pyright` 0 errors; `uv run pytest tests/integration/test_chat_api.py -v --no-cov` → 59 passed (no SSE/`done`/`tool_result`/`tool_calls=` change) +- **Docs:** README.md — both `BOR_AGENT_MAX_ROUNDS` table rows note the per-result remaining-count telemetry; AGENTS.md — no doc impact +- **Decisions:** no new setting, no default change (LOCKED held); `agent_max_rounds=0` kill switch byte-identical; truncation `ToolResultPiece`/`read_coverage` untouched +- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md` diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.validate new file mode 100644 index 0000000..e808bc6 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a1.validate @@ -0,0 +1,122 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 41%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 55%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 80%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 94%] +........................................................................ [ 96%] +........................................................................ [ 98%] +................................................................. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 270 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 440 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 466 0 100% +-------------------------------------------------- +TOTAL 5751 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.md new file mode 100644 index 0000000..11e4c2e --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.md @@ -0,0 +1,10 @@ +All gates re-verified green. The previous attempt's work is intact in the working tree; nothing was lost, and no additional code changes were needed this pass. + +**Report** + +- **Task 01 state (verified this pass):** `TOOL_BUDGET_REMAINING`/`TOOL_BUDGET_LAST`/`TOOL_BUDGET_SPENT` + pure `_budget_note()` in `app/rag/agent.py`; trailer appended in `run_agent` after `rounds += 1` (`result + _budget_note(max_rounds - rounds)`), `_execute_tool` untouched; docstring point 4 + `.env.example` comment updated +- **Tests:** new unit tests (boundary table 2/3/1/0/−1, 3-round decreasing 2→1→spent pinned exact, cap-1 spent line, kill-switch no-tool-message assert, rejection + note, n=9/n=8 pins); ~130 existing tool-message assertions kept green via raw-result `startswith`/`removesuffix` pins +- **Gates (this pass):** `uv run pytest` → **3665 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90%); `uv run ruff check .` → clean; `uv run pyright` → 0 errors; `uv run pytest tests/integration/test_chat_api.py -v --no-cov` → 59 passed (SSE/`done`/`tool_result`/`tool_calls=` unchanged); budget cluster `-k budget|round_cap|zero_max_rounds|rejected` → 9 passed +- **Docs:** README.md both `BOR_AGENT_MAX_ROUNDS` rows note the per-result telemetry; AGENTS.md no impact (not the phase's final pass) +- **Decisions:** no new setting, default 10 unchanged (LOCKED held); kill switch byte-identical; truncation/`read_coverage` paths untouched +- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/02_reject_calls_at_the_cap.md` diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.validate new file mode 100644 index 0000000..e808bc6 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__01_remaining_in_tool_results.a2.validate @@ -0,0 +1,122 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 41%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 55%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 80%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 94%] +........................................................................ [ 96%] +........................................................................ [ 98%] +................................................................. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 270 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 440 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 466 0 100% +-------------------------------------------------- +TOTAL 5751 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.md new file mode 100644 index 0000000..c376ce1 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.md @@ -0,0 +1,11 @@ +Everything is verified. The previous run had already implemented task 02's code and tests; I audited every work item against the spec, fixed one lint defect (an over-long signature line in the new integration test), and ran the full gates. + +**Report** + +- **Audited/verified (implemented by prior run, all spec-conformant):** `ROUND_CAP_WRAPUP_INSTRUCTION` + `ROUND_CAP_FALLBACK_ANSWER` constants; `AgentHolder.rejected_calls` (+ docstring); `run_agent` forced-final block (coached `messages_capped`, in-loop `ToolCallPiece` rejection → `rejected_calls`, fallback yield when `final_content == 0`, `MalformedReplyError` deleted on cap path only, both warnings); `chat.py::_pump` `tools_offered` (deflected + recovery `False`, grounded `True`) with `dropped_calls` local + debug line; module/run_agent docstrings updated +- **Fixed:** `tests/integration/test_chat_api.py:1521` E501 — split `test_cap_path_rejects_late_calls_and_fallbacks` signature across lines +- **Gates (all green):** `uv run pytest --cov=app --cov-report=term-missing` → **3672 passed, 0 failed, TOTAL 99%** (>90%); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings, 0 informations +- **Spec pins intact:** phase-45 kill-switch single `tools=None` request, `round cap reached (rounds=N)` warning, non-cap `MalformedReplyError` (grounded + deflected twice-scaffolding tests green), trailer on executed results (task 01) +- **Docs:** no doc impact from this task (no new env/setting/API shape; `budget_exhausted` + Continue button are tasks 03/04; task 01's README/.env.example notes already in tree) +- **No live-infrastructure changes** (DB container was already up for tests) +- **Next pending task:** `.agents/phases/todo/160_tool_budget_telemetry/03_budget_exhausted_flag_and_log.md` diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.validate new file mode 100644 index 0000000..dbd0a10 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__02_reject_calls_at_the_cap.a1.validate @@ -0,0 +1,122 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 41%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 50%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 80%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 94%] +........................................................................ [ 96%] +........................................................................ [ 98%] +........................................................................ [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 277 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 449 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 466 0 100% +-------------------------------------------------- +TOTAL 5767 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.md new file mode 100644 index 0000000..4cc83b4 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.md @@ -0,0 +1,16 @@ +**Task 03 complete — report** + +**Implemented:** +- `AgentHolder.rounds` / `AgentHolder.cap_reached` (`app/rag/agent.py`), documented field style; `holder.rounds` mirrored beside `rounds += 1` and once before the forced final; `cap_reached` set exactly in the round-cap block +- `ChatDoneEvent.budget_exhausted: bool = False` (`app/schemas.py`) — serializer pops it when False (ABSENT, never false; byte-identical off, `answer_image` precedent) +- `app/api/chat.py`: `budget_exhausted=holder.cap_reached` on the done frame (deflected turns correct by construction); per-turn log line gains `rounds=%d cap_hit=%d` immediately after `tool_calls=%d`, all existing fields unchanged; module docstring paragraph +- `.agents/PLAN.md`: §4 `done {…, budget_exhausted?}` + the recorded phase-71 cap-path revision sentence; §9 `rounds=N cap_hit=0|1` after `tool_calls=N` — nothing else changed +- Tests: 4 new `ChatDoneEvent` serialization pins (absent/true/independent/back-compat), 4 new `AgentHolder` state tests (refusals count as rounds, in-cap `cap_reached=False`, full count at cap, kill-switch defaults), 2 new integration tests (cap turn → `budget_exhausted: true` + `rounds=1 cap_hit=1`; in-cap/deflected turns → key absent + `cap_hit=0`), field-order pin updated in `test_chat_gate.py` + +**Gates (all green):** `uv run pytest` → 3682 passed; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL 99% (schemas 100%); `uv run ruff check .` + `uv run pyright` → clean. E2E regressions in isolation (DB up): smoke, agent_unlimited_tools, agent_document_tools, harness_aligned_tools, grep_regex_teaching, chat_rag — all green. + +**Docs:** PLAN.md §4/§9 per the task's explicit work item 5 (recorded revision, owner-confirmed 2026-10-05); no README/AGENTS.md impact (they don't document the log line or done keys). + +**Decisions:** no migration, no new setting; `agent_max_rounds` default 10 untouched; the single `ChatDoneEvent` construction site updated (deflected turns run no agent, so absent is correct by construction). + +**Next pending task:** `04_continue_researching_button.md` (UI: the Continue button, persisted marker, restore path, CSS). diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.validate new file mode 100644 index 0000000..ba2ad27 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__03_budget_exhausted_flag_and_log.a1.validate @@ -0,0 +1,123 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 41%] +........................................................................ [ 43%] +........................................................................ [ 44%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 50%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 80%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 99%] +.......... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 277 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 469 0 100% +-------------------------------------------------- +TOTAL 5775 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.md new file mode 100644 index 0000000..a7a5e8a --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.md @@ -0,0 +1,13 @@ +All complete and verified. Final report: + +**Phase 160 · Task 04 — "Continue researching" button: complete, 100% green** + +- **UI (`app.js`)**: `CONTINUE_ICON` (inline aria-hidden SVG, RETRY_ICON stroke family), `CONTINUE_QUESTION = "continue"`, `appendContinueButton` (getActionsRow pattern, one-per-bubble guard, listitem ARIA, click → `continueTurn`); `continueTurn` (in-flight no-op → clear every `.continue-btn` → plain `runTurn(CONTINUE_QUESTION)`, full history); done settle persists `budget_exhausted` (`undefined` drops the key) + clears stale buttons + flag-gated append; restore branch keys on `m.budget_exhausted`; `startNewChat` untouched (list reset drops buttons) +- **Schema**: `ChatMessage.budget_exhausted: bool | None = None` (phase-120/48 style; no serializer/migration — `chats.py` unchanged); shared page never shows the control (no `app.js` in shared.html; `shared.js` zero interactive controls) — verified + pinned +- **CSS**: `.continue-btn` in the `.retry-btn` family (global `:focus-visible` ring — no per-class extension needed; no animation → reduced-motion parity needs nothing) + mobile-block 44px lockstep +- **Tests**: new `tests/unit/test_frontend_tool_budget.py` (18 pins incl. byte-unchanged `retryLastTurn`/`markLastRetryable`/`rememberBrainTurn`); `test_chats_api.py` +4 byte-identical round-trip tests (POST/GET/PUT/shared); updated golden pins (`test_schemas`, `test_chat_message_failed`, `test_chat_image_questions`, `PINNED_RUN_TURN` sanctioned addition, `test_navbar_refresh` key set) +- **Gates**: `uv run pytest` 3704 passed · `--cov=app` TOTAL 99% (>90%) · `ruff check .` clean · `pyright` 0 errors · e2e isolated: `test_navbar_refresh` 7/7, `test_chat_rag` 3/3, `test_smoke` 3/3 · `node --check app.js` OK +- **Docs**: no README/AGENTS.md impact (no new env/commands/config; README doesn't enumerate bubble actions; the phase-160 env-table line landed in task 01) +- **Decisions**: (1) the done-settle append sits after `markLastEditable()` (still after `markLastRetryable()`) to keep the phase-159 paired-settle pin green — row order remains Tune → Retry → Continue → Save-as-doc; (2) added the done-settle remove-first clear to satisfy "earlier answers' buttons gone once a newer turn lands"; (3) restore shows the button on every flagged record (per spec — no restore re-targeting) + +Next pending task: `05_budget_tests.md` (mock round-cap marker + the isolated `tests/e2e/test_tool_budget_telemetry.py`). diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.validate new file mode 100644 index 0000000..d02f75f --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__04_continue_researching_button.a1.validate @@ -0,0 +1,123 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 34%] +........................................................................ [ 36%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 50%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 69%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 83%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 99%] +................................ [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 277 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 470 0 100% +-------------------------------------------------- +TOTAL 5776 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.err b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.md b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.md new file mode 100644 index 0000000..50be3c7 --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.md @@ -0,0 +1,13 @@ +All completion criteria are met. The stray `image.png` (Oct 5, pre-dates this session) is left untouched. Final report: + +**Task 05 — Budget tests: complete, 100% green** + +- **Implemented** + - `tests/e2e/mock_llm.py`: deterministic `ROUND_CAP_TRIGGER` ("run out of tool calls", last-user-message-only) flow — tools-offered requests stream `ls` calls with incrementing `call_cap_N` ids (guaranteed cap hit); the forced final (no tools) streams the dumped `read` call (`call_cap_dump`, seed path, no content); the `continue` question (trigger in an earlier user message) streams the fixed `ROUND_CAP_CONTINUE_ANSWER`; documented in the module docstring's marker table + - `tests/e2e/test_tool_budget_telemetry.py` (new, mock-only, 7 tests, module app with `BOR_AGENT_MAX_ROUNDS=3`, TRUNCATE-then-seed kubernetes.md, autouse `saved_chats` truncate): trailer countdown via the mock's request log, zero tool lines past the cap + fallback answer + no error frame, `done.budget_exhausted` true/absent, Continue button fresh turn + record shape, double-click single-shot (exact `/api/chat` POST count), flag survives refresh, deflected-turn negative + - Unit/integration pins (item 3) already covered by tasks 01–03's `test_agent.py` round-cap cluster (`_budget_note` boundaries, executed+refused trailers, `max_rounds=0/1`, wrapup-instruction, rejection/fallback) — no new file needed; item 4's `test_frontend_tool_budget.py` (18 pins, task 04) left as-is — nothing unpinned + +- **Gates**: `uv run pytest` → 3704 passed · `--cov=app` → TOTAL 99% (>90%; the 2 missing lines in agent.py/chat.py are pre-existing, not phase-160) · `ruff check .` + `pyright` clean · `tests/e2e/test_tool_budget_telemetry.py` → 7/7 in isolation (two consecutive runs) · regressions isolated: agent_unlimited_tools 4, agent_document_tools 4, harness_aligned_tools 3, grep_regex_teaching 2, big_read_progress 4, read_range_tool_lines 3, chat_rag 3, tool_scaffolding_guardrails 3, smoke 3 +- **Docs**: no doc impact (test-only change; README env-table note landed in task 01) +- **Decisions**: `ROUND_CAP_CONTINUE_ANSWER` is a fixed suite-constant naming the seed doc (the continuation turn deflects, so its prompt carries no `` block — the `SCOPED_SEARCH_PATH` convention); cap question verified ≈0.44 cosine against the mock gate +- **Next pending task**: none — task 05 is the phase's last task (phase dir moves to `complete/` by the harness) diff --git a/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.validate b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.validate new file mode 100644 index 0000000..d02f75f --- /dev/null +++ b/.agents/reports/160_tool_budget_telemetry/160_tool_budget_telemetry__05_budget_tests.a1.validate @@ -0,0 +1,123 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 11%] +........................................................................ [ 13%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 27%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 34%] +........................................................................ [ 36%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 50%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 69%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 83%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 99%] +................................ [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 277 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 196 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 266 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/security_headers.py 20 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 68 0 100% +app/models.py 135 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 764 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 71 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 155 0 100% +app/rag/retriever.py 248 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 470 0 100% +-------------------------------------------------- +TOTAL 5776 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.env.example b/.env.example index 2a2bdd7..288fd1b 100644 --- a/.env.example +++ b/.env.example @@ -66,7 +66,7 @@ BOR_RECENCY_HALF_LIFE_DAYS=365 # recency decay timescale, days (phase 106); t BOR_NAME_HIT_BONUS=0.005 # selection-time bonus a name-hit doc's best fused score gets (phase 119); 0 = off (pre-phase order byte-identical) # --- Agent document tools (grounded turns may extend context: ls / read / grep) --- -# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools) +# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools); every tool result carries the model's remaining count (phase 160) # BOR_READ_MAX_CHARS=128000 # cap on a `read` result (chars); over-cap reads truncate + a notice that teaches the grep -> read(offset, limit) workflow (phase 95, rewritten phase 131) # --- Import scope (A9 default; ANY well-formed extension is allowed) --- diff --git a/README.md b/README.md index b09f01d..9a17997 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,24 @@ long — trim it and re-ask" (or a mistyped question) is fixed by trimming it in the editor instead of retyping it. Earlier bubbles never carry the button (last-question-only scope, owner-locked). +**When a long answer spends its research budget** (phase 160): the +answer's meta row carries a **Continue researching** button (row order +Tune → Retry → Continue → Save-as-doc; not admin-gated — chat is +public, so any visitor can keep a long research session going). Each +tool result tells the model how many tool calls remain; calls +attempted after the budget is spent are **rejected** (not executed, +not shown) instead of being dumped into the chat, and the turn ends +with a final no-tools reply — the model is coached to answer from the +results it already has and to tell you it can keep going, and a +deterministic "budget spent" fallback line stands in if that reply +carries no visible content. Clicking the button sends one plain +`continue` turn — the budget is per-turn, so the new turn has a whole +new round budget, and nothing server-side knows or cares that this is +a continuation (A10, stateless). The button is inert while a turn is +streaming and single-shot (a stale button is removed when the new +turn starts); the flag is persisted on the saved record, so the button +restores on a refresh — but a shared chat never shows it. + ### RAG / Knowledge base (`/sources.html`) The indexed document list. The *Path* column opens each document in the same @@ -497,7 +515,18 @@ The model may call tools as many times as needed, bounded by a round cap: | Env | Default | Meaning | |-----|---------|---------| -| `BOR_AGENT_MAX_ROUNDS` | `10` | hard cap on agent tool rounds per grounded turn (0 = no tools, the kill switch) | +| `BOR_AGENT_MAX_ROUNDS` | `10` | hard cap on agent tool rounds per grounded turn (0 = no tools, the kill switch); every tool result carries the model's remaining count (phase 160) | + +At the cap, further tool calls are **rejected** (never executed, never +shown) and the turn ends with a final no-tools reply: the model is +coached to answer from the results it already has and to tell the user +they can say "continue" to keep researching; a reply with no visible +content answers with a deterministic "budget spent" fallback line +instead of an error frame (phase 160, the recorded revision of the +phase-71 cap-path terminal). The `done` frame carries an additive +`budget_exhausted` key (ABSENT on every non-cap turn) that drives the +**Continue researching** button on the answer's meta row — see the +[Chat (`/`)](#chat--) section. --- @@ -898,7 +927,7 @@ to `.env` and adjust. | Env | Default | Meaning | |-----|---------|---------| -| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools) | +| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools); every tool result carries the model's remaining count (phase 160) | | `BOR_READ_MAX_CHARS` | `128000` | Cap on a `read` tool result (chars); over-cap reads truncate + a notice that teaches the grep → `read(offset, limit)` workflow (phase 95, rewritten phase 131) | ### Import & sources diff --git a/TODO.md b/TODO.md index 4640904..aac1521 100644 --- a/TODO.md +++ b/TODO.md @@ -1 +1,4 @@ # TODO + +- [ ] Need word wrap for image explanations and other long text rather than scrolling horizontally +- [ ] There's an empty box where the LLM response will go that shows up before the LLM responds. It looks ugly. ![example image](image.png) \ No newline at end of file diff --git a/app/api/chat.py b/app/api/chat.py index 0b632da..b32e13a 100644 --- a/app/api/chat.py +++ b/app/api/chat.py @@ -122,6 +122,29 @@ the LOW prompt never carries tools, and with ``agent_max_rounds`` at **0** ``run_agent`` makes exactly one ``tools=None`` request, reproducing the pre-phase behavior (the kill switch). +No ``tool`` frame without tools (phase 160, task 02): a request made +WITHOUT tools never produces a ``tool`` frame — a structured ``tool_calls`` +reply to a ``tools=None`` request (the documented ``lite`` behavior, the +phase-71 incident class) is rejected at the transport: ``_pump`` +(``tools_offered=False`` on the deflected turn and its recovery) drops the +``ToolCallPiece`` (counted in a local, named by one debug line) instead of +rendering it as a tool line, and the grounded cap path's rejected calls are +suppressed inside ``run_agent`` (``holder.rejected_calls``). The ``tool`` +event shape (A15) is unchanged — a frame is simply no longer emitted for a +call that cannot run. + +Budget-exhausted report (phase 160, task 03, TODO L4): a turn that +hits ``BOR_AGENT_MAX_ROUNDS`` reports itself — the done frame carries +one additive key, ``budget_exhausted: true`` (driven by +``AgentHolder.cap_reached``; ABSENT — never ``false`` — on every other +turn, so non-cap frames stay byte-identical to pre-phase; the UI's +"Continue researching" affordance keys on it, task 04). The per-turn +log line records ``rounds=N`` and ``cap_hit=0|1`` after +``tool_calls=N`` (PLAN §9): ``N`` is the rounds CONSUMED (an executed +call or a rejection; a cap-path rejected call is not a round) and +``cap_hit=1`` only on a cap-reached turn — ``rounds=0 cap_hit=0`` on +deflected turns (the agent never runs) and kill-switch turns. + LLM retries (phase 67, ``BOR_LLM_RETRIES`` / ``BOR_LLM_RETRY_DELAY``, owner-locked 2026-09-01): when the aipi endpoint dies before a request has streamed its first output frame (locked A2), the request is @@ -890,11 +913,21 @@ async def chat( thinking_chars = 0 content_chars = 0 # phase 71: the turn's visible (clean) content scaffold_stripped = 0 # phase 71: sum across the turn's requests + # Phase 160 (task 02): the transport half of the cap path — a + # request made WITHOUT tools (the deflected turn, A8, and its + # recovery) never produces a ``tool`` frame: a structured + # ``tool_calls`` reply is rejected at ``_pump`` (dropped, + # counted in a local, named by one debug line). The grounded + # first pass keeps its current behavior (``run_agent`` + # suppresses the cap-path calls internally, and the kill + # switch is byte-identical — unchanged, phase 45). + tools_offered = not plan.deflected async def _pump( pieces: AsyncIterator[ StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece ], + tools_offered: bool = True, ) -> AsyncIterator[str]: """One request's piece loop (phase 71 extraction): the thinking/tool/retry/tool_result/delta handling shared by @@ -903,10 +936,27 @@ async def chat( (pinned by the existing integration suite). Phase 95: the ``ToolResultPiece`` branch emits the additive ``tool_result`` SSE frame (the seventh, optional event - type — the A15 extension).""" + type — the A15 extension). Phase 160 (task 02): + ``tools_offered=False`` (a request made WITHOUT tools — + the deflected turn and its recovery) drops any + ``ToolCallPiece`` — a ``tools=None`` request cannot run a + call, so it is rejected at the transport (no ``tool`` + frame, no visible markup) and counted in a local so one + debug line can name it. The grounded first pass keeps its + current behavior (``tools_offered=True``; ``run_agent`` + suppresses the cap-path calls internally).""" nonlocal thinking_chars, content_chars, retries_used + dropped_calls = 0 # phase 160: tools=None requests only async for piece in pieces: # StreamPiece | ToolCallPiece | RetryPiece if isinstance(piece, ToolCallPiece): + if not tools_offered: + # Phase 160 (task 02): a tools=None request + # cannot run a call — reject it (no ``tool`` + # frame, no visible markup); count it so one + # debug line can name it. Never executed, + # never rendered. + dropped_calls += 1 + continue # Phase 37 (PLAN §4 extension; phase 70): one SSE # ``tool`` frame per model-requested call. # ``argument`` is the single string the model @@ -1013,9 +1063,15 @@ async def chat( else: content_chars += len(piece.text) yield sse_event({"type": "delta", "text": piece.text}) + if dropped_calls: + logger.debug( + "chat: %d tool call(s) from a tools=None request " + "rejected, not shown (phase 160)", + dropped_calls, + ) try: - async for frame in _pump(answer_stream): + async for frame in _pump(answer_stream, tools_offered=tools_offered): yield frame if plan.deflected and deflected_filter is not None: scaffold_stripped = deflected_filter.stripped_chars @@ -1052,7 +1108,7 @@ async def chat( delay=settings.llm_retry_delay, scaffolding=recovery_filter, ) - async for frame in _pump(recovery_stream): + async for frame in _pump(recovery_stream, tools_offered=False): yield frame scaffold_stripped += recovery_filter.stripped_chars if content_chars == 0: @@ -1197,7 +1253,8 @@ async def chat( "question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d " "suggested=%d tuning=%d " "kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r " - "thinking_chars=%d tool_calls=%d total_ms=%d retries=%d scaffold_stripped=%d", + "thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d " + "retries=%d scaffold_stripped=%d", request.message, embed_ms, plan.top_score, @@ -1212,6 +1269,8 @@ async def chat( source_paths, thinking_chars, holder.tool_calls, + holder.rounds, + int(holder.cap_reached), total_ms, retries_used, scaffold_stripped, @@ -1269,6 +1328,13 @@ async def chat( related=related_refs, suggestions=plan.suggestions, answer_image=answer_ref, + # Phase 160 (task 03): the turn reports the cap + # hit — ``True`` only when the agent's round + # cap forced the final no-tools answer. + # Deflected turns run no agent, so the default + # (``False`` → ABSENT on serialization) is + # correct by construction. + budget_exhausted=holder.cap_reached, ).model_dump() ) finally: diff --git a/app/rag/agent.py b/app/rag/agent.py index eb00559..c0a100a 100644 --- a/app/rag/agent.py +++ b/app/rag/agent.py @@ -211,12 +211,34 @@ task 04): that keeps emitting rejected calls is bounded by the cap (point 4). 4. Every call the model emits is appended back to the message history as the assistant tool-call message + the tool result (refusals included), - consumes one round, and the model is called again. At the round cap — + consumes one round, and the model is called again. Every tool result + that goes on the wire carries the budget-telemetry trailer (phase 160, + task 01) — :data:`TOOL_BUDGET_REMAINING` (≥ 2 left), + :data:`TOOL_BUDGET_LAST` (1 left) or :data:`TOOL_BUDGET_SPENT` + (0 left) via :func:`_budget_note` — the count is what the model has + LEFT after the call it is reading (so the first result of a 10-round + budget reads 9, and the boundary call's result reads the spent line). + At the round cap — ``max_rounds = settings.agent_max_rounds`` (``BOR_AGENT_MAX_ROUNDS``, default 10) — the loop forces one final retried no-tools request (``chat_stream_retried`` with ``tools=None``) and returns: the cap is the **only** forced exit (besides "the stream carried no calls"), and - it bounds pathological rejected-call streams. + it bounds pathological rejected-call streams. Phase 160 (task 02): + that forced final's single system message carries + :data:`ROUND_CAP_WRAPUP_INSTRUCTION` folded in (the phase-71 + :data:`CORRECTION_INSTRUCTION` pattern), coaching the model that no + tool calls remain — do not emit any (they now have no effect), answer + from the results already in the conversation, and, if incomplete, + tell the user they can say "continue" to keep researching. A tool + call the model attempts on that ``tools=None`` final is **rejected** + — never executed, never yielded (no ``tool`` frame, no visible + markup), and counted in ``holder.rejected_calls`` (``tool_calls`` + stays executed-only). If that forced final streams no visible content + (only calls, only scaffolding, or nothing), the turn settles with the + deterministic :data:`ROUND_CAP_FALLBACK_ANSWER` — an honest + "budget spent" line — instead of a terminal + ``MalformedReplyError`` (the recorded revision of the phase-71 + cap-path terminal; the non-cap recovery path's error is unchanged). 5. A rare stream that carries both content and a tool call keeps the content (it was already emitted) **and** still runs the tool. 6. *holder* (an :class:`AgentHolder`) records the read documents and the @@ -724,6 +746,45 @@ LS_MAX_FILE_LINES = 50 #: round (no silent argument normalization). NOT_A_FOLDER = "'{arg}' is not a folder — {parent} has: {subfolders}" +#: The tool-call budget telemetry (phase 160, task 01; TODO L4 — "the +#: LLM needs to know how many tool calls it has left"): appended to +#: EVERY tool result the model receives — the raw result plus the note +#: — so the model plans its last reads and wraps up on its own instead +#: of hitting the cap mid-research. Appended in :func:`run_agent` +#: (after ``rounds += 1``), NEVER inside :func:`_execute_tool` (the +#: pure seam — its raw return stays byte-identical, the phase-95 +#: ``ToolResultPiece`` reads ``holder.read_truncations`` untouched, and +#: a rejection's refusal text carries the note too — a refusal +#: consumes a round, so the count is exactly as true there). +#: **The count is what the model has LEFT after the call it is +#: reading** — the first result of a 10-round budget reads 9, the +#: boundary call's result reads the spent line (0). The three forms +#: (``n ≥ 2`` / ``n == 1`` / ``n ≤ 0``) live in the one pure helper +#: :func:`_budget_note` so the wording sits in exactly one place. +TOOL_BUDGET_REMAINING: str = "\n\n[tool calls remaining: {n}]" +TOOL_BUDGET_LAST: str = "\n\n[1 tool call remaining — make it count, then answer]" +TOOL_BUDGET_SPENT: str = ( + '\n\n[No tool calls remain — answer now from what you have, or ' + 'tell the user to say "continue" to keep researching.]' +) + + +def _budget_note(remaining: int) -> str: + """The phase-160 budget-telemetry trailer for a tool result. + + *remaining* is ``max_rounds - rounds`` after the call's own round + was consumed — what the model has LEFT: ``≥ 2`` → the formatted + :data:`TOOL_BUDGET_REMAINING` line, ``1`` → :data:`TOOL_BUDGET_LAST`, + ``≤ 0`` → :data:`TOOL_BUDGET_SPENT`. Pure (the unit tests hit it + directly); the wording lives in the three constants above. + """ + if remaining >= 2: + return TOOL_BUDGET_REMAINING.format(n=remaining) + if remaining == 1: + return TOOL_BUDGET_LAST + return TOOL_BUDGET_SPENT + + #: The harness-owned recovery line (phase 71, task 03) — folded into the #: ORIGINAL single system message of the one bounded recovery request #: (``system_prompt + "\n" + CORRECTION_INSTRUCTION``; provider-safe, @@ -735,6 +796,40 @@ CORRECTION_INSTRUCTION: str = ( "text — no tool syntax." ) +#: The round-cap wrap-up coaching (phase 160, task 02) — folded into the +#: ORIGINAL single system message of the one forced final ``tools=None`` +#: request (``system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION``, the +#: phase-71 :data:`CORRECTION_INSTRUCTION` pattern — the user message and +#: the tool history stay in place). In the :data:`CORRECTION_INSTRUCTION` +#: voice it says exactly four things: no tool calls remain; do NOT emit a +#: tool call, tool-call JSON, or any ``<|tool_call…|>``-style markup (a +#: tool call now has no effect at all); answer from the results already in +#: the conversation, stating plainly what is still unchecked; and, if the +#: answer is incomplete, end by telling the user they can say "continue" +#: to let the model keep researching. +ROUND_CAP_WRAPUP_INSTRUCTION: str = ( + "You have no tool calls left — the research budget for this turn is " + "spent. Do NOT emit a tool call, tool-call JSON, or any " + "<|tool_call…|>-style markup: a tool call now has no effect at all. " + "Answer from the results already in this conversation, stating " + "plainly what you checked and what is still unchecked. If your " + 'answer is incomplete, end by telling the user they can say "continue"' + " to let you keep researching." +) + +#: The round-cap fallback answer (phase 160, task 02) — the deterministic +#: backstop :func:`run_agent` streams when the forced final carries NO +#: visible content (only rejected calls, only scaffolding, or nothing). +#: One short sentence in the brand voice: the budget ran out before an +#: answer came together, what was gathered above still stands, say +#: "continue" to keep digging. Free of markdown decoration (the UI renders +#: it through the normal markdown path). +ROUND_CAP_FALLBACK_ANSWER: str = ( + "I've run out of research budget for this turn before the answer came " + "together, but what I gathered above still stands — say \"continue\" " + "to let me keep digging." +) + class MalformedReplyError(LLMError): """The model kept replying in raw tool-scaffolding (phase 71). @@ -1488,6 +1583,32 @@ class AgentHolder: rejected calls (unknown tool, unknown/missing arguments or document, already-in-context) do not count. Drives the per-turn log line's ``tool_calls=N`` field (task 04). + ``rejected_calls`` (phase 160, task 02): how many tool calls the + model attempted AFTER the round cap — on the forced final + ``tools=None`` request a model (the documented ``lite`` behavior) can + still emit structured ``tool_calls``; those are REJECTED (never + executed, never shown — no ``tool`` frame, no visible markup) and + counted HERE only. They count in ``tool_calls`` in nothing (the + executed-only counter stays clean), the refusal style: "counts in + nothing". Drives the cap-path rejection warning log. + ``rounds`` (phase 160, task 03): the number of rounds CONSUMED by + the agent loop — one per call the model emitted on a tools- + offered round (an executed call or a rejection; both consume the + round — a call rejected past the cap on the forced final is NOT + a round, it counts in ``rejected_calls``). Mirrored from the + loop's local counter (``holder.rounds = rounds`` beside + ``rounds += 1``, and once before the forced final, so a turn that + ends at the cap reports the full count) — the API layer reads + it, never re-derives it. Drives the per-turn log line's + ``rounds=N`` field. + ``cap_reached`` (phase 160, task 03): ``True`` exactly where the + round cap forces the final no-tools answer (the ``run_agent`` + round-cap block); ``False`` on every turn that answered inside + the cap (and every ``agent_max_rounds=0`` kill-switch turn — the + loop never runs). Drives the per-turn log line's + ``cap_hit=0|1`` field and the done frame's ``budget_exhausted`` + key (the UI's Continue-researching affordance keys on it, task + 04). ``scaffold_stripped``: how many chars of tool-scaffolding the turn's filters removed across the turn's requests (rounds + the forced final + any recovery, phase 71) — drives the per-turn log @@ -1538,6 +1659,9 @@ class AgentHolder: read_docs: list[Document] = field(default_factory=list) tool_calls: int = 0 + rejected_calls: int = 0 + rounds: int = 0 + cap_reached: bool = False scaffold_stripped: int = 0 read_truncations: list[tuple[str, int, int]] = field(default_factory=list) shown_images: list[Document] = field(default_factory=list) @@ -2230,6 +2354,7 @@ async def run_agent( with db_factory() as tool_db: result = _execute_tool(tool_db, call, seed_docs, holder, settings) rounds += 1 # every call the model emits consumes a round + holder.rounds = rounds # phase 160 (task 03): mirror for the API layer logger.info( "agent tool=%s args=%s round=%d/%d", call.name, @@ -2253,7 +2378,20 @@ async def run_agent( ], } ) - messages.append({"role": "tool", "tool_call_id": call.id, "content": result}) + # Phase 160 (task 01): the budget-telemetry trailer — the model + # sees how many tool rounds it has LEFT after this call (appended + # here, never inside :func:`_execute_tool`, which stays the pure + # ``→ str`` seam: the raw result, the phase-95 ToolResultPiece and + # the read_coverage bookkeeping are untouched; a rejected call's + # refusal carries the note too — it consumed a round, so the + # count is exactly as true there). + messages.append( + { + "role": "tool", + "tool_call_id": call.id, + "content": result + _budget_note(max_rounds - rounds), + } + ) # Phase 95: surface this round's truncation(s) to the API layer. # The piece lands AFTER the round's ``tool`` frame (the matching # ``ToolCallPiece`` was already streamed, above) and BEFORE the @@ -2273,17 +2411,41 @@ async def run_agent( "no-tools answer", rounds, ) + # Phase 160 (task 03): report the cap — ``cap_reached`` + # is set exactly here (the forced no-tools final is about + # to run) and ``rounds`` is mirrored once more before it, + # so a turn that ends at the cap reports the FULL count + # (the API layer reads the holder; it never re-derives). + holder.cap_reached = True + holder.rounds = rounds # Phase 48: the forced final answer gets the same explicit # teardown as the loop rounds (consumer abandon mid-final # answer must still close the model's stream). Phase 67: the # forced call retries under the same locked-A2 rule as the # loop rounds. Phase 71: the forced final runs through a # fresh filter too — raw scaffolding can never reach the - # user from ANY grounded request. + # user from ANY grounded request. Phase 160 (task 02): the + # single system message carries the wrap-up coaching + # (``system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION``, + # the phase-71 :data:`CORRECTION_INSTRUCTION` pattern — the + # user message and the tool history stay in place) so the + # model is told the budget is spent and to answer or hand + # off; a ``tools=None`` request can still come back with + # structured ``tool_calls`` (the documented ``lite`` + # behavior), and those are REJECTED in the loop below — + # never executed, never yielded (no ``tool`` frame, no + # visible markup), counted in ``holder.rejected_calls``. + messages_capped = [ + { + "role": "system", + "content": system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION, + }, + *messages[1:], + ] final_filter = ScaffoldingFilter() final = chat_stream_retried( llm, - messages, + messages_capped, tools=None, retries=settings.llm_retries, delay=settings.llm_retry_delay, @@ -2292,15 +2454,22 @@ async def run_agent( final_content = 0 try: async for piece in final: + if isinstance(piece, ToolCallPiece): + # Phase 160 (task 02): the budget is spent — a + # call the model attempts now has no effect. + # Reject it: never executed, never yielded (no + # ``tool`` frame, no visible markup), counted + # here only (``tool_calls`` stays executed-only). + holder.rejected_calls += 1 + continue if isinstance(piece, StreamPiece) and piece.kind == "content": final_content += len(piece.text) yield piece finally: await final.aclose() - # Phase 71: the same capture log + turn total; a - # scaffolding-only forced final (this turn used no recovery, - # so nothing is doubled up) settles with the same terminal - # malformed-reply error rather than a silently empty answer. + # Phase 71: the same capture log + turn total (the per-span + # strip warning is the capture mechanism for new registry + # entries — unchanged). holder.scaffold_stripped += final_filter.stripped_chars for span in final_filter.stripped_spans: logger.warning( @@ -2310,13 +2479,29 @@ async def run_agent( rounds + 1, span[:200], ) - if final_content == 0 and final_filter.stripped_chars > 0: + if holder.rejected_calls: logger.warning( - "agent: the forced final answer was pure tool-scaffolding " - "— settling with a malformed-reply error" + "agent: %d tool call(s) attempted after the round cap — " + "rejected, not shown", + holder.rejected_calls, ) - raise MalformedReplyError( - "the forced final answer was raw tool-scaffolding — no " - "clean answer to stream" + # Phase 160 (task 02 — recorded revision of the phase-71 + # cap-path terminal, AGENTS.md rule 3, owner-confirmed + # 2026-10-05): a forced final with NO visible content (only + # rejected calls, only scaffolding, or nothing) settles with + # the deterministic budget-fallback answer — an honest + # "budget spent" line — instead of a terminal + # ``MalformedReplyError`` (an error frame is the wrong + # signal for the exact symptom this TODO reports). The + # NON-cap recovery path's ``MalformedReplyError`` is + # unchanged. + if final_content == 0: + logger.warning( + "agent: round cap hit with no visible reply " + "(rejected_calls=%d, scaffold_stripped=%d) — " + "answering with the budget fallback", + holder.rejected_calls, + holder.scaffold_stripped, ) + yield StreamPiece(kind="content", text=ROUND_CAP_FALLBACK_ANSWER) return diff --git a/app/schemas.py b/app/schemas.py index 03be9ff..0331f63 100644 --- a/app/schemas.py +++ b/app/schemas.py @@ -312,6 +312,18 @@ class ChatDoneEvent(BaseModel): serialization — ABSENT, never ``null`` — so those frames stay byte-identical to pre-phase (the byte-identical-off contract, task 01; the :class:`SourceRef.image_url` omission precedent). + + Phase 160 (task 03, TODO L4): ``budget_exhausted`` — the turn + reports that it hit ``BOR_AGENT_MAX_ROUNDS`` (the agent's round + cap forced the final no-tools answer; driven by + ``AgentHolder.cap_reached``). ``True`` ONLY on a cap-reached turn; + the UI's "Continue researching" affordance keys on it (task 04). + ``False`` — every deflected turn (they run no agent), every + in-cap grounded turn, and the whole pre-phase — is DROPPED on + serialization — ABSENT, never ``false`` — so those frames stay + byte-identical to pre-phase (the byte-identical-off contract, + PLAN §13; the :attr:`answer_image` / :class:`SourceRef.image_url` + omission precedent). """ type: str = "done" @@ -323,6 +335,10 @@ class ChatDoneEvent(BaseModel): #: :class:`SourceRef` (no list — LOCKED A3: one image per answer); #: the ref's own phase-83 field caps bind. answer_image: SourceRef | None = None + #: Phase 160 (task 03, TODO L4) — see the class docstring. True + #: ONLY on the cap-reached turn; ``False`` (every other turn) + #: serializes WITHOUT the key — ABSENT, never ``false``. + budget_exhausted: bool = False @model_serializer(mode="wrap") def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any: @@ -333,10 +349,15 @@ class ChatDoneEvent(BaseModel): :class:`SourceRef.image_url` / phase-123 ``image`` omission precedents). A shown ref keeps all its fields (its own serializer drops a ``None`` ``image_url`` — a shown doc is an - image doc, so in practice it always carries it).""" + image doc, so in practice it always carries it). Phase 160 + (task 03): ``budget_exhausted`` ``False`` (every non-cap turn) + serializes WITHOUT the key too — ABSENT, never ``false`` — the + same byte-identical-off contract.""" data = handler(self) if self.answer_image is None: data.pop("answer_image", None) + if not self.budget_exhausted: + data.pop("budget_exhausted", None) return data @@ -1325,6 +1346,23 @@ class ChatMessage(BaseModel): # serializer below), so pre-phase and image-less chats round-trip # byte-identically (the phase-50 contract). answer_image: SourceRef | None = None + # Phase 160 (task 04, TODO L4): the BUDGET-EXHAUSTED marker — the + # persisted answer that ran out of tool calls (a turn that hit + # ``BOR_AGENT_MAX_ROUNDS``, task 03's additive ``done`` key). The + # phase-120 ``failed`` / phase-48 ``stopped`` style: a marker on + # the brain record itself, no separate error table, no new API. + # It is the CLIENT'S RESTORE HINT, never a server behavior: the + # done-time auto-save (``rememberBrainTurn``) persists the done + # frame's ``budget_exhausted`` into this key, and the restore + # path re-renders the "Continue researching" button from it (the + # ``answer_image`` precedent — the field must exist so the + # ``extra="forbid"`` boundary does NOT 422 those saves; no + # migration: ``saved_chats.messages`` is raw JSONB, phase 50, and + # ``app/api/chats.py``'s logic is unchanged). ``None`` (every + # non-cap turn, every pre-phase record) flows as absent/null + # exactly like ``stopped`` — the phase-50 byte-identical + # round-trip contract covers the new key automatically. + budget_exhausted: bool | None = None @model_serializer(mode="wrap") def _drop_image_when_absent(self, handler: SerializerFunctionWrapHandler) -> Any: diff --git a/frontend/assets/app.js b/frontend/assets/app.js index 9480d0c..7fa3f64 100644 --- a/frontend/assets/app.js +++ b/frontend/assets/app.js @@ -618,6 +618,15 @@ let lastUserWrap = null; const RETRY_ICON = ''; +/* Phase 160 (task 04, TODO L4): the "Continue researching" button's + * forward glyph (aria-hidden decoration — the visible "Continue + * researching" label carries the accessible name). Same stroke family + * as RETRY_ICON: fill="none", currentColor stroke (the CSS themes it + * — ink-soft → ink on hover, the phase-08 palette), 24×24 viewBox, + * stroke-width 1.7. No icon font, no CDN (A11). */ +const CONTINUE_ICON = + ''; + /* Phase 159 (task 01, TODO.md L3): the Edit button's pencil glyph * (aria-hidden decoration — the visible "Edit" label carries the * accessible name). Same stroke family as RETRY_ICON: fill="none", @@ -648,6 +657,34 @@ function appendRetryButton(wrap) { meta.appendChild(btn); } +/* Phase 160 (task 04, TODO L4): the "Continue researching" button — + * the one-click continuation of a turn that spent its whole tool + * budget (the `done` frame's additive `budget_exhausted` key, task + * 03 — ABSENT on every non-cap turn, so the button never appears + * elsewhere). The house pattern through getActionsRow (phase 130 + * task 02 — the role=list row joins the button as a listitem so ARIA + * stays valid); one button per bubble. NOT admin-gated, like Retry — + * chat is public, so every visitor can keep a long research session + * going. On a capped answer the row order is Tune → Retry → Continue + * → Save-as-doc (the insertion order phase 130 relies on). Click → + * continueTurn() — a plain next turn: the budget is per-turn, so the + * new turn has a whole new round budget, and nothing server-side + * knows or cares that this is a continuation (A10). */ +const CONTINUE_QUESTION = "continue"; + +function appendContinueButton(wrap) { + const meta = getActionsRow(wrap); + if (!meta) return; + if (meta.querySelector(".continue-btn")) return; // one per bubble + const btn = document.createElement("button"); + btn.type = "button"; + btn.className = "continue-btn"; + if (meta.getAttribute("role") === "list") btn.setAttribute("role", "listitem"); + btn.innerHTML = CONTINUE_ICON + "Continue researching"; + btn.addEventListener("click", () => continueTurn()); + meta.appendChild(btn); +} + /* "Edit" button in the last USER bubble's actions row (phase 159, * task 01, TODO.md L3 — the trim-and-re-ask remedy: a "Question too * long — trim it and re-ask" turn, or any mistyped / half-formed @@ -2688,6 +2725,14 @@ function renderStoredMessage(m) { // precedes the failed record, unchanged; a mid-conversation failed // bubble gets no button — the phase-49 last-bubble-only rule). if (m.failed && m.error) appendFailedNote(wrap, m.error); + // Phase 160 (task 04, TODO L4): a cap-reached answer keeps its + // Continue affordance on a page refresh — a restored `history` is + // exactly what the continuation turn needs (runTurn's plain send + // carries conversation.slice(0, -1)). The marker is a client + // restore hint only (the schema's budget_exhausted key, never a + // server behavior), so pre-phase / non-cap records restore + // byte-identically (no key → no button). + if (m.budget_exhausted) appendContinueButton(wrap); lastBrainWrap = wrap; // phase 49: the LAST restored brain bubble wins } @@ -3304,6 +3349,30 @@ function retryLastTurn(wrap) { return runTurn(text, { reask: true }); } +/* Phase 160 (task 04, TODO L4): the "Continue researching" click — + * the retryLastTurn discipline, one turn at a time. No-op while a + * turn is in flight (thinking / streaming — the runTurn guard; the + * button is never double-fired into two turns), then clear EVERY + * rendered .continue-btn FIRST (the markLastRetryable remove-first + * pattern: the moment the new turn's answer lands, a stale button on + * any earlier bubble is gone by construction; the one-per-bubble + * guard + the clear make a double-send impossible), then a PLAIN send + * (runTurn's `reask` defaults to false): the "continue" user bubble + * appears, and the record is pushed + saved through the ordinary save + * points (conversation.push + saveConversation + persistConversation). + * The request carries the FULL history (conversation.slice(0, -1)) — + * the model resumes with everything it already read and gets a fresh + * agent_max_rounds. ASSUMPTION: the continuation question is the + * literal word "continue" — what the model is taught to ask the user + * for (ROUND_CAP_WRAPUP_INSTRUCTION / the tool-budget-spent trailer, + * phase 160 tasks 01–02), so the visible bubble, the typed + * equivalent, and the model's own suggestion all agree. */ +function continueTurn() { + if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return; + messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove()); + return runTurn(CONTINUE_QUESTION); +} + /* Phase 123 (task 02, TODO L6): the send flow's UPLOAD STEP (locked * A8) — the attached File goes to POST /api/chat-images as multipart * (the session cookie rides the browser; the endpoint is user-gated @@ -3933,6 +4002,12 @@ async function runTurn(text, { reask = false, image = null } = {}) { // row exactly as it looked live. related: ev.related?.length ? ev.related : undefined, suggestions: ev.suggestions, + // Phase 160 (task 04, TODO L4): the cap-reached marker + // persists with the turn — `undefined` drops the key from + // the JSON (the house optional-meta pattern, the + // answer_image line above is the precedent), so every + // non-cap record stays byte-identical to pre-phase. + budget_exhausted: ev.budget_exhausted || undefined, }, leavePartialIndex); lastBrainWrap = wrap; // this bubble is now the last brain answer // Phase 130 (task 02): the row order is the DOM order — the @@ -3943,6 +4018,20 @@ async function runTurn(text, { reask = false, image = null } = {}) { appendTuneButton(wrap); // every completed brain bubble is tunable markLastRetryable(); // phase 49: the Retry button is last-bubble-only markLastEditable(); // phase 159: the done turn's question is editable (the in-flight guard no-ops here — the runTurn idle settle lands it) + // Phase 160 (task 04, TODO L4): a cap-reached answer offers + // the one-click continuation — first, every earlier bubble's + // continue button is cleared (the markLastRetryable + // remove-first pattern: the moment a newer turn lands, an old + // bubble's stale affordance is gone — its "continue" would + // append to a history the conversation already moved past), + // then the button is appended ONLY when the done frame is + // flagged (ABSENT on every other turn, so the button never + // appears elsewhere); the row order stays Tune → Retry → + // Continue → Save-as-doc (the insertion order phase 130 + // relies on — markLastEditable above acts on the user + // bubble, so the brain bubble's row order is unchanged). + messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove()); + if (ev.budget_exhausted) appendContinueButton(wrap); // Phase 59: the RAW persisted markdown (never the rendered // HTML) — exactly the string rememberBrainTurn stores above, // so a reload (the restore path) offers the identical draft. diff --git a/frontend/assets/styles.css b/frontend/assets/styles.css index ef0d50e..d9b9158 100644 --- a/frontend/assets/styles.css +++ b/frontend/assets/styles.css @@ -1027,6 +1027,39 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und .retry-btn svg { width: 14px; height: 14px; display: block; } .retry-btn:hover { background: var(--brand-soft); color: var(--ink); } +/* Phase 160 (task 04, TODO L4): the "Continue researching" button — + the one-click continuation in the actions row + (.msg-meta.msg-actions — phase 130) of a cap-reached brain bubble + (the done frame's budget_exhausted, task 03). The exact visual + family of .retry-btn (same pill size/spacing, the global + :focus-visible ring, >=44px via min-height) so the meta actions + read as a set — but neutral: ink-soft -> ink on hover (Tune keeps + the brand pair). The forward glyph rides currentColor. No + per-button auto margin (phase 130 — the row's justify right-aligns + the cluster). No animation of its own, so the + prefers-reduced-motion blocks need nothing for it (the family's + parity — .retry-btn / .edit-btn have none either). Contrast: ink-soft on --bg ~8.6:1, ink + ~15:1, hover ink on --brand-soft ~15.7:1 — all AA. */ +.continue-btn { + display: inline-flex; + align-items: center; + justify-content: center; + gap: 0.35rem; + min-height: 44px; + padding: 0.35rem 0.8rem; + border-radius: 999px; + border: 1px solid var(--line); + background: transparent; + color: var(--ink-soft); + font: inherit; + font-weight: 600; + font-size: 0.82rem; + white-space: nowrap; + cursor: pointer; +} +.continue-btn svg { width: 14px; height: 14px; display: block; } +.continue-btn:hover { background: var(--brand-soft); color: var(--ink); } + /* Phase 159 (task 01, TODO L3): the "Edit" button — the * trim-and-re-ask action in the .msg-meta.msg-actions row (phase 130) * of the NEWEST user bubble (markLastEditable — the phase-49 @@ -5817,6 +5850,7 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und .tune-btn { min-height: 44px; } .retry-btn { min-height: 44px; } .edit-btn { min-height: 44px; } + .continue-btn { min-height: 44px; } /* Phase 27: the tuning page squeezes like the other cards — the row padding tightens; the icon+label actions keep their 44px floor and the note text ellipsizes (min-width: 0). */ diff --git a/image.png b/image.png new file mode 100644 index 0000000..cc22dcc Binary files /dev/null and b/image.png differ diff --git a/tests/e2e/mock_llm.py b/tests/e2e/mock_llm.py index 2983bd5..ce50d8e 100644 --- a/tests/e2e/mock_llm.py +++ b/tests/e2e/mock_llm.py @@ -842,6 +842,57 @@ Implements just enough of the aipi surface: deflected turns never hit it); verified 2026-10-01: no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. + - user message containing ``run out of tool calls`` + (``ROUND_CAP_TRIGGER``, phase 160 task 05 — the + tool-call-budget telemetry's dedicated suite + ``tests/e2e/test_tool_budget_telemetry.py``) — detected on + the LAST user message only (the mock's existing invariant: + the last ``user`` message is the current question, the + earlier non-system messages are prior turns): + * while ``tools`` are offered (the request carries + ``tools`` — the app issues such a request only while the + round budget is unspent, i.e. no tool result answers the + last emitted call yet) and the flow is active: stream + ONLY ``tool_calls`` deltas — ``ls`` with no arguments, + incrementing synthetic ids ``call_cap_1``, + ``call_cap_2``, … (the next id is derived from the + ``call_cap_N`` ids already in the messages' ``assistant`` + tool calls), ``finish_reason: "tool_calls"``, no content — + the model NEVER stops asking, so the turn is guaranteed + to hit ``BOR_AGENT_MAX_ROUNDS``; + * on the request WITHOUT ``tools`` (the forced final — the + round-cap path; the request still carries the full + history, so the trigger stays the last user message): + stream a ``tool_calls`` delta ANYWAY — ``read`` on the + SEED path (the system prompt's ```` section's + FIRST ```` block, the joined ``source/path`` — + the ``read`` tool's combined identity, phase 70), id + ``call_cap_dump``, NO content — the EXACT dumped-call + shape the phase kills (and the shape ``SCAFFOLD_TRIGGER`` + does not cover — that one emits textual markup, this one + is structured). The app REJECTS it (never executed, never + shown, counted in ``AgentHolder.rejected_calls``); + * the ``continue`` question (the last user message is + EXACTLY ``continue`` — the UI's ``CONTINUE_QUESTION`` — + and an EARLIER user message carries the trigger; the + client history rides in the stateless A10 request): the + stable ``ROUND_CAP_CONTINUE_ANSWER`` (a fixed sentence + naming the flow's seed document — the + ``SCOPED_SEARCH_PATH`` suite-constant convention: the + literal "continue" shares no tokens with the KB, so the + continuation turn deflects and its prompt carries no + ```` block; the sentence names the document + directly), so a suite asserts a real continuation answer + rather than a tool call. + Streaming only (the app's chat path is the only streaming + consumer; a non-streaming marker request never issues and + falls through to the regular answer, like the other marker + flows). Checked BEFORE the plain ``TOOLS_TRIGGER`` flow + (disjoint trigger phrases — the phase-71/72/94/118/131/ + 139/141/145 ordering convention); verified 2026-10-05: no + existing E2E question or fixture file contains the phrase + (or sends a ``continue`` question with the trigger in the + history), so every other suite is unaffected. Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic dead-endpoint behavior for the retry E2E suite (``tests/e2e/ @@ -1193,6 +1244,42 @@ HISTORY_TRIGGER = "echo my history" #: phrase, so every other suite is unaffected. FOLDER_MAP_TRIGGER = "repeat your folder map" +#: Phase 160 (task 05, tool-call budget telemetry, TODO.md L4 — +#: the dedicated suite +#: ``tests/e2e/test_tool_budget_telemetry.py``): a user message +#: containing this substring (case-insensitive), detected on the +#: LAST user message only (the mock's existing invariant — the +#: last ``user`` message is the current question, the earlier +#: non-system messages are prior turns), drives the deterministic +#: ROUND-CAP flow: while ``tools`` are offered the mock NEVER +#: answers (``ls`` calls, incrementing ``call_cap_N`` ids — the +#: turn is guaranteed to hit ``BOR_AGENT_MAX_ROUNDS``), and the +#: forced final (no ``tools``) streams a structured ``read`` call +#: anyway (id ``call_cap_dump``, the seed path, NO content) — the +#: exact dumped-call shape the phase kills. The ``continue`` +#: question (exactly ``continue``, the trigger in an EARLIER user +#: message) gets :data:`ROUND_CAP_CONTINUE_ANSWER`. Disjoint from +#: every other trigger phrase; verified 2026-10-05: no existing +#: E2E question or fixture file contains the phrase, so every +#: other suite is unaffected. +ROUND_CAP_TRIGGER = "run out of tool calls" + +#: Phase 160 (task 05): the ROUND-CAP flow's CONTINUATION answer +#: — the fixed sentence the ``continue`` question gets. The +#: literal "continue" shares no tokens with the KB, so the +#: continuation turn DEFLECTS (A8 LOW) and its prompt carries no +#: ```` block — the sentence therefore names the +#: flow's seed document DIRECTLY (the +#: :data:`SCOPED_SEARCH_PATH` suite-constant convention; the +#: suite seeds exactly that document, so the name IS the +#: seed's). Byte-stable on purpose: the suite asserts on it, so +#: it must not drift. +ROUND_CAP_CONTINUE_ANSWER = ( + "Picking up where the tool budget ran out — still digging into " + "docs/homelab/kubernetes.md: it says the cluster runs Talos " + "Linux on three nodes, and Cilium handles networking." +) + #: Phase 122 (image documents, LOCKED A3): the fixed description the #: mock's vision (CHAT) model returns for an ``IMAGE_DESCRIPTION_MODE`` #: request — the multimodal user message @@ -3079,6 +3166,106 @@ def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None: return ("call", target, "call_0") +#: Phase 160 (task 05): the round-cap flow's incrementing call ids +#: — ``call_cap_1``, ``call_cap_2``, … The NEXT id is derived from +#: the ids already in the messages' ``assistant`` tool calls (the +#: flow is stateless over the message history — a retry that +#: re-issues the same request re-derives the same next id). +_CAP_CALL_ID_RE = re.compile(r"^call_cap_(\d+)$") + + +def _round_cap_next_call_id(body: dict[str, Any]) -> str: + """The next incrementing ``call_cap_N`` id (stateless). + + The mock's ``ls`` calls in the round-cap flow carry + incrementing synthetic ids — the NEXT id is one past the + highest ``call_cap_N`` id already in the messages' + ``assistant`` tool calls (``call_cap_1`` on the first + request — no ids yet). + """ + n = 0 + for m in _messages(body): + if m.get("role") != "assistant": + continue + for call in m.get("tool_calls") or []: + if not isinstance(call, dict): + continue + match = _CAP_CALL_ID_RE.match(str(call.get("id") or "")) + if match: + n = max(n, int(match.group(1))) + return f"call_cap_{n + 1}" + + +def _round_cap_seed_path(body: dict[str, Any]) -> str: + """The seed document's combined ``source/path`` — or the + sentinel (the premise broke: no ```` block; the + suite fails loudly on the sentinel in the dump call). + + Phase 160 (task 05): the FORCED FINAL's request (the + round-cap path) still carries the grounded system prompt — + the ```` section's FIRST ```` block is + the seed document (the suite seeds exactly one). The mock + joins the labeled ``source`` / ``path`` fields (the + ``read`` tool's combined identity, phase 70). + """ + block = _DOCUMENT_BLOCK_RE.search(_system(body)) + if not block: + return "seed/unknown.md" + return f"{block.group('source')}/{block.group('path')}" + + +def _round_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None: + """Phase 160 (task 05): the ROUND-CAP marker flow — or ``None``. + + Discriminated on the LAST user message (the mock's existing + invariant: the last ``user`` message is the current + question, the earlier non-system messages are prior turns — + the ``_history_echo`` ``last_user`` scan convention): + + * the last user message carries :data:`ROUND_CAP_TRIGGER`: + - the request offers ``tools`` (the app issues such a + request only while the round budget is unspent — no + tool result answers the last emitted call yet): + ``("call", call_cap_N)`` — the model NEVER stops + asking (``ls``, no arguments, the next incrementing + id) — the turn is guaranteed to hit + ``BOR_AGENT_MAX_ROUNDS``; + - the request carries NO ``tools`` (the forced final — + the round-cap path; the trigger is still the last user + message): ``("dump", )`` — a ``read`` on + the seed path (id ``call_cap_dump``, NO content) — the + exact dumped-call shape the phase kills; + * the last user message is EXACTLY ``continue`` (the UI's + ``CONTINUE_QUESTION``) and an EARLIER user message + carries the trigger (the client history rides in the + stateless A10 request): ``("continue",)`` — the stable + :data:`ROUND_CAP_CONTINUE_ANSWER` (a fixed sentence + naming the flow's seed document — the + :data:`SCOPED_SEARCH_PATH` suite-constant convention: the + literal "continue" shares no tokens with the KB, so the + continuation turn deflects and its prompt carries no + ```` block). + """ + users = [ + _content_text(m.get("content")) + for m in _messages(body) + if m.get("role") == "user" + ] + if not users: + return None + last_user = users[-1].strip().lower() + if ROUND_CAP_TRIGGER in last_user: + if body.get("tools"): + return ("call", _round_cap_next_call_id(body)) + return ("dump", _round_cap_seed_path(body)) + if ( + last_user == "continue" + and any(ROUND_CAP_TRIGGER in u.lower() for u in users[:-1]) + ): + return ("continue",) + return None + + def long_answer() -> str: """~900-word deterministic walkthrough (phase 11): numbered steps plus a unique final line that must survive the stream untruncated.""" @@ -4563,6 +4750,42 @@ def chat_completions(body: dict[str, Any]) -> Any: media_type="text/event-stream", headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, ) + # Phase 160 (task 05, tool-call budget telemetry, + # TODO.md L4): the deterministic ROUND-CAP marker flow + # (the budget-telemetry suite's dedicated flow — see the + # module docstring). Checked BEFORE the plain + # TOOLS_TRIGGER flow (disjoint trigger phrases — the + # phase-71/72/94/118/131/139/141/145 ordering + # convention). + round_cap = _round_cap_flow(body) + if round_cap is not None: + if round_cap[0] == "call": + # The model never stops asking: ``ls`` with no + # arguments, the next incrementing id, NO content + # — the turn is guaranteed to hit + # ``BOR_AGENT_MAX_ROUNDS``. + stream = _tool_call_stream("ls", {}, round_cap[1]) + elif round_cap[0] == "dump": + # The forced final (no ``tools``) — the model + # dumps a structured ``read`` call anyway (the + # seed path, id ``call_cap_dump``, NO content) — + # the exact shape the phase kills (the app + # REJECTS it — never executed, never shown). + stream = _tool_call_stream( + "read", {"path": round_cap[1]}, "call_cap_dump" + ) + else: # "continue" — the stable continuation answer + stream = _sse_stream( + _apply_max_tokens( + ROUND_CAP_CONTINUE_ANSWER, body.get("max_tokens") + ), + 0.0, + ) + return StreamingResponse( + stream, + media_type="text/event-stream", + headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, + ) # Phase 124 (task 07, image chips and answers, TODO L3): the # scripted SHOW_IMAGE flow — the mock shows the seeded image # document (the ```` block with the image- diff --git a/tests/e2e/test_navbar_refresh.py b/tests/e2e/test_navbar_refresh.py index cb11667..cbf80ca 100644 --- a/tests/e2e/test_navbar_refresh.py +++ b/tests/e2e/test_navbar_refresh.py @@ -628,7 +628,8 @@ def test_api_created_chats_carry_the_bor_chat_v1_shape( # brain records (accepted by the ChatMessage schema — its earlier # absence 422'd the done-time auto-save, phase-118 verification fix). allowed = {"who", "text", "sources", "related", "deflected", "suggestions", - "thinking", "tools", "stopped", "failed", "error"} + "thinking", "tools", "stopped", "failed", "error", + "budget_exhausted"} assert all({"who", "text"} <= set(m) <= allowed for m in body["messages"]) # The History row renders it (the Open link's text IS the title). page.click("#nav-history") diff --git a/tests/e2e/test_tool_budget_telemetry.py b/tests/e2e/test_tool_budget_telemetry.py new file mode 100644 index 0000000..c64e7b6 --- /dev/null +++ b/tests/e2e/test_tool_budget_telemetry.py @@ -0,0 +1,816 @@ +"""Phase 160 E2E (Playwright, mock-only): the agent knows its +tool-call budget (and never dumps tool calls into the answer). + +Source: ``TODO.md`` L4 — "The LLM need to know how many tool calls +it has left. During long research sessions it will sometimes end the +conversation with tool calls that just get dumped into chat because +it ran out of turns. …" +Run in isolation (DB must be up: ``podman compose up -d db``): + + uv run pytest tests/e2e/test_tool_budget_telemetry.py -v --no-cov + +MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the contract +is the deterministic ROUND-CAP marker flow in ``tests/e2e/mock_llm.py`` +(``ROUND_CAP_TRIGGER`` = ``run out of tool calls``, detected on the +LAST user message only — the mock's existing invariant: the last +``user`` message is the current question, the earlier non-system +messages are prior turns): + +* while ``tools`` are offered (the app issues such a request only + while the round budget is unspent — i.e. no tool result answers + the last emitted call yet) the mock NEVER answers — it streams + ONLY ``tool_calls`` deltas calling ``ls`` with no arguments, + incrementing ids ``call_cap_1``, ``call_cap_2``, … (the next id + is derived from the ids already in the messages' ``assistant`` + tool calls), ``finish_reason: "tool_calls"``, no content — the + turn is guaranteed to hit ``BOR_AGENT_MAX_ROUNDS``; +* on the request WITHOUT ``tools`` (the forced final) the mock + streams a ``tool_calls`` delta ANYWAY — ``read`` on the seed + path (id ``call_cap_dump``, NO content) — the exact dumped-call + shape the phase kills (the app REJECTS it: never executed, never + shown, counted in ``AgentHolder.rejected_calls``); +* the ``continue`` question (exactly ``continue`` — the UI's + ``CONTINUE_QUESTION`` — with the trigger in an EARLIER user + message; the client history rides in the stateless A10 request) + gets the stable ``ROUND_CAP_CONTINUE_ANSWER`` (a fixed sentence + naming the seed document — the ``SCOPED_SEARCH_PATH`` + suite-constant convention: the literal "continue" shares no tokens + with the KB, so the continuation turn deflects and its prompt + carries no ```` block; the sentence names the document + directly). + +The e2e app server boots with ``BOR_AGENT_MAX_ROUNDS=3`` (the +module-scoped ``app_server`` override — the conftest env shape with +one swap): the cap flow burns exactly three executed rounds, and +the tool results carry ``[tool calls remaining: 2]``, ``[1 tool +call remaining — make it count, then answer]`` and the spent line +(``[No tool calls remain — answer now from what you have, or tell +the user to say "continue" to keep researching.]`` — the +``app.rag.agent`` constants, imported so the pin cannot drift). + +KB fixture (the ``test_failed_turn_retry.py`` TRUNCATE-then-seed +pattern): ONE fixture document (``homelab/kubernetes.md``) — the +cap question's tokens clear the mock-calibrated gate (≈0.44 against +the mock's token-overlap embeddings, past the 0.30 threshold — +HIGH without any FTS help; the E2E app pins the mock-calibrated +values in conftest). + +Test → contract mapping (``00_phase.md`` §Completion criteria): + +1. ``test_tool_results_carry_the_remaining_count`` — server-side + pin at the wire level: the mock's recorded request log (the + ``/v1/e2e/captured`` pair — reset per turn) shows each + successive ``tool``-role message ending with the decreasing + trailer (``[tool calls remaining: 2]``, the last-call line, the + spent line), the incrementing ``call_cap_N`` ids in the + ``assistant`` messages, and the final request carrying the spent + tool result + the coached system message (no ``tools`` key); +2. ``test_capped_turn_never_shows_tool_lines_after_the_cap`` — + with ``BOR_AGENT_MAX_ROUNDS=3``: the chat shows EXACTLY three + tool lines (one per executed round), the dump call produces NO + tool line (no ``Reading `` line anywhere), the answer + bubble holds the non-empty ``ROUND_CAP_FALLBACK_ANSWER`` (the + deterministic backstop — the mock streams no content on the + forced final), the turn settles with ``done`` (never the error + banner) and the SSE capture has no ``error`` frame; +3. ``test_done_flags_budget_exhausted`` — the captured SSE + stream's ``done`` frame carries ``budget_exhausted: true`` on + the capped turn; a normal grounded turn's ``done`` carries NO + ``budget_exhausted`` key at all (absent, never false — the + byte-identical control case in the same file); the capped + answer's Continue button is gone once the newer turn lands; +4. ``test_continue_button_runs_a_fresh_turn`` — the capped answer + bubble shows **Continue researching**; clicking it appends + exactly ONE user bubble ``continue``, the mock answers with + ``ROUND_CAP_CONTINUE_ANSWER`` (a real continuation answer — a + fixed sentence naming the seed document, not a tool call), the + earlier tool lines do not duplicate (still exactly three — the + continuation turn streams no tool frames), and ``bor.chat.v1`` + holds ``[user, brain(flagged), user("continue"), brain]`` (the + flagged record carries ``budget_exhausted: true``); +5. ``test_continue_is_inert_while_streaming_and_single_shot`` — + double-clicking Continue (``page.dblclick`` — the second click + lands after the first handler removed every ``.continue-btn`` + and started the turn; the in-flight no-op guard + the + remove-first clear make a double-send impossible) starts ONE + turn (counted via ``page.on("request")`` — exactly one + ``/api/chat`` POST; the single ``continue`` user bubble + the + single continuation answer pin it); +6. ``test_flag_survives_a_refresh`` — ``page.reload()`` after a + capped turn: the flagged answer restores WITH the Continue + button (the restore path keys on ``m.budget_exhausted``) and + clicking it still works (the mock's continuation answer + streams); +7. negative: ``test_deflected_turn_gets_no_continue_button`` — a + deflected question (A8 LOW mode — token-disjoint from the KB: + ``Who composed Rhapsody in Blue?`` ≈0.05, zero FTS hits) + settles deflected with NO ``budget_exhausted`` key, NO ``tool`` + frame and NO Continue button. +""" +from __future__ import annotations + +import hashlib +import json +import os +import re +import subprocess +import sys +import time +from collections.abc import Iterator +from datetime import UTC, datetime +from pathlib import Path + +import httpx +import pytest +from playwright.sync_api import Page, expect +from sqlalchemy import text +from sqlalchemy.orm import Session + +from app.config import Settings as _Settings +from app.db import SessionLocal +from app.models import Chunk, Document +from app.rag.agent import ( + ROUND_CAP_FALLBACK_ANSWER, + ROUND_CAP_WRAPUP_INSTRUCTION, + TOOL_BUDGET_LAST, + TOOL_BUDGET_REMAINING, + TOOL_BUDGET_SPENT, +) +from e2e.auth_helpers import login +from e2e.conftest import ( + ADMIN_PASSWORD, + SESSION_SECRET, + USE_REAL_LLM, + _wait_http, +) +from tests.e2e.mock_llm import ROUND_CAP_CONTINUE_ANSWER, embed_text + +REPO = Path(__file__).resolve().parents[2] + +#: This module app binds its OWN port (the conftest session app +#: owns its port in a combined run — a same-port second uvicorn +#: dies on bind and would drive the wrong server; the +#: test_big_read_progress.py convention, env-overridable). +APP_PORT = int(os.environ.get("E2E_APP_PORT_TOOLBUDGET", "8129")) +APP_URL = f"http://127.0.0.1:{APP_PORT}" + +#: The cap the whole suite runs under (the module app's env +#: override): the cap flow burns exactly three executed rounds — +#: the trailer math runs over a 3-round budget (2 → 1 → spent). +MAX_ROUNDS = 3 + +# -------------------------------------------------------------------------- +# Seed + questions (see the module docstring for the gate notes) +# -------------------------------------------------------------------------- + +SEED_SOURCE = "docs" +SEED_PATH = "homelab/kubernetes.md" +SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}" +KUB_CONTENT = (REPO / "tests" / "fixtures" / "docs" / SEED_PATH).read_text() + +#: The cap-flow question: the tokens clear the mock-calibrated +#: gate (≈0.44 against the mock's token-overlap embeddings, past +#: the 0.30 threshold — HIGH without any FTS help), and it carries +#: ``ROUND_CAP_TRIGGER``. +CAP_QUESTION = ( + "Explain my kubernetes cluster: talos linux nodes, cilium " + "networking, local-path-provisioner and longhorn storage. " + "run out of tool calls" +) +assert "run out of tool calls" in CAP_QUESTION.lower() + +#: The normal grounded control (test 3 — the byte-identical +#: ``done`` case): token overlap with the seed (≈0.21, past the +#: 0.15 lexical floor) + FTS hits — grounded, no trigger, no tool +#: calls (the mock answers plain). +CONTROL_QUESTION = "How is my Kubernetes cluster set up?" + +#: The negative (test 7 — A8 LOW): token-disjoint from the KB +#: (≈0.05, zero FTS hits — deflected; the phase-112 calibrated +#: question). +DEFLECT_QUESTION = "Who composed Rhapsody in Blue?" + +STORAGE_KEY = "bor.chat.v1" + +# -------------------------------------------------------------------------- +# The module app (conftest env shape + the BOR_AGENT_MAX_ROUNDS=3 +# swap) +# -------------------------------------------------------------------------- + + +@pytest.fixture(scope="module") +def app_server(mock_llm: int) -> Iterator[str]: + """The app under test — the conftest's session-app env shape + (including the phase-61/62 leak guards) with one swap: + ``BOR_AGENT_MAX_ROUNDS=3`` (the cap flow's denominator — the + trailer math runs over a 3-round budget).""" + env = dict(os.environ) + env.pop("DEBUGPY", None) + env["BOR_ENVIRONMENT"] = "e2e" + env["BOR_STATIC_DIR"] = str(REPO / "frontend") + env["BOR_LLM_BASE_URL"] = ( + "https://aipi.reeseapps.com/v1" + if USE_REAL_LLM + else f"http://127.0.0.1:{mock_llm}/v1" + ) + # The E2E mock's token-overlap embeddings have their own score + # distribution (phase 09) — the mock-calibrated values (the + # conftest pattern). + env["BOR_RELEVANCE_THRESHOLD"] = "0.30" + env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15" + env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15" + # Phase 67: instant retry waits; the real retry budget (the + # conftest pattern). + env["BOR_LLM_RETRY_DELAY"] = "0" + env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default) + env.setdefault( + "BOR_DATABASE_URL", + "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese", + ) + # Phase 16: admin auth must be set or create_app() refuses to boot. + env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD + env["BOR_SESSION_SECRET"] = SESSION_SECRET + # Phase 61/62 leak guards (the conftest pattern): force the code + # defaults so an operator's local (gitignored) ``.env`` cannot + # leak into the app under test. + env["BOR_DOCS_REPO"] = "" + env["BOR_SUGGESTIONS"] = json.dumps( + _Settings.model_fields["suggestions"].default + ) + env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default + env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default + # The suite's swap: the cap flow burns exactly three executed + # rounds (the trailer math: 2 → 1 → spent). + env["BOR_AGENT_MAX_ROUNDS"] = str(MAX_ROUNDS) + proc = subprocess.Popen( + [sys.executable, "-m", "uvicorn", "app.main:app", + "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"], + cwd=REPO, + env=env, + ) + try: + _wait_http(f"{APP_URL}/api/health") + yield APP_URL + finally: + proc.terminate() + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() + + +@pytest.fixture(scope="module") +def app_url(app_server: str) -> str: + return app_server + + +# -------------------------------------------------------------------------- +# DB seeding (TRUNCATE-then-seed, cf. test_failed_turn_retry.py) +# -------------------------------------------------------------------------- + + +def _seed(db: Session) -> None: + """The single fixture document (see the module docstring).""" + md = Document( + source=SEED_SOURCE, + path=SEED_PATH, + full_path=f"/tmp/{SEED_PATH}", + title="Kubernetes", + content=KUB_CONTENT, + content_hash=hashlib.sha256(KUB_CONTENT.encode()).hexdigest(), + indexed_at=datetime.now(UTC), + ) + db.add(md) + db.flush() + # One chunk carrying the mock's own embedding → genuine token + # overlap for the grounded questions (the FTS path carries them + # to HIGH). + db.add( + Chunk( + document_id=md.id, + position=0, + content=KUB_CONTENT, + embedding=embed_text(KUB_CONTENT), + ) + ) + + +def _reset_db() -> None: + """Truncate the KB (plus the prompt-shaping tables), then + re-seed. + + ``steering_notes`` / ``kb_overview`` are truncated too, so the + prompts are byte-stable regardless of leftovers from other + suites. + """ + with SessionLocal() as db: + db.execute( + text( + "TRUNCATE chunks, documents, query_log, steering_notes, " + "kb_overview" + ) + ) + db.commit() + _seed(db) + db.commit() + + +@pytest.fixture(autouse=True) +def clean_chats() -> Iterator[None]: + """The send auto-saves a ``saved_chats`` row per turn (phase 55) — + truncate around every test so the suite starts from (and leaves) + an empty deployment (the phase-80/103 isolation pattern).""" + with SessionLocal() as db: + db.execute(text("TRUNCATE saved_chats")) + db.commit() + yield + # The auto-save is fire-and-forget (the browser never awaits the + # PUT): a short margin lets the last turn's PUT land server-side + # before the TRUNCATE, so the teardown never 500s an in-flight + # upsert (the "Could not refresh instance" race). This runs AFTER + # the page closes (LIFO finalization), so the PUT is either done + # or aborted — never in flight. + time.sleep(0.75) + with SessionLocal() as db: + db.execute(text("TRUNCATE saved_chats")) + db.commit() + + +# -------------------------------------------------------------------------- +# Page helpers (the SSE capture — the phase-37/67 pattern) + the +# mock's request-log capture +# -------------------------------------------------------------------------- + +SSE_HOOK = """ +() => { + if (window.__sseInstalled) return; + window.__sseInstalled = true; + window.__sseFrames = []; + const origFetch = window.fetch; + window.fetch = async function (...args) { + const res = await origFetch.apply(this, args); + try { + const url = typeof args[0] === 'string' ? args[0] : args[0].url; + if (url.includes('/api/chat')) { + res.clone().text().then((bodyText) => { + for (const block of bodyText.split('\\n\\n')) { + const line = block.trim(); + if (line.startsWith('data: ')) { + window.__sseFrames.push(line.slice(6)); + } + } + }); + } + } catch (e) { /* non-clonable responses: ignored */ } + return res; + }; +} +""" + + +def _install_sse_hook(page: Page) -> None: + page.evaluate(SSE_HOOK) + + +def _frames(page: Page, terminal: str = "done") -> list[dict]: + """The captured SSE frames of the CURRENT turn, once the + *terminal* frame lands.""" + deadline = time.monotonic() + 30.0 + while True: + raw = page.evaluate("() => window.__sseFrames || []") + parsed = [json.loads(line) for line in raw if line] + if any(f.get("type") == terminal for f in parsed): + return parsed + if time.monotonic() > deadline: + raise AssertionError( + f"SSE hook captured no `{terminal}` frame (frames so far: " + f"{len(parsed)}) — hook install failed?" + ) + time.sleep(0.05) + + +def _stored(page: Page) -> dict: + """The persisted ``bor.chat.v1`` conversation (localStorage — the + phase-14 store the restore reads on refresh).""" + raw = page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')") + assert raw is not None, "the conversation key must exist in localStorage" + return json.loads(raw) + + +def _submit(page: Page, question: str) -> None: + page.fill("#message-input", question) + page.click("#send-btn") + # The user bubble lands synchronously with the submit handler. + expect(page.locator(".msg.user .bubble").last).to_contain_text(question) + + +def _wait_settled(page: Page) -> None: + """The turn is complete: answer text in the bubble, button + recovered (the phase-48 settle convention).""" + expect(page.locator(".msg.brain .bubble").last).not_to_have_text( + "", timeout=30_000 + ) + expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000) + expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000) + + +def _mock_capture_reset(mock_port: int) -> None: + """Clear the mock's request capture so a suite asserts on + EXACTLY the requests of the turn it is about to drive (the + phase-123 capture pair).""" + r = httpx.post( + f"http://127.0.0.1:{mock_port}/v1/e2e/captured/reset", timeout=5 + ) + assert r.status_code == 200 + + +def _mock_captured(mock_port: int) -> list[dict]: + """The mock's recorded chat-completions request bodies (oldest + first) — the wire-level pin surface for test 1.""" + r = httpx.get( + f"http://127.0.0.1:{mock_port}/v1/e2e/captured", timeout=5 + ) + assert r.status_code == 200 + return r.json() + + +def _tool_msgs(messages: list[dict]) -> list[str]: + """The ``tool``-role result contents of one request, in order.""" + return [ + str(m.get("content") or "") + for m in messages + if m.get("role") == "tool" + ] + + +# -------------------------------------------------------------------------- +# 1. The budget trailer on every executed tool result (wire level) +# -------------------------------------------------------------------------- + + +def test_tool_results_carry_the_remaining_count( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _mock_capture_reset(mock_llm) + + _submit(page, CAP_QUESTION) + _wait_settled(page) + + # Wire level (the mock's recorded request log — the app's + # EXACT requests): exactly four chat requests — three executed + # tool rounds + the forced final. + bodies = _mock_captured(mock_llm) + assert len(bodies) == 4, len(bodies) + + # Request 1: no tool results yet; the tools are offered. + assert bodies[0].get("tools"), "request 1 must offer the tools" + assert _tool_msgs(bodies[0]["messages"]) == [] + + # Round 1's result (in request 2): the decreasing trailer — + # ``[tool calls remaining: 2]`` (two of three left after the + # call it is reading). + msgs2 = _tool_msgs(bodies[1]["messages"]) + assert len(msgs2) == 1, msgs2 + assert msgs2[0].endswith(TOOL_BUDGET_REMAINING.format(n=2)), msgs2[0] + + # Round 2's result (in request 3): the last-call line (one left + # — "make it count, then answer"). + msgs3 = _tool_msgs(bodies[2]["messages"]) + assert len(msgs3) == 2, msgs3 + assert msgs3[0].endswith(TOOL_BUDGET_REMAINING.format(n=2)), msgs3[0] + assert msgs3[1].endswith(TOOL_BUDGET_LAST), msgs3[1] + + # The boundary call's result (in request 4 — the forced final): + # the SPENT line (the budget is gone; say "continue"). + msgs4 = _tool_msgs(bodies[3]["messages"]) + assert len(msgs4) == 3, msgs4 + assert msgs4[2].endswith(TOOL_BUDGET_SPENT), msgs4[2] + + # The mock's incrementing ids (the flow's stateless + # derivation): call_cap_1 → call_cap_2 → call_cap_3 in the + # assistant messages. + asst_ids = [ + bodies[i + 1]["messages"][2 * i + 2]["tool_calls"][0]["id"] + for i in range(3) + ] + assert asst_ids == ["call_cap_1", "call_cap_2", "call_cap_3"], asst_ids + + # The final request is the no-tools one, coached: no ``tools`` + # key (or null), the system message carries the wrap-up + # instruction, and the user question + the tool history stay in + # place. + assert bodies[3].get("tools") is None + systems = [ + str(m.get("content") or "") + for m in bodies[3]["messages"] + if m.get("role") == "system" + ] + assert any(ROUND_CAP_WRAPUP_INSTRUCTION in s for s in systems), systems + users = [ + str(m.get("content") or "") + for m in bodies[3]["messages"] + if m.get("role") == "user" + ] + assert CAP_QUESTION in users, users + + +# -------------------------------------------------------------------------- +# 2. Past the cap: zero tool lines after the cap, the fallback +# answer, no error +# -------------------------------------------------------------------------- + + +def test_capped_turn_never_shows_tool_lines_after_the_cap( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + _submit(page, CAP_QUESTION) + _wait_settled(page) + + frames = _frames(page) + + # The turn settles with ``done`` — never the terminal ``error`` + # frame (the pre-phase-160 cap path's MalformedReplyError class + # is gone; the budget fallback is the terminal). + assert not [f for f in frames if f.get("type") == "error"], frames + assert frames[-1]["type"] == "done" + + # Wire: EXACTLY three ``tool`` frames — one per executed round + # (the cap). The dump call (the forced final's structured + # ``read`` on the seed path) produces NO frame — it was + # rejected (never executed, never shown). + tool_frames = [f for f in frames if f.get("type") == "tool"] + assert len(tool_frames) == MAX_ROUNDS, tool_frames + assert all(f["name"] == "ls" for f in tool_frames), tool_frames + assert not [f for f in tool_frames if f.get("argument") == SEED_SP] + + # UI: exactly three tool lines (all the no-arg top-level + # listings); the dump call's ``Reading `` line is + # NOWHERE — the rejected call never renders. + lines = page.locator(".msg.brain .tool-call") + expect(lines).to_have_count(3) + for i in range(3): + expect(lines.nth(i)).to_contain_text("Listing documents") + msg_text = page.locator(".msg.brain").last.text_content() or "" + assert "Reading " not in msg_text, msg_text + + # The answer: the deterministic backstop line (the mock streams + # no content on the forced final — only the rejected dump + # call) — non-empty and readable. + bubble = page.locator(".msg.brain .bubble").last + expect(bubble).to_contain_text(ROUND_CAP_FALLBACK_ANSWER) + + # No error banner (the turn settled, never the failure path). + expect(page.locator("#kb-banner")).to_be_hidden() + + +# -------------------------------------------------------------------------- +# 3. done.budget_exhausted: true exactly on the capped turn, ABSENT +# (never false) on the normal grounded control +# -------------------------------------------------------------------------- + + +def test_done_flags_budget_exhausted( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + # The capped turn: ``done`` carries ``budget_exhausted: true``. + _submit(page, CAP_QUESTION) + _wait_settled(page) + frames = _frames(page) + done = next(f for f in frames if f.get("type") == "done") + assert done["budget_exhausted"] is True, done + assert done["deflected"] is False + # The flagged answer offers the Continue affordance (task 04). + expect(page.locator(".continue-btn")).to_have_count(1) + + # The normal grounded control: the ``done`` frame carries NO + # ``budget_exhausted`` key at all (absent, never false — the + # byte-identical off rule, PLAN §13). + page.evaluate("() => { window.__sseFrames = []; }") + _submit(page, CONTROL_QUESTION) + _wait_settled(page) + frames = _frames(page) + done = next(f for f in frames if f.get("type") == "done") + assert "budget_exhausted" not in done, done + assert done["deflected"] is False + + # The newer turn's landing cleared the capped answer's Continue + # button (the remove-first pattern — a stale "continue" would + # append to a history the conversation moved past). + expect(page.locator(".continue-btn")).to_have_count(0) + + +# -------------------------------------------------------------------------- +# 4. Continue researching: one fresh turn, the real continuation +# answer, the record shape +# -------------------------------------------------------------------------- + + +def test_continue_button_runs_a_fresh_turn( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + _submit(page, CAP_QUESTION) + _wait_settled(page) + expect(page.locator(".msg.brain .bubble").last).to_contain_text( + ROUND_CAP_FALLBACK_ANSWER + ) + + # The flagged answer offers **Continue researching**. + btn = page.locator(".continue-btn") + expect(btn).to_have_count(1) + expect(btn).to_contain_text("Continue researching") + + # Fresh capture window: the continuation turn's frames only + # (the cap turn's frames are still in the hook's buffer). + page.evaluate("() => { window.__sseFrames = []; }") + btn.click() + + # Exactly ONE ``continue`` user bubble (the plain send). + expect(page.locator(".msg.user .bubble")).to_have_count(2) + expect(page.locator(".msg.user .bubble").last).to_have_text("continue") + + # … and the mock's REAL continuation answer (a fixed sentence + # naming the seed document — not a tool call). + bubble = page.locator(".msg.brain .bubble").last + expect(bubble).to_contain_text(ROUND_CAP_CONTINUE_ANSWER, timeout=30_000) + _wait_settled(page) + + # The continuation turn streams NO tool frames — the earlier + # tool lines do not duplicate (still exactly three). + frames = _frames(page) + assert [f for f in frames if f.get("type") == "tool"] == [] + expect(page.locator(".msg.brain .tool-call")).to_have_count(3) + done = next(f for f in frames if f.get("type") == "done") + # A FRESH turn: the budget is whole again — the flag is absent + # (the continuation turn deflects — the literal "continue" + # shares no tokens with the KB, so the A8 gate is LOW). + assert "budget_exhausted" not in done, done + assert done["deflected"] is True + + # The record: [user, brain(flagged), user("continue"), brain]. + stored = _stored(page) + assert [m["who"] for m in stored["messages"]] == [ + "user", + "brain", + "user", + "brain", + ] + assert stored["messages"][1]["budget_exhausted"] is True + assert stored["messages"][1]["text"] == ROUND_CAP_FALLBACK_ANSWER + assert stored["messages"][2]["text"] == "continue" + assert ROUND_CAP_CONTINUE_ANSWER in stored["messages"][3]["text"] + assert not stored["messages"][3].get("budget_exhausted") + + +# -------------------------------------------------------------------------- +# 5. Single-shot: double-clicking Continue starts ONE turn +# -------------------------------------------------------------------------- + + +def test_continue_is_inert_while_streaming_and_single_shot( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + _submit(page, CAP_QUESTION) + _wait_settled(page) + expect(page.locator(".continue-btn")).to_have_count(1) + + # Count the ``/api/chat`` POSTs (the single-shot pin) — the + # exact path (the auto-save's ``PUT /api/chats/…`` must not + # match: ``/api/chats`` is not ``/api/chat``). + posts: list[int] = [] + page.on( + "request", + lambda req: ( + posts.append(1) + if req.method == "POST" + and req.url.split("?", 1)[0].endswith("/api/chat") + else None + ), + ) + + # The double-click: the FIRST click runs the handler (removes + # every ``.continue-btn`` and starts the turn); the SECOND + # click lands after the removal — no second target (and the + # in-flight no-op guard would swallow it anyway) — ONE turn. + page.locator(".continue-btn").dblclick() + _wait_settled(page) + assert len(posts) == 1, posts + + # ONE ``continue`` user bubble + ONE continuation answer (no + # double-send: a second turn would add a second user bubble + + # a second brain answer). + expect(page.locator(".msg.user .bubble")).to_have_count(2) + expect(page.locator(".msg.brain")).to_have_count(2) + bubble = page.locator(".msg.brain .bubble").last + expect(bubble).to_contain_text(ROUND_CAP_CONTINUE_ANSWER, timeout=30_000) + + +# -------------------------------------------------------------------------- +# 6. The flag survives a refresh (the restore path) +# -------------------------------------------------------------------------- + + +def test_flag_survives_a_refresh( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + _submit(page, CAP_QUESTION) + _wait_settled(page) + expect(page.locator(".msg.brain .bubble").last).to_contain_text( + ROUND_CAP_FALLBACK_ANSWER + ) + expect(page.locator(".continue-btn")).to_have_count(1) + + # The refresh: the flagged answer restores WITH the Continue + # button (the restore path keys on ``m.budget_exhausted``). + page.reload() + expect(page.locator("#messages > .msg")).to_have_count(2) + expect(page.locator(".msg.user .bubble")).to_have_text(CAP_QUESTION) + bubble = page.locator(".msg.brain .bubble").last + expect(bubble).to_contain_text(ROUND_CAP_FALLBACK_ANSWER, timeout=30_000) + btn = page.locator(".continue-btn") + expect(btn).to_have_count(1) + expect(btn).to_contain_text("Continue researching") + + # The restored button WORKS: clicking it runs the continuation + # turn (the mock's continuation answer streams). + btn.click() + expect(page.locator(".msg.user .bubble")).to_have_count(2) + expect(page.locator(".msg.brain .bubble").last).to_contain_text( + ROUND_CAP_CONTINUE_ANSWER, timeout=30_000 + ) + _wait_settled(page) + + # The record: [user, brain(flagged), user("continue"), brain]. + stored = _stored(page) + assert [m["who"] for m in stored["messages"]] == [ + "user", + "brain", + "user", + "brain", + ] + assert stored["messages"][1]["budget_exhausted"] is True + assert stored["messages"][2]["text"] == "continue" + assert ROUND_CAP_CONTINUE_ANSWER in stored["messages"][3]["text"] + + +# -------------------------------------------------------------------------- +# 7. Negative: a deflected turn — no flag, no Continue button +# -------------------------------------------------------------------------- + + +def test_deflected_turn_gets_no_continue_button( + page: Page, app_url: str, mock_llm: int, db_ready: None +) -> None: + _reset_db() + page.set_default_timeout(30_000) + login(page, app_url, next="/") + _install_sse_hook(page) + + _submit(page, DEFLECT_QUESTION) + _wait_settled(page) + + frames = _frames(page) + done = next(f for f in frames if f.get("type") == "done") + # Settled DEFLECTED (A8 LOW), with NO ``budget_exhausted`` key + # (the flag is cap-reached turns only) and NO ``tool`` frame + # (a tools=None request never emits one). + assert done["deflected"] is True, done + assert "budget_exhausted" not in done, done + assert [f for f in frames if f.get("type") == "tool"] == [] + + # The deflected bubble — the honest copy, and NO Continue + # button (the affordance is cap-reached answers only). + last = page.locator(".msg.brain").last + expect(last).to_have_class(re.compile(r"is-deflected")) + expect(page.locator(".msg.brain .tool-call")).to_have_count(0) + expect(page.locator(".continue-btn")).to_have_count(0) + expect(page.locator(".msg.brain .bubble").last).to_contain_text( + "I haven't done anything like that" + ) diff --git a/tests/integration/test_agent_tools.py b/tests/integration/test_agent_tools.py index d623df8..8b20799 100644 --- a/tests/integration/test_agent_tools.py +++ b/tests/integration/test_agent_tools.py @@ -320,13 +320,13 @@ def test_ls_top_level_lists_registered_sources_through_run_agent( holder, llm = _run_call(db, "ls", {}) assert llm.requests[0][1] == AGENT_TOOLS - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "2 sources:\n" "\n" "Deployments — 1 documents\n" "Homelab — 2 documents\n" " Home lab notes." - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] @@ -352,7 +352,7 @@ def test_ls_top_level_empty_registry_through_run_agent( db.commit() holder, llm = _run_call(db, "ls", {}) - assert llm.requests[1][0][3]["content"] == "0 sources:" + assert llm.requests[1][0][3]["content"].startswith("0 sources:") assert holder.tool_calls == 1 @@ -375,20 +375,20 @@ def test_ls_source_scope_lists_root_folder_through_run_agent(kb, registry, db) - holder, llm = _run_call(db, "ls", {"path": "Homelab"}) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Homelab — 1 documents, 2 folders:\n" "\n" " backups/ — 2 documents: Backup notes.\n" " networking/ — 1 documents\n" "\n" "source: Homelab | path: readme.md | title: Readme | date: 2024-06-15" - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] # A registered source with no documents: the header line alone. holder0, llm0 = _run_call(db, "ls", {"path": "Deployments"}) - assert llm0.requests[1][0][3]["content"] == "Deployments — 0 documents, 0 folders:" + assert llm0.requests[1][0][3]["content"].startswith("Deployments — 0 documents, 0 folders:") assert holder0.tool_calls == 1 @@ -405,23 +405,23 @@ def test_ls_nested_folder_scope_drills_one_level_through_run_agent( holder, llm = _run_call(db, "ls", {"path": "Homelab/networking"}) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Homelab/networking — 0 documents, 2 folders:\n" "\n" " networking/lan/ — 2 documents\n" " networking/vpn/ — 1 documents" - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] # One level deeper. holder2, llm2 = _run_call(db, "ls", {"path": "Homelab/networking/lan"}) - assert llm2.requests[1][0][3]["content"] == ( + assert llm2.requests[1][0][3]["content"].startswith( "Homelab/networking/lan — 2 documents, 0 folders:\n" "\n" "source: Homelab | path: networking/lan/a.md | title: A | date: 2024-06-15\n" "source: Homelab | path: networking/lan/b.md | title: B | date: 2024-06-15" - ) + ) assert holder2.tool_calls == 1 @@ -434,7 +434,9 @@ def test_ls_folder_file_cap_through_run_agent(kb, registry, db) -> None: holder, llm = _run_call(db, "ls", {"path": "Homelab/big"}) - content = llm.requests[1][0][3]["content"] + # Phase 160 (task 01): the budget-telemetry trailer rides the + # listing — the raw listing is the RAW part (the trailer stripped). + content = llm.requests[1][0][3]["content"].removesuffix(agent._budget_note(9)) lines = content.splitlines() assert lines[0] == "Homelab/big — 51 documents, 0 folders:" assert lines[2] == ( @@ -460,9 +462,8 @@ def test_ls_scoped_unknown_source_refused_through_run_agent(kb, src, db) -> None holder, llm = _run_call(db, "ls", {"path": "Ghost"}) - assert ( - llm.requests[1][0][3]["content"] - == agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost") + assert llm.requests[1][0][3]["content"].startswith( + agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost") ) assert holder.tool_calls == 0 and holder.read_docs == [] @@ -477,9 +478,8 @@ def test_ls_path_like_scope_unknown_source_gets_no_source_refusal(kb, src, db) - holder, llm = _run_call(db, "ls", {"path": "app/rag/importer.py"}) - assert ( - llm.requests[1][0][3]["content"] - == agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app") + assert llm.requests[1][0][3]["content"].startswith( + agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app") ) assert holder.tool_calls == 0 and holder.read_docs == [] assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -500,10 +500,10 @@ def test_ls_unknown_folder_gets_not_a_folder_with_parents_subfolders( holder, llm = _run_call(db, "ls", {"path": "Homelab/netwoking"}) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "'Homelab/netwoking' is not a folder — Homelab has: " "backups/ containers/ networking/" - ) + ) assert holder.tool_calls == 0 and holder.read_docs == [] assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -523,11 +523,11 @@ def test_read_combined_path_through_run_agent(kb, db) -> None: # Phase 106 (D5): the date rides every read — the SECOND line (the # first line stays the byte-identical header). - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document Alpha/deep/nested/doc.md:\n" "date: 2024-06-15\n" "FULL-TEXT" - ) + ) assert holder.tool_calls == 1 # SEC-14-04: short-lived session loads fresh copies assert [d.id for d in holder.read_docs] == [created.id] @@ -542,8 +542,8 @@ def test_read_bare_source_name_refused_through_run_agent(kb, db) -> None: holder, llm = _run_call(db, "read", {"path": "Alpha"}) - assert ( - llm.requests[1][0][3]["content"] == "No document at 'Alpha' — check the ls output." + assert llm.requests[1][0][3]["content"].startswith( + "No document at 'Alpha' — check the ls output." ) assert holder.tool_calls == 0 and holder.read_docs == [] @@ -558,9 +558,8 @@ def test_read_unknown_combined_path_refused_through_run_agent(kb, db) -> None: holder, llm = _run_call(db, "read", {"path": "Alpha/nope/deep.md"}) - assert ( - llm.requests[1][0][3]["content"] - == "No document at 'Alpha/nope/deep.md' — check the ls output." + assert llm.requests[1][0][3]["content"].startswith( + "No document at 'Alpha/nope/deep.md' — check the ls output." ) assert holder.tool_calls == 0 and holder.read_docs == [] @@ -592,19 +591,19 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) -> # Round 1: the bare path resolves to no combined identity, but it IS # the indexed document's path — the refusal names the one combined # identity to use (not counted, the tools stay offered). - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'deep/nested/doc.md' — " "did you mean 'Alpha/deep/nested/doc.md'?" - ) + ) assert llm.requests[1][1] == AGENT_TOOLS # Round 2: the corrected combined identity succeeds — the full # content (plus the phase-106 D5 date line), the holder records the # row, and it counts. - assert llm.requests[2][0][5]["content"] == ( + assert llm.requests[2][0][5]["content"].startswith( "Document Alpha/deep/nested/doc.md:\n" "date: 2024-06-15\n" "FULL-TEXT" - ) + ) assert llm.requests[2][1] == AGENT_TOOLS # SEC-14-04: short-lived session loads fresh copies assert [d.id for d in holder.read_docs] == [created.id] @@ -633,13 +632,13 @@ def test_read_bare_path_two_sources_one_of_suggestion_then_corrected_read( holder = AgentHolder() asyncio.run(_consume(cast("LLMClient", llm), lambda: SessionLocal(), holder)) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'shared/x.md' — did you mean one of: " "'Alpha/shared/x.md', 'Beta/shared/x.md'?" - ) - assert llm.requests[2][0][5]["content"] == ( + ) + assert llm.requests[2][0][5]["content"].startswith( "Document Alpha/shared/x.md:\ndate: 2024-06-15\nA-TEXT" - ) + ) # SEC-14-04: short-lived session loads fresh copies assert [d.id for d in holder.read_docs] == [a.id] assert holder.tool_calls == 1 # only the corrected read executed @@ -673,10 +672,10 @@ def test_grep_whole_kb_through_run_agent(kb, db) -> None: # Offered: the first request carries AGENT_TOOLS (the 3-tool list). assert llm.requests[0][1] == AGENT_TOOLS # Executed against the real DB: catalog order, grep-style lines. - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Alpha/a/one.md:2: needle in one\n" "Beta/b/two.md:2: NEEDLE in two" - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] # locked A5: grep adds no context @@ -689,7 +688,7 @@ def test_grep_scoped_through_run_agent(kb, db) -> None: holder, llm = _run_call(db, "grep", {"pattern": "needle", "path": "Alpha/a/one.md"}) # Only the named document is searched — the other one's hit is absent. - assert llm.requests[1][0][3]["content"] == "Alpha/a/one.md:2: Needle here" + assert llm.requests[1][0][3]["content"].startswith("Alpha/a/one.md:2: Needle here") assert holder.tool_calls == 1 assert holder.read_docs == [] @@ -703,9 +702,8 @@ def test_grep_scoped_missing_doc_refused_through_run_agent(kb, db) -> None: holder, llm = _run_call(db, "grep", {"pattern": "needle", "path": "Alpha/ghost.md"}) - assert ( - llm.requests[1][0][3]["content"] - == "No document at 'Alpha/ghost.md' — check the ls output." + assert llm.requests[1][0][3]["content"].startswith( + "No document at 'Alpha/ghost.md' — check the ls output." ) assert holder.tool_calls == 0 and holder.read_docs == [] @@ -716,8 +714,8 @@ def test_grep_no_matches_through_run_agent(kb, db) -> None: holder, llm = _run_call(db, "grep", {"pattern": "zebra"}) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No matches for 'zebra' in the knowledge base." - ) + ) assert holder.tool_calls == 1 # an executed grep with zero hits assert holder.read_docs == [] diff --git a/tests/integration/test_agent_tools_dates.py b/tests/integration/test_agent_tools_dates.py index 559dce4..69e41e3 100644 --- a/tests/integration/test_agent_tools_dates.py +++ b/tests/integration/test_agent_tools_dates.py @@ -28,7 +28,7 @@ from sqlalchemy.orm import Session from app.config import Settings from app.db import SessionLocal from app.models import Document, GitSource -from app.rag.agent import AgentHolder, run_agent +from app.rag.agent import AgentHolder, _budget_note, run_agent from app.rag.llm import LLMClient, RetryPiece, StreamPiece, ToolCallPiece, ToolResultPiece if TYPE_CHECKING: @@ -160,7 +160,9 @@ def test_read_result_second_line_is_stored_date(kb, db) -> None: holder, llm = _run_call(db, "read", {"path": "Alpha/deep/nested/doc.md"}) - content = llm.requests[1][0][3]["content"] + # Phase 160 (task 01): the trailer rides the read result — the raw + # result is the RAW part (the trailer stripped). + content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9)) lines = content.splitlines() assert lines[0] == "Document Alpha/deep/nested/doc.md:" # byte-identical header assert lines[1] == f"date: {D2_STR}" # the STORED date (row's UTC date part) @@ -178,13 +180,13 @@ def test_read_result_date_is_the_row_date_not_a_constant(kb, db) -> None: db.commit() holder_a, llm_a = _run_call(db, "read", {"path": "Alpha/a.md"}) - assert llm_a.requests[1][0][3]["content"] == ( + assert llm_a.requests[1][0][3]["content"].startswith( f"Document Alpha/a.md:\ndate: {D1_STR}\nA-TEXT" - ) + ) holder_b, llm_b = _run_call(db, "read", {"path": "Alpha/b.md"}) - assert llm_b.requests[1][0][3]["content"] == ( + assert llm_b.requests[1][0][3]["content"].startswith( f"Document Alpha/b.md:\ndate: {D3_STR}\nB-TEXT" - ) + ) # SEC-14-04: short-lived sessions assert [d.id for d in holder_a.read_docs] == [a.id] assert [d.id for d in holder_b.read_docs] == [b.id] @@ -207,7 +209,9 @@ def test_ls_drill_file_lines_carry_their_dates(kb, src, db) -> None: holder, llm = _run_call(db, "ls", {"path": "Homelab"}) - content = llm.requests[1][0][3]["content"] + # Phase 160 (task 01): the trailer rides the listing — stripped for + # the raw-part pins below. + content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9)) lines = content.splitlines() # The root level: one direct file (readme.md — D1) + the two # subfolder lines (date-free) + the date-free header. @@ -221,7 +225,7 @@ def test_ls_drill_file_lines_carry_their_dates(kb, src, db) -> None: # Drill into backups: BOTH files list, each with its OWN date. holder2, llm2 = _run_call(db, "ls", {"path": "Homelab/backups"}) - lines2 = llm2.requests[1][0][3]["content"].splitlines() + lines2 = llm2.requests[1][0][3]["content"].removesuffix(_budget_note(9)).splitlines() assert lines2[0] == "Homelab/backups — 2 documents, 0 folders:" assert lines2[2] == ( f"source: Homelab | path: backups/cron.md | title: Cron | date: {D1_STR}" @@ -247,7 +251,7 @@ def test_ls_top_level_source_lines_carry_no_date(kb, db) -> None: holder, llm = _run_call(db, "ls", {}) - content = llm.requests[1][0][3]["content"] + content = llm.requests[1][0][3]["content"].removesuffix(_budget_note(9)) assert content == "1 sources:\n\nHomelab — 1 documents" assert "date" not in content assert holder.tool_calls == 1 diff --git a/tests/integration/test_chat_api.py b/tests/integration/test_chat_api.py index 747815f..77e5468 100644 --- a/tests/integration/test_chat_api.py +++ b/tests/integration/test_chat_api.py @@ -82,6 +82,7 @@ class FakeRagLLM: stream_error: Exception | None = None, fail_mid_stream: bool = False, tool_script: list[list[StreamPiece | ToolCallPiece]] | None = None, + no_tools_script: list[list[StreamPiece | ToolCallPiece]] | None = None, embed_fail_count: int = 0, stream_fail_count: int = 0, answer_sequence: list[str] | None = None, @@ -123,6 +124,18 @@ class FakeRagLLM: #: thinking + answer stream below, so a deflected turn through this #: fake is byte-identical to the plain fake's output. self.tool_script: list[list[StreamPiece | ToolCallPiece]] = list(tool_script or []) + #: Phase 160 (task 02): canned piece lists for requests made + #: WITHOUT tools (``tools=None``) — the deflected turn, the + #: cap-forced final, the kill-switch single-request path, and the + #: phase-71 recovery. Empty (the default) keeps the pre-phase + #: behavior: a tools=None request yields the thinking + answer + #: stream below, so every existing test is untouched. When set, + #: the *i*-th tools=None request consumes ``no_tools_script[i]`` + #: (so a mock can make a tools=None request return structured + #: ``ToolCallPiece``s — the documented ``lite`` behavior). + self.no_tools_script: list[list[StreamPiece | ToolCallPiece]] = list( + no_tools_script or [] + ) async def embed( self, @@ -197,6 +210,8 @@ class FakeRagLLM: raw: list[StreamPiece | ToolCallPiece] if tools is not None and self.tool_script: raw = self.tool_script.pop(0) + elif tools is None and self.no_tools_script: + raw = self.no_tools_script.pop(0) elif self.fail_mid_stream: raw = [StreamPiece("content", "partial ")] mid_stream_drop = True @@ -980,9 +995,10 @@ marker in the model's context.""" m for r in scripted.seen_messages for m in r if m.get("role") == "tool" ] assert tool_msgs, "the executed read must be appended as a tool message" - assert ( - tool_msgs[-1]["content"] - == "Document docs/fits.md:\ndate: 2024-06-15\n" + content + # Phase 160 (task 01): the budget-telemetry trailer rides the + # read result — the raw result is the RAW part. + assert tool_msgs[-1]["content"].startswith( + "Document docs/fits.md:\ndate: 2024-06-15\n" + content ) assert TRUNCATION_MARKER not in tool_msgs[-1]["content"] # Still a successful read. @@ -1165,9 +1181,10 @@ def test_untruncated_read_streams_no_tool_result_frame( m for r in scripted.seen_messages for m in r if m.get("role") == "tool" ] assert tool_msgs - assert ( - tool_msgs[-1]["content"] - == "Document docs/big-read.md:\ndate: 2024-06-15\n" + content + # Phase 160 (task 01): the budget-telemetry trailer rides the + # read result — the raw result is the RAW part. + assert tool_msgs[-1]["content"].startswith( + "Document docs/big-read.md:\ndate: 2024-06-15\n" + content ) assert TRUNCATION_MARKER not in tool_msgs[-1]["content"] @@ -1500,6 +1517,176 @@ def test_zero_max_rounds_reproduce_pre_phase_single_request( assert "scaffold_stripped=0" in lines[-1] # phase 71: uniform clean-turn field +def test_cap_path_rejects_late_calls_and_fallbacks( + client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch, + caplog: pytest.LogCaptureFixture, +) -> None: + """A grounded turn with ``agent_max_rounds=1`` whose forced final + (``tools=None``) returns a ``ToolCallPiece`` (the documented + ``lite`` behavior — a no-tools request that still comes back with + structured calls): the executed round streams ``tool`` ×1, then + NOTHING of the sort — no second ``tool`` frame, no ``error`` frame — + and the turn settles with ``done`` whose delta text is the + ``ROUND_CAP_FALLBACK_ANSWER`` (the model gave no visible content on + the forced final). The late call is never executed and never counted + in ``tool_calls`` (the executed-only log field stays 1).""" + scripted = FakeRagLLM( + tool_script=[[ToolCallPiece(id="call_1", name="ls", arguments={})]], + no_tools_script=[ + [ + ToolCallPiece( + id="call_f1", + name="read", + arguments={"path": "docs/homelab/kubernetes.md"}, + ) + ] + ], + ) + live = get_settings() + monkeypatch.setattr( + chat_api, + "get_settings", + lambda: Settings( + _env_file=None, # pyright: ignore[reportCallIssue] + relevance_threshold=live.relevance_threshold, + agent_max_rounds=1, + ), + ) + fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted + try: + caplog.set_level(logging.INFO, logger="app.chat") + _, _, frames = _stream_chat(client, QUESTION) + finally: + fastapi_app.dependency_overrides.clear() + types = [f["type"] for f in frames] + # Only the executed round 1 call streams as a tool frame; the forced + # final's call is rejected — no second tool frame, no error frame. + assert types.count("tool") == 1 + assert "error" not in types + # The turn settles with done; the answer is the budget fallback. + done = frames[-1] + assert done["type"] == "done" + assert done["deflected"] is False + delta_text = "".join(f.get("text", "") for f in frames if f["type"] == "delta") + assert delta_text == agent.ROUND_CAP_FALLBACK_ANSWER + # The late ``read`` was never executed: ``tool_calls=1`` (the + # executed round only) in the per-turn log line. + lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()] + assert lines and "tool_calls=1" in lines[-1] + + +def test_cap_reached_turn_reports_budget_exhausted_and_log_fields( + client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch, + caplog: pytest.LogCaptureFixture, +) -> None: + """A grounded turn with ``agent_max_rounds=1`` that hits the cap: + the ``done`` frame carries ``"budget_exhausted": true`` (the + additive key — the ``data:`` line the client sees) and the + per-turn log line reads ``rounds=1 cap_hit=1`` (the cap hit, + named — the owner can grep how often the cap bites).""" + scripted = FakeRagLLM( + tool_script=[[ToolCallPiece(id="call_1", name="ls", arguments={})]], + no_tools_script=[[StreamPiece("content", "the wrap-up answer")]], + ) + live = get_settings() + monkeypatch.setattr( + chat_api, + "get_settings", + lambda: Settings( + _env_file=None, # pyright: ignore[reportCallIssue] + relevance_threshold=live.relevance_threshold, + agent_max_rounds=1, + ), + ) + fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted + try: + caplog.set_level(logging.INFO, logger="app.chat") + _, _, frames = _stream_chat(client, QUESTION) + finally: + fastapi_app.dependency_overrides.clear() + done = frames[-1] + assert done["type"] == "done" + assert done["deflected"] is False + assert done["budget_exhausted"] is True # the cap-reached flag + assert "error" not in [f["type"] for f in frames] + lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()] + assert lines and "rounds=1 cap_hit=1" in lines[-1] + assert "tool_calls=1" in lines[-1] + + +def test_in_cap_and_deflected_turns_report_no_budget_exhausted( + client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture +) -> None: + """An ordinary grounded turn (answered in-cap) and a deflected + turn: the ``done`` frames carry NO ``budget_exhausted`` key + (ABSENT, never ``false`` — the byte-identical-off contract), and + the per-turn log lines read ``cap_hit=0`` — the grounded one + with ``rounds=`` the executed rounds (2), the deflected one + ``rounds=0 cap_hit=0`` (the agent never runs).""" + scripted = FakeRagLLM( + tool_script=[ + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [ToolCallPiece(id="call_2", name="ls", arguments={})], + ] + ) + fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted + try: + caplog.set_level(logging.INFO, logger="app.chat") + _, _, frames = _stream_chat(client, QUESTION) + _, _, deflected_frames = _stream_chat(client, OFF_TOPIC) + finally: + fastapi_app.dependency_overrides.clear() + done = frames[-1] + assert done["type"] == "done" + assert done["deflected"] is False + assert "budget_exhausted" not in done # ABSENT, never false + d_done = deflected_frames[-1] + assert d_done["type"] == "done" + assert d_done["deflected"] is True + assert "budget_exhausted" not in d_done + lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()] + assert len(lines) == 2 + assert "tool_calls=2 rounds=2 cap_hit=0" in lines[0] # the grounded turn + assert "rounds=0 cap_hit=0" in lines[1] # the deflected turn + + +def test_deflected_turn_rejects_structured_calls( + client, db, seeded_kb: FakeRagLLM +) -> None: + """A deflected turn (``tools=None``, A8) whose model returns a + structured ``tool_calls`` (the documented ``lite`` behavior) emits + NO ``tool`` frame — the call is rejected at the transport + (``_pump`` drops it, ``tools_offered=False``), never rendered, never + executed. The turn still settles (``done``) with the model's + content (the rejection only drops the call, not the answer).""" + scripted = FakeRagLLM( + no_tools_script=[ + [ + ToolCallPiece(id="call_1", name="ls", arguments={}), + StreamPiece("content", "the deflected answer"), + ] + ], + ) + fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted + try: + _, _, frames = _stream_chat(client, OFF_TOPIC) + finally: + fastapi_app.dependency_overrides.clear() + types = [f["type"] for f in frames] + assert not any(t == "tool" for t in types) # no tool frame + assert "error" not in types + done = frames[-1] + assert done["type"] == "done" + assert done["deflected"] is True + # The model's content still streamed (the rejection only drops the + # call, not the answer). + delta_text = "".join(f.get("text", "") for f in frames if f["type"] == "delta") + assert delta_text == "the deflected answer" + # One request, no tools key (the deflected direct path, A8). + assert len(scripted.seen_messages) == 1 + assert scripted.seen_tools == [None] + + def test_tool_execution_db_failure_yields_error_event( client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch ) -> None: diff --git a/tests/integration/test_chats_api.py b/tests/integration/test_chats_api.py index e27efba..4727740 100644 --- a/tests/integration/test_chats_api.py +++ b/tests/integration/test_chats_api.py @@ -232,6 +232,12 @@ def _expect(records: list[dict[str, Any]]) -> list[dict[str, Any]]: "stopped": m.get("stopped"), "failed": m.get("failed"), "error": m.get("error"), + # Phase 160 (task 04): the budget-exhausted marker — + # explicit null where the record does not carry it (the + # bor.chat.v1 shape is explicit-null for every optional + # key; a record WITH the key round-trips it byte- + # identically — the pins below). + "budget_exhausted": m.get("budget_exhausted"), } for m in records ] @@ -464,8 +470,10 @@ def test_create_round_trips_full_brain_record(admin_client: TestClient) -> None: assert r.status_code == 201 # The bor.chat.v1-shaped payload round-trips losslessly: every # optional key (sources/related/deflected/suggestions/thinking/ - # tools/stopped) survives identical. - assert r.json()["messages"][1] == FULL_BRAIN + # tools/stopped/failed/error/budget_exhausted — the phase-160 + # additive marker, explicit null where absent) survives identical + # (the _expect full stored shape). + assert r.json()["messages"][1] == _expect([_user(FIRST_QUESTION), FULL_BRAIN])[1] def test_create_rejects_empty_messages(admin_client: TestClient) -> None: @@ -1036,6 +1044,128 @@ def test_shared_serve_includes_the_brain_answer_image_ref( assert "answer_image" not in messages[0] +# --------------------------------------------------------------------------- +# Phase 160 (task 04, TODO L4): the budget-exhausted marker — the +# persisted answer that ran out of tool calls. The done-time +# auto-save (``rememberBrainTurn``) persists the done frame's +# ``budget_exhausted`` into the brain record's ``budget_exhausted`` +# key (the ``answer_image`` precedent — the field must exist so the +# ``extra="forbid"`` boundary does NOT 422 those saves; the +# phase-113 lesson, applied up front), and the restore path re- +# renders the "Continue researching" button from it (task 04 — the +# marker is a client restore hint only, never a server behavior; no +# migration: ``saved_chats.messages`` is raw JSONB, phase 50, and +# ``app/api/chats.py``'s logic is unchanged). These round-trips are +# that regression pin. +# --------------------------------------------------------------------------- + + +def test_create_round_trips_the_budget_exhausted_marker( + admin_client: TestClient, +) -> None: + """The phase-113 regression pin (the A2 quiet-failure path): a + brain record carrying ``budget_exhausted: true`` → + ``POST /api/chats`` is 201 (NOT a silent 422) and the admin GET + round-trips the record BYTE-FOR-BYTE through Postgres (the + ``_expect`` full bor.chat.v1 shape — the restore path re-renders + the Continue button from the marker). The user record carries the + key as an explicit null (it never did, and the stored shape is + explicit-null for every optional key).""" + brain = {**FULL_BRAIN, "budget_exhausted": True} + r = admin_client.post( + "/api/chats", + json={"messages": [_user(FIRST_QUESTION), brain]}, + ) + assert r.status_code == 201, f"expected 201, got {r.status_code}: {r.text}" + body = r.json() + assert body["messages"][1]["budget_exhausted"] is True + assert body["messages"][0]["budget_exhausted"] is None + + got = admin_client.get(f"/api/chats/{body['id']}") + assert got.status_code == 200 + assert got.json()["messages"] == _expect( + [_user(FIRST_QUESTION), {**FULL_BRAIN, "budget_exhausted": True}] + ) + + +def test_create_without_budget_exhausted_marker_round_trips_unchanged( + admin_client: TestClient, +) -> None: + """A record WITHOUT the marker (every non-cap turn, every pre- + phase record) round-trips with the key as an explicit null — the + stored shape never gains a value (the byte-identical contract, + the phase-50 precedent: the key is absent from the client's + ``bor.chat.v1`` record via the ``undefined``-drops-the-key + pattern, and explicit-null here keeps the restore path null-safe). + The ``FULL_BRAIN`` shape (no marker) is the pin: its stored and + served form is byte-identical to pre-phase-160 apart from the + one new null key.""" + r = admin_client.post( + "/api/chats", + json={"messages": [_user(FIRST_QUESTION), FULL_BRAIN]}, + ) + assert r.status_code == 201, r.text + body = r.json() + assert body["messages"][1]["budget_exhausted"] is None + got = admin_client.get(f"/api/chats/{body['id']}") + assert got.status_code == 200 + assert got.json()["messages"] == _expect([_user(FIRST_QUESTION), FULL_BRAIN]) + + +def test_shared_serve_includes_the_budget_exhausted_marker( + admin_client: TestClient, +) -> None: + """The shared view is faithful (the data is in the public + snapshot), but the control is NOT: the shared page + (``/shared/``, rendered server-side by ``shared.js`` — + never by ``app.js``, owner-locked zero interactive controls) + IGNORES the key by construction — no Continue control renders for + a flagged record (work item 8; pinned in + ``test_frontend_tool_budget.py::test_shared_page_never_shows_ + the_continue_control``). The user record stays with the explicit + null.""" + brain = {**FULL_BRAIN, "budget_exhausted": True} + r = admin_client.post( + "/api/chats", + json={"messages": [_user(FIRST_QUESTION), brain]}, + ) + assert r.status_code == 201, r.text + share_url = _share(admin_client, r.json()["id"])["share_url"] + + anon = TestClient(fastapi_app) # fresh jar: truly anonymous + got = anon.get(f"/api{share_url}") + assert got.status_code == 200 + messages = got.json()["messages"] + assert messages[1]["budget_exhausted"] is True # faithful snapshot + assert messages[0]["budget_exhausted"] is None + + +def test_put_round_trips_the_budget_exhausted_marker( + admin_client: TestClient, +) -> None: + """The re-Save upsert (the auto-save path the client's restore + relies on) round-trips the marker byte-identically too: a PUT + with the flagged record replaces the stored JSONB and the 200 + body + the GET both carry it (a refreshed chat page restores the + Continue affordance from the persisted record).""" + created = admin_client.post( + "/api/chats", + json={"messages": [_user(FIRST_QUESTION), FULL_BRAIN]}, + ).json() + brain = {**FULL_BRAIN, "budget_exhausted": True} + r = admin_client.put( + f"/api/chats/{created['id']}", + json={"messages": [_user(FIRST_QUESTION), brain]}, + ) + assert r.status_code == 200, r.text + assert r.json()["messages"][1]["budget_exhausted"] is True + got = admin_client.get(f"/api/chats/{created['id']}") + assert got.status_code == 200 + assert got.json()["messages"] == _expect( + [_user(FIRST_QUESTION), {**FULL_BRAIN, "budget_exhausted": True}] + ) + + # --------------------------------------------------------------------------- # Phase 159 (task 02): edit-and-resend — the SAVE shape. The client # edits the last question: the record is replaced in place, EVERY diff --git a/tests/integration/test_read_line_ranges_handbook.py b/tests/integration/test_read_line_ranges_handbook.py index d33dccf..424f7dd 100644 --- a/tests/integration/test_read_line_ranges_handbook.py +++ b/tests/integration/test_read_line_ranges_handbook.py @@ -443,7 +443,10 @@ def test_plain_read_truncates_then_range_read_answers_the_find_traps_question( assert isinstance(plain_result["content"], str) notice = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=len(doc.content)) assert plain_result["content"].startswith(f"Document {IDENTITY}:\n") - assert plain_result["content"].endswith(f"\n{TRUNCATION_MARKER}\n{notice}") + # Phase 160 (task 01): the budget-telemetry trailer rides the + # truncated result — the marker + notice end the RAW part. + raw = plain_result["content"].removesuffix(agent._budget_note(9)) + assert raw.endswith(f"\n{TRUNCATION_MARKER}\n{notice}") # The truncation surfaced to the API layer as a ToolResultPiece # (the SSE ``tool_result`` frame's wire value). truncations = [p for p in pieces if isinstance(p, ToolResultPiece)] diff --git a/tests/integration/test_read_range_dedupe_two_ranges.py b/tests/integration/test_read_range_dedupe_two_ranges.py index 46c1990..5b6c28a 100644 --- a/tests/integration/test_read_range_dedupe_two_ranges.py +++ b/tests/integration/test_read_range_dedupe_two_ranges.py @@ -411,7 +411,7 @@ def test_two_ranges_of_one_doc_both_reach_the_model(two_range_doc: Document) -> # the byte-identical line is now true). reread_result = llm.requests[3][0][7] assert reread_result["role"] == "tool" - assert reread_result["content"] == agent.ALREADY_IN_CONTEXT + assert reread_result["content"].startswith(agent.ALREADY_IN_CONTEXT) # Round 4: the answer echoes BOTH markers (the bug: pre-phase, # the echo could contain only marker A — range B's slice never @@ -485,7 +485,10 @@ def test_truncated_plain_read_then_range_b_proceeds_and_plain_reread_teaches( assert isinstance(plain_result["content"], str) notice = READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content)) assert plain_result["content"].startswith(f"Document {IDENTITY}:\n") - assert plain_result["content"].endswith(f"\n{TRUNCATION_MARKER}\n{notice}") + # Phase 160 (task 01): the budget-telemetry trailer rides the + # truncated result — the marker + notice end the RAW part. + raw = plain_result["content"].removesuffix(agent._budget_note(9)) + assert raw.endswith(f"\n{TRUNCATION_MARKER}\n{notice}") assert MARKER_A in plain_result["content"] truncations = [p for p in pieces if isinstance(p, ToolResultPiece)] assert [(p.argument, p.chars_shown, p.chars_total) for p in truncations] == [ @@ -512,12 +515,12 @@ def test_truncated_plain_read_then_range_b_proceeds_and_plain_reread_teaches( # resolved doc. reread_result = llm.requests[3][0][7] assert reread_result["role"] == "tool" - assert reread_result["content"] == agent.REREAD_PARTIAL.format( + assert reread_result["content"].startswith(agent.REREAD_PARTIAL.format( covered=f"1–{covered}, {marker_b_line - 10}–{marker_b_line - 10 + RANGE_B_LIMIT - 1}", source=SOURCE, path=DOC_PATH, total=total_lines, - ) + )) # Round 4: the answer echoes both markers. answer = _answer(pieces) diff --git a/tests/unit/test_agent.py b/tests/unit/test_agent.py index 872e08b..ef8b33a 100644 --- a/tests/unit/test_agent.py +++ b/tests/unit/test_agent.py @@ -40,8 +40,13 @@ length tuple ``chars_total``), the phase-68 ``grep`` contract under its new name (the locked A5 pins: fixed substring, case-insensitive, 20-cap in catalog order, 200-char lines, locator-only — ``read_docs`` untouched, no-match lines counted), the round cap -forcing a final no-tools answer, the kill switch -(``agent_max_rounds=0`` single-call path), and the phase-67 per-round +forcing a final no-tools answer, the phase-160 budget-telemetry +trailer on every tool result (the ``[tool calls remaining: N]`` line +decreasing — the last-call line, then the spent line; a rejection's +refusal carries it too; the pure ``_budget_note`` boundary table), the +kill switch +(``agent_max_rounds=0`` single-call path, no ``role: "tool"`` message +at all — no trailer), and the phase-67 per-round retries (a dead-then-recovered round restarts before its first piece with a ``RetryPiece``; a mid-stream drop stays terminal — locked A2; the forced final no-tools call retries too; ``llm_retries=0`` is one @@ -564,11 +569,14 @@ def test_ls_then_read_then_answer(monkeypatch: pytest.MonkeyPatch) -> None: } ], } - assert msgs[3] == { - "role": "tool", - "tool_call_id": "call_1", - "content": "2 sources:\n\nDeployments — 1 documents\nHomelab — 1 documents", - } + assert msgs[3]["role"] == "tool" + assert msgs[3]["tool_call_id"] == "call_1" + # Phase 160 (task 01): the budget-telemetry trailer rides the + # result — round 1 of a 10-round budget reads 9 left. + assert msgs[3]["content"] == ( + "2 sources:\n\nDeployments — 1 documents\nHomelab — 1 documents" + + agent.TOOL_BUDGET_REMAINING.format(n=9) + ) # The second follow-up request carries the read call + the FULL text. msgs = llm.requests[2][0] assert msgs[4]["role"] == "assistant" @@ -576,15 +584,15 @@ def test_ls_then_read_then_answer(monkeypatch: pytest.MonkeyPatch) -> None: assert json.loads(msgs[4]["tool_calls"][0]["function"]["arguments"]) == { "path": "Homelab/aws-route53.md" } - assert msgs[5] == { - "role": "tool", - "tool_call_id": "call_2", - "content": ( - "Document Homelab/aws-route53.md:\n" - "date: 2024-06-15\n" - "R53-CONTENT" - ), # full text, no cap (date: the D5 second line) - } + assert msgs[5]["role"] == "tool" + assert msgs[5]["tool_call_id"] == "call_2" + # Phase 160 (task 01): round 2 of a 10-round budget reads 8 left. + assert msgs[5]["content"] == ( + "Document Homelab/aws-route53.md:\n" + "date: 2024-06-15\n" + "R53-CONTENT" + + agent.TOOL_BUDGET_REMAINING.format(n=8) + ) # full text, no cap (date: the D5 second line) + the phase-160 trailer def test_content_and_tool_call_in_one_stream_keeps_both( @@ -604,7 +612,7 @@ def test_content_and_tool_call_in_one_stream_keeps_both( pieces = asyncio.run(_run(llm, holder, _settings())) assert [type(p) for p in pieces] == [StreamPiece, ToolCallPiece, StreamPiece] assert holder.tool_calls == 1 # the tool ran despite the content - assert llm.requests[1][0][3]["content"] == "0 sources:" + assert llm.requests[1][0][3]["content"].startswith("0 sources:") # ---------- phase 74: client history between system and user ---------- @@ -703,13 +711,13 @@ def test_ls_top_level_lists_sources_with_summaries( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "2 sources:\n" "\n" "Deployments — 0 documents\n" "Homelab — 1 documents\n" " The homelab notes." - ) + ) assert holder.tool_calls == 1 @@ -724,7 +732,7 @@ def test_ls_empty_registry_says_zero_sources(monkeypatch: pytest.MonkeyPatch) -> [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "0 sources:" + assert llm.requests[1][0][3]["content"].startswith("0 sources:") assert holder.tool_calls == 1 @@ -734,7 +742,7 @@ def test_ls_empty_registry_says_zero_sources(monkeypatch: pytest.MonkeyPatch) -> ({"path": " "}, "blank path"), ({"path": 7}, "non-string path"), ], -) + ) def test_ls_blank_path_lists_top_level( monkeypatch: pytest.MonkeyPatch, arguments: dict[str, Any], label: str ) -> None: @@ -748,7 +756,7 @@ def test_ls_blank_path_lists_top_level( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "1 sources:\n\nS — 1 documents" + assert llm.requests[1][0][3]["content"].startswith("1 sources:\n\nS — 1 documents") assert holder.tool_calls == 1 @@ -774,14 +782,14 @@ def test_ls_source_scope_lists_root_folder(monkeypatch: pytest.MonkeyPatch) -> N [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Homelab — 1 documents, 2 folders:\n" "\n" " backups/ — 2 documents: Backup notes.\n" " networking/ — 1 documents\n" "\n" "source: Homelab | path: readme.md | title: Readme | date: 2024-06-15" - ) + ) assert holder.tool_calls == 1 @@ -814,12 +822,12 @@ def test_ls_nested_folder_scope_lists_one_level_deeper( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Homelab/networking — 2 documents, 0 folders:\n" "\n" "source: Homelab | path: networking/lan.md | title: LAN | date: 2024-06-15\n" "source: Homelab | path: networking/vpn.md | title: VPN | date: 2024-06-15" - ) + ) assert holder.tool_calls == 1 @@ -841,7 +849,7 @@ def test_ls_scoped_known_source_with_zero_docs_counts( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "Homelab — 0 documents, 0 folders:" + assert llm.requests[1][0][3]["content"].startswith("Homelab — 0 documents, 0 folders:") assert holder.tool_calls == 1 # an executed ls, not a refusal assert llm.requests[1][1] == AGENT_TOOLS @@ -858,9 +866,8 @@ def test_ls_scoped_unknown_source_refused(monkeypatch: pytest.MonkeyPatch) -> No ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 # a refusal counts in nothing - assert ( - llm.requests[1][0][3]["content"] - == agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost") + assert llm.requests[1][0][3]["content"].startswith( + agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="Ghost") ) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -886,9 +893,8 @@ def test_ls_path_like_scope_unknown_source_gets_no_source_refusal( ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 # a refusal counts in nothing - assert ( - llm.requests[1][0][3]["content"] - == agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app") + assert llm.requests[1][0][3]["content"].startswith( + agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app") ) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -908,9 +914,8 @@ def test_ls_dot_scope_gets_not_a_directory_teaching_refusal( ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 # a refusal counts in nothing - assert ( - llm.requests[1][0][3]["content"] - == agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope=".") + assert llm.requests[1][0][3]["content"].startswith( + agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope=".") ) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -945,10 +950,10 @@ def test_ls_unknown_top_level_folder_gets_not_a_folder_teaching( ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 # a refusal counts in nothing - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "'Homelab/netwoking' is not a folder — Homelab has: " "backups/ containers/ networking/" - ) + ) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -980,10 +985,10 @@ def test_ls_unknown_nested_folder_gets_not_a_folder_with_nested_parent( ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "'Homelab/networking/lan/x' is not a folder — " "Homelab/networking/lan has: none" - ) + ) def test_ls_file_path_scope_gets_not_a_folder(monkeypatch: pytest.MonkeyPatch) -> None: @@ -1009,9 +1014,9 @@ def test_ls_file_path_scope_gets_not_a_folder(monkeypatch: pytest.MonkeyPatch) - ) asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "'S/notes.md' is not a folder — S has: a/" - ) + ) # ---------- ls_top / ls_folder: the drill-down accessors (pure + composed) ---------- @@ -1487,11 +1492,11 @@ def test_read_combined_path_resolves_and_returns_full_content( assert calls == [("Homelab", "active/container_caddy/caddy.md")] assert holder.read_docs == [doc] assert holder.tool_calls == 1 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document Homelab/active/container_caddy/caddy.md:\n" "date: 2024-06-15\n" "CADDY-CONTENT" - ) + ) def test_read_bare_source_name_refused_without_db(monkeypatch: pytest.MonkeyPatch) -> None: @@ -1515,9 +1520,9 @@ def test_read_bare_source_name_refused_without_db(monkeypatch: pytest.MonkeyPatc ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'Homelab' — check the ls output." - ) + ) assert llm.requests[1][1] == AGENT_TOOLS @@ -1537,9 +1542,9 @@ def test_read_unknown_path_refused_echoing_argument( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'S/ghost.md' — check the ls output." - ) + ) assert llm.requests[1][1] == AGENT_TOOLS # tools stay offered (cap bounds) @@ -1628,7 +1633,7 @@ def test_read_bare_path_exact_match_gets_did_you_mean( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 # a refusal - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( agent.NO_DOCUMENT_DID_YOU_MEAN.format( arg="active/container_caddy/caddy.md", source="Homelab", @@ -1636,10 +1641,10 @@ def test_read_bare_path_exact_match_gets_did_you_mean( ) ) # The rendered line, pinned byte-for-byte. - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'active/container_caddy/caddy.md' — " "did you mean 'Homelab/active/container_caddy/caddy.md'?" - ) + ) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -1674,10 +1679,10 @@ def test_read_bare_path_suffix_match_gets_did_you_mean( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 # a refusal - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'container_caddy/caddy.md' — " "did you mean 'Homelab/active/container_caddy/caddy.md'?" - ) + ) def test_read_bare_path_two_sources_gets_one_of_suggestion( @@ -1699,16 +1704,16 @@ def test_read_bare_path_two_sources_gets_one_of_suggestion( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 # a refusal - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( agent.NO_DOCUMENT_DID_YOU_MEAN_MANY.format( arg="shared/x.md", candidates="'A/shared/x.md', 'B/shared/x.md'" ) ) # The rendered line, pinned byte-for-byte. - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'shared/x.md' — did you mean one of: " "'A/shared/x.md', 'B/shared/x.md'?" - ) + ) def test_read_bare_path_four_sources_capped_at_three_suggestions( @@ -1727,10 +1732,10 @@ def test_read_bare_path_four_sources_capped_at_three_suggestions( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'shared/x.md' — did you mean one of: " "'A/shared/x.md', 'B/shared/x.md', 'C/shared/x.md'?" - ) + ) assert "'D/shared/x.md'" not in llm.requests[1][0][3]["content"] assert holder.read_docs == [] and holder.tool_calls == 0 # a refusal @@ -1755,9 +1760,9 @@ def test_read_bare_filename_without_slash_keeps_no_db_refusal( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'caddy.md' — check the ls output." - ) + ) assert llm.requests[1][1] == AGENT_TOOLS @@ -1805,15 +1810,15 @@ def test_read_bare_path_of_suggested_doc_gets_suggestion_then_succeeds( # Round 1 refused (the teaching), round 2 executed (A6). assert holder.read_docs == [seed[0]] assert holder.tool_calls == 1 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'app/rag/importer.py' — " "did you mean 'sample/app/rag/importer.py'?" - ) - assert llm.requests[2][0][5]["content"] == ( + ) + assert llm.requests[2][0][5]["content"].startswith( "Document sample/app/rag/importer.py:\n" "date: 2024-06-15\n" "IMPORTER" - ) + ) @pytest.mark.parametrize( @@ -1825,7 +1830,7 @@ def test_read_bare_path_of_suggested_doc_gets_suggestion_then_succeeds( ({"path": 7}, "non-string path"), ({"path": None}, "null path"), ], -) + ) def test_read_missing_arguments_refused( monkeypatch: pytest.MonkeyPatch, arguments: dict[str, Any], label: str ) -> None: @@ -1840,7 +1845,7 @@ def test_read_missing_arguments_refused( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == agent.MISSING_READ_ARGS + assert llm.requests[1][0][3]["content"].startswith(agent.MISSING_READ_ARGS) assert llm.requests[1][1] == AGENT_TOOLS @@ -1875,9 +1880,9 @@ def test_reading_a_suggested_seed_doc_adds_its_full_text( asyncio.run(_run(llm, holder, _settings(), seed_docs=seed)) assert holder.read_docs == [seed[0]] assert holder.tool_calls == 1 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document Homelab/kubernetes.md:\ndate: 2024-06-15\nK8S-CONTENT" - ) + ) assert llm.requests[1][1] == AGENT_TOOLS @@ -1923,10 +1928,10 @@ def test_re_reading_a_suggested_doc_is_deduped(monkeypatch: pytest.MonkeyPatch) # retired — the merged span (1, 1) == the whole 1-line doc makes # the claim true) — two lookups. assert lookups == [("Homelab", "kubernetes.md"), ("Homelab", "kubernetes.md")] - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document Homelab/kubernetes.md:\ndate: 2024-06-15\nK8S-CONTENT" - ) - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + ) + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) # Rejected → the tools are still offered on the next request (the # round cap is the only bound). assert llm.requests[2][1] == AGENT_TOOLS @@ -1946,10 +1951,10 @@ def test_reading_an_already_read_doc_is_deduped(monkeypatch: pytest.MonkeyPatch) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [doc] # appended exactly once assert holder.tool_calls == 1 # the re-read counts nothing - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/a.md:\ndate: 2024-06-15\nA-CONTENT" - ) - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + ) + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) # Rejected → the tools are still offered on the next request… assert llm.requests[2][1] == AGENT_TOOLS @@ -1975,9 +1980,9 @@ def test_read_exactly_at_cap_is_byte_identical_and_untruncated( out = asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) # The read result plus the phase-106 D5 date line (no marker, no # notice). - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/big.md:\ndate: 2024-06-15\n" + content - ) + ) assert TRUNCATION_MARKER not in llm.requests[1][0][3]["content"] # No truncation recorded, none surfaced to the loop. assert holder.read_truncations == [] @@ -2014,7 +2019,7 @@ def test_read_at_cap_plus_one_truncates_with_marker_and_notice( + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=cap + 1) ) - assert llm.requests[1][0][3]["content"] == expected + assert llm.requests[1][0][3]["content"].startswith(expected) # (argument, chars_shown, chars_total) — the raw argument the tool # frame carries, the cap kept, the true length. assert holder.read_truncations == [("S/big.md", cap, cap + 1)] @@ -2131,7 +2136,7 @@ def test_read_truncation_paths_with_suggested_seed( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings(read_max_chars=100), seed_docs=[seed])) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/seed.md:\n" "date: 2024-06-15\n" + big[:100] @@ -2139,7 +2144,7 @@ def test_read_truncation_paths_with_suggested_seed( + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=100, total=5000) - ) + ) # (argument, cap, total) — the raw argument, the cap kept, the # true length. assert holder.read_truncations == [("S/seed.md", 100, 5000)] @@ -2152,9 +2157,9 @@ def test_read_truncation_paths_with_suggested_seed( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm2, holder2, _settings(read_max_chars=100), seed_docs=[seed])) - assert llm2.requests[1][0][3]["content"] == ( + assert llm2.requests[1][0][3]["content"].startswith( "No document at 'S/missing.md' — check the ls output." - ) + ) assert holder2.read_truncations == [] assert holder2.tool_calls == 0 and holder2.read_docs == [] @@ -2240,7 +2245,7 @@ def test_read_range_args_plain_and_valid() -> None: (None, 0), # a zero limit (None, -3), # a negative limit ], -) + ) def test_read_range_args_bad_values_are_refused( offset: Any, limit: Any ) -> None: @@ -2347,9 +2352,9 @@ def test_bad_range_args_refused_before_any_lookup( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == agent.BAD_RANGE_ARGS.format( + assert llm.requests[1][0][3]["content"].startswith(agent.BAD_RANGE_ARGS.format( offset=2.5, limit=None - ) + )) assert llm.requests[1][1] == AGENT_TOOLS @@ -2374,9 +2379,9 @@ def test_bad_range_args_beats_the_dedupe( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [doc] and holder.tool_calls == 1 - assert llm.requests[2][0][5]["content"] == agent.BAD_RANGE_ARGS.format( + assert llm.requests[2][0][5]["content"].startswith(agent.BAD_RANGE_ARGS.format( offset=0, limit=None - ) + )) def test_offset_past_end_refused_naming_line_count( @@ -2399,9 +2404,9 @@ def test_offset_past_end_refused_naming_line_count( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format( + assert llm.requests[1][0][3]["content"].startswith(agent.OFFSET_PAST_END.format( source="S", path="ten.md", total_lines=10, offset=11 - ) + )) def test_offset_past_end_on_one_line_doc( @@ -2423,9 +2428,9 @@ def test_offset_past_end_on_one_line_doc( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format( + assert llm.requests[1][0][3]["content"].startswith(agent.OFFSET_PAST_END.format( source="S", path="one.md", total_lines=1, offset=2 - ) + )) assert holder.read_docs == [] and holder.tool_calls == 0 # (b) the same doc read from line 1, limit overshoots — clamped. holder2 = AgentHolder() @@ -2440,9 +2445,9 @@ def test_offset_past_end_on_one_line_doc( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm2, holder2, _settings())) - assert llm2.requests[1][0][3]["content"] == ( + assert llm2.requests[1][0][3]["content"].startswith( "Document S/one.md:\ndate: 2024-06-15\nlines: 1-1 of 1\nonly" - ) + ) assert holder2.read_docs == [doc] and holder2.tool_calls == 1 @@ -2464,9 +2469,9 @@ def test_matrix_row1_plain_read_unread_doc_reads_full( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [doc] and holder.tool_calls == 1 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/m1.md:\ndate: 2024-06-15\none\ntwo\nthree" - ) + ) def test_matrix_row2_plain_read_truncated_doc_gets_reread_teaching( @@ -2500,21 +2505,21 @@ def test_matrix_row2_plain_read_truncated_doc_gets_reread_teaching( asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) # Round 1: the truncated plain read (the stuck trigger) — the # pre-phase-131 truncation bytes, the holder tuple recorded. - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/m2.md:\ndate: 2024-06-15\n" + doc.content[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content)) - ) + ) assert holder.read_truncations == [("S/m2.md", cap, len(doc.content))] # Round 2: the plain re-read → the teaching — the covered span # (1–14: the cut's truth) named, the true line count from the # resolved document. - assert llm.requests[2][0][5]["content"] == agent.REREAD_PARTIAL.format( + assert llm.requests[2][0][5]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1–14", source="S", path="m2.md", total=20 - ) + )) # A refusal: the counters stay at the first read's, no re-append; # the re-read RESOLVED first (the fast path is retired) — two # lookups. @@ -2548,7 +2553,7 @@ def test_matrix_row3_plain_read_full_doc_stays_already_in_context( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_docs == [doc] and holder.tool_calls == 1 # The re-read now goes through the resolve (the merged span (1, 1) # == the whole 1-line doc makes the claim true) — two lookups. @@ -2593,8 +2598,13 @@ def test_matrix_row4_range_read_truncated_doc_matrix( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) - # Round 1: the truncated plain read (covers lines 1–14). - assert llm.requests[1][0][3]["content"].endswith( + # Round 1: the truncated plain read (covers lines 1–14). The + # phase-160 trailer rides AFTER the raw result — the marker + + # notice end the RAW part. + content = llm.requests[1][0][3]["content"] + trailer = agent._budget_note(9) + assert content.endswith(trailer) + assert content[: -len(trailer)].endswith( "\n" + TRUNCATION_MARKER + "\n" @@ -2603,14 +2613,14 @@ def test_matrix_row4_range_read_truncated_doc_matrix( # Round 2: the range read of lines INSIDE the covered span (5-7 ⊆ # 1-14) is refused — the lines ARE in the prompt, the claim is # true. - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) # Round 3: the range read of the part the truncated read did NOT # reach (15-16) PROCEEDS — the stuck-scenario fix — lines 15-16 # of 20, exactly. - assert llm.requests[3][0][7]["content"] == ( + assert llm.requests[3][0][7]["content"].startswith( "Document S/m4.md:\ndate: 2024-06-15\nlines: 15-16 of 20\n" "line 15\nline 16" - ) + ) assert holder.read_docs == [doc] # appended exactly once assert holder.tool_calls == 2 # the refusal counted nothing assert holder.read_truncations == [("S/m4.md", cap, len(doc.content))] @@ -2639,7 +2649,7 @@ def test_matrix_row5_range_read_full_doc_stays_already_in_context( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_docs == [doc] and holder.tool_calls == 1 assert holder.read_truncations == [] @@ -2664,9 +2674,9 @@ def test_matrix_row6_range_read_unread_doc_proceeds( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/m6.md:\ndate: 2024-06-15\nlines: 3-4 of 8\nline 3\nline 4" - ) + ) assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -2711,19 +2721,19 @@ def test_matrix_row_range_read_of_different_range_proceeds( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/m7.md:\ndate: 2024-06-15\nlines: 5-7 of 12\n" "line 5\nline 6\nline 7" - ) + ) # Round 2: the DIFFERENT range PROCEEDS (the owner's stuck # scenario, fixed — both ranges reach the model). - assert llm.requests[2][0][5]["content"] == ( + assert llm.requests[2][0][5]["content"].startswith( "Document S/m7.md:\ndate: 2024-06-15\nlines: 10-11 of 12\n" "line 10\nline 11" - ) + ) # Round 3: the re-read of FULLY-covered lines 5-7 is refused — # the claim in the byte-identical line is now true. - assert llm.requests[3][0][7]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[3][0][7]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_docs == [doc] # appended exactly once assert holder.tool_calls == 2 # the refusal counted nothing assert holder.read_coverage == [("S", "m7.md", 5, 7), ("S", "m7.md", 10, 11)] @@ -2779,26 +2789,26 @@ def test_matrix_row_disjoint_spans_gap_plain_teaching_and_merge( asyncio.run(_run(llm, holder, _settings())) # Rounds 1-2: the two disjoint range reads (both proceed — first # reads of their spans). - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/m8.md:\ndate: 2024-06-15\nlines: 1-2 of 8\nline 1\nline 2" - ) - assert llm.requests[2][0][5]["content"] == ( + ) + assert llm.requests[2][0][5]["content"].startswith( "Document S/m8.md:\ndate: 2024-06-15\nlines: 5-6 of 8\nline 5\nline 6" - ) + ) # Round 3: the plain re-read of the PARTIALLY-covered doc — BOTH # merged spans named (comma-joined, en dash), the true total. - assert llm.requests[3][0][7]["content"] == agent.REREAD_PARTIAL.format( + assert llm.requests[3][0][7]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1–2, 5–6", source="S", path="m8.md", total=8 - ) + )) # Round 4: the range SPANNING THE GAP (2-5 crosses the uncovered # 3-4) is NOT fully covered → PROCEEDS. - assert llm.requests[4][0][9]["content"] == ( + assert llm.requests[4][0][9]["content"].startswith( "Document S/m8.md:\ndate: 2024-06-15\nlines: 2-5 of 8\n" "line 2\nline 3\nline 4\nline 5" - ) + ) # Round 5: after the merge ((1, 2) + (2, 5) + (5, 6) → (1, 6)), # 1-6 is FULLY covered → the (now true) refusal. - assert llm.requests[5][0][11]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[5][0][11]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_docs == [doc] assert holder.tool_calls == 3 # the two refusals counted nothing assert holder.read_coverage == [ @@ -2855,10 +2865,10 @@ def test_matrix_row_adjacent_spans_merge_into_full_coverage( "Document S/m9.md:\ndate: 2024-06-15\nlines: 4-6 of 6\n" ) # Round 3: 2-5 sits inside the merged (1, 6) → refused. - assert llm.requests[3][0][7]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[3][0][7]["content"].startswith(agent.ALREADY_IN_CONTEXT) # Round 4: the plain re-read — the merged span IS the whole doc # (1, 6) → the full-doc row, the same line. - assert llm.requests[4][0][9]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[4][0][9]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_docs == [doc] assert holder.tool_calls == 2 assert holder.read_coverage == [("S", "m9.md", 1, 3), ("S", "m9.md", 4, 6)] @@ -2918,15 +2928,15 @@ def test_scenario_adjacent_reads_cover_half_doc_teaching_and_rest_proceed( # Round 3: the plain re-read — the MERGED span named as one # (adjacent spans join: 1–100), the true total from the resolved # doc. - assert llm.requests[3][0][7]["content"] == agent.REREAD_PARTIAL.format( + assert llm.requests[3][0][7]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1–100", source="S", path="half.md", total=200 - ) + )) # Round 4: the REST of the document (offset=101, no limit → to # the end) PROCEEDS — the covered span stops at line 100. - assert llm.requests[4][0][9]["content"] == ( + assert llm.requests[4][0][9]["content"].startswith( "Document S/half.md:\ndate: 2024-06-15\nlines: 101-200 of 200\n" + "\n".join(f"line {i}" for i in range(101, 201)) - ) + ) assert holder.read_docs == [doc] assert holder.tool_calls == 3 # the teaching refusal counted nothing assert holder.read_coverage == [ @@ -2957,9 +2967,9 @@ def test_range_read_offset_one_no_limit_reads_whole_doc( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/r1.md:\ndate: 2024-06-15\nlines: 1-3 of 3\na\nb\nc" - ) + ) assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -2982,9 +2992,9 @@ def test_range_read_offset_at_total_reads_last_line( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/r2.md:\ndate: 2024-06-15\nlines: 3-3 of 3\nc" - ) + ) def test_range_read_limit_overshoot_clamps_to_total( @@ -3006,9 +3016,9 @@ def test_range_read_limit_overshoot_clamps_to_total( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/r3.md:\ndate: 2024-06-15\nlines: 2-3 of 3\nb\nc" - ) + ) def test_range_read_limit_without_offset_counts_from_start( @@ -3029,9 +3039,9 @@ def test_range_read_limit_without_offset_counts_from_start( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/r4.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb" - ) + ) def test_range_read_image_doc_carries_marker_after_lines_line( @@ -3056,11 +3066,11 @@ def test_range_read_image_doc_carries_marker_after_lines_line( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/img.png:\ndate: 2024-06-15\nlines: 2-2 of 2\n" f"{agent.IMAGE_DOC_MARKER}\n" "line-2-text" - ) + ) assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3093,14 +3103,14 @@ def test_range_read_slice_over_cap_truncates_with_slice_length_total( slice_text = "\n".join(lines[4:10]) assert len(slice_text) == 203 assert len(content) == 339 # the document is much bigger - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cap.md:\ndate: 2024-06-15\nlines: 5-10 of 10\n" + slice_text[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text)) - ) + ) # The tuple: the argument, the cap kept, the SLICE's length (203, # not the document's 339). assert holder.read_truncations == [("S/cap.md", cap, 203)] @@ -3149,12 +3159,16 @@ def test_range_read_truncated_then_plain_reread_teaching_then_range_proceeds( ) asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) # Round 1: the truncated range read (the whole doc, offset=1) — - # the slice is the whole doc here (339 chars), the tuple's total - # is the slice's length. + # The slice is the whole doc here (339 chars), the tuple's total + # is the slice's length. The phase-160 trailer rides AFTER the + # raw result — the marker + notice end the RAW part. assert llm.requests[1][0][3]["content"].startswith( "Document S/t.md:\ndate: 2024-06-15\nlines: 1-10 of 10\n" ) - assert llm.requests[1][0][3]["content"].endswith( + content = llm.requests[1][0][3]["content"] + trailer = agent._budget_note(9) + assert content.endswith(trailer) + assert content[: -len(trailer)].endswith( "\n" + TRUNCATION_MARKER + "\n" @@ -3163,21 +3177,21 @@ def test_range_read_truncated_then_plain_reread_teaching_then_range_proceeds( # Round 2: the plain re-read → the teaching — the covered span # (line 1 only: the cut string's splitlines() is one line) named, # the true line count from the resolved document. - assert llm.requests[2][0][5]["content"] == agent.REREAD_PARTIAL.format( + assert llm.requests[2][0][5]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1", source="S", path="t.md", total=10 - ) + )) # Round 3: the range read of the truncated doc PROCEEDS — lines 5-6 # (66 + 1 = 67 chars > the cap → truncated again; the tuple's total # is the slice's length, the document's is irrelevant). slice_text = "\n".join(lines[4:6]) - assert llm.requests[3][0][7]["content"] == ( + assert llm.requests[3][0][7]["content"].startswith( "Document S/t.md:\ndate: 2024-06-15\nlines: 5-6 of 10\n" + slice_text[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text)) - ) + ) assert holder.read_truncations == [("S/t.md", cap, 339), ("S/t.md", cap, 67)] assert holder.read_docs == [doc] # appended exactly once, still assert holder.tool_calls == 2 # the refusal counted nothing @@ -3210,12 +3224,12 @@ def test_plain_read_after_range_read_gets_partial_teaching( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/p.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb" - ) - assert llm.requests[2][0][5]["content"] == agent.REREAD_PARTIAL.format( + ) + assert llm.requests[2][0][5]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1–2", source="S", path="p.md", total=3 - ) + )) assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3239,9 +3253,9 @@ def test_coverage_plain_read_under_cap_records_full_span( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cv1.md:\ndate: 2024-06-15\nline 1\nline 2\nline 3\nline 4" - ) + ) assert holder.read_coverage == [("S", "cv1.md", 1, 4)] assert holder.read_docs == [doc] and holder.tool_calls == 1 assert holder.read_truncations == [] @@ -3289,14 +3303,14 @@ def test_coverage_truncated_plain_read_cut_at_line_boundary( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cv3.md:\ndate: 2024-06-15\n" + doc.content[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content)) - ) + ) assert holder.read_coverage == [("S", "cv3.md", 1, 2)] assert holder.read_truncations == [("S/cv3.md", cap, 11)] @@ -3325,9 +3339,9 @@ def test_coverage_range_read_under_cap_records_true_extent( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cv4.md:\ndate: 2024-06-15\nlines: 2-4 of 4\nb\nc\nd" - ) + ) assert holder.read_coverage == [("S", "cv4.md", 2, 4)] # (b) offset without limit → reads to the end (2-3 of 3). doc3 = _doc("S", "cv4b.md", "Cv4b", "a\nb\nc") @@ -3356,9 +3370,9 @@ def test_coverage_range_read_under_cap_records_true_extent( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm3, holder3, _settings())) - assert llm3.requests[1][0][3]["content"] == ( + assert llm3.requests[1][0][3]["content"].startswith( "Document S/cv4.md:\ndate: 2024-06-15\nlines: 1-2 of 4\na\nb" - ) + ) assert holder3.read_coverage == [("S", "cv4.md", 1, 2)] @@ -3390,14 +3404,14 @@ def test_coverage_truncated_range_read_records_cut_extent( asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) slice_text = "\n".join(lines[4:10]) # 203 chars > the cap assert len(slice_text[:cap].splitlines()) == 1 # only line 5 is visible - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cv5.md:\ndate: 2024-06-15\nlines: 5-10 of 10\n" + slice_text[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text)) - ) + ) assert holder.read_coverage == [("S", "cv5.md", 5, 5)] assert holder.read_truncations == [("S/cv5.md", cap, len(slice_text))] assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3426,14 +3440,14 @@ def test_coverage_truncated_range_read_cut_at_line_boundary( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/cv6.md:\ndate: 2024-06-15\nlines: 1-3 of 3\n" + doc.content[:cap] + "\n" + TRUNCATION_MARKER + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content)) - ) + ) assert holder.read_coverage == [("S", "cv6.md", 1, 2)] assert holder.read_truncations == [("S/cv6.md", cap, 8)] @@ -3463,10 +3477,10 @@ def test_coverage_spans_accumulate_in_read_order( asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) # Round 2 PROCEEDED (the truncated-then-range row) — lines 15-16 # (15 chars — under the cap, no second truncation). - assert llm.requests[2][0][5]["content"] == ( + assert llm.requests[2][0][5]["content"].startswith( "Document S/cv7.md:\ndate: 2024-06-15\nlines: 15-16 of 20\n" "line 15\nline 16" - ) + ) assert holder.read_coverage == [("S", "cv7.md", 1, 14), ("S", "cv7.md", 15, 16)] assert holder.tool_calls == 2 assert len(holder.read_truncations) == 1 @@ -3497,10 +3511,10 @@ def test_coverage_spans_keyed_by_the_resolved_pair( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/dir/nested/file.md:\ndate: 2024-06-15\n" "lines: 1-1 of 2\none" - ) + ) assert holder.read_coverage == [("S", "dir/nested/file.md", 1, 1)] assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3528,9 +3542,9 @@ def test_coverage_bad_range_args_records_nothing( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == agent.BAD_RANGE_ARGS.format( + assert llm.requests[1][0][3]["content"].startswith(agent.BAD_RANGE_ARGS.format( offset=2.5, limit=None - ) + )) assert holder.read_coverage == [] assert holder.read_docs == [] and holder.tool_calls == 0 @@ -3548,9 +3562,9 @@ def test_coverage_unknown_doc_records_nothing( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'S/ghost.md' — check the ls output." - ) + ) assert holder.read_coverage == [] assert holder.read_docs == [] and holder.tool_calls == 0 @@ -3573,9 +3587,9 @@ def test_coverage_offset_past_end_records_nothing( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format( + assert llm.requests[1][0][3]["content"].startswith(agent.OFFSET_PAST_END.format( source="S", path="cvr3.md", total_lines=10, offset=11 - ) + )) assert holder.read_coverage == [] assert holder.read_docs == [] and holder.tool_calls == 0 @@ -3596,7 +3610,7 @@ def test_coverage_plain_reread_of_full_doc_records_nothing_new( ) asyncio.run(_run(llm, holder, _settings())) assert llm.requests[1][0][3]["content"].startswith("Document S/cvr4.md:") - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_coverage == [("S", "cvr4.md", 1, 1)] # still ONE span assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3618,9 +3632,9 @@ def test_coverage_plain_reread_of_truncated_doc_records_nothing_new( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings(read_max_chars=cap))) - assert llm.requests[2][0][5]["content"] == agent.REREAD_PARTIAL.format( + assert llm.requests[2][0][5]["content"].startswith(agent.REREAD_PARTIAL.format( covered="1–14", source="S", path="cvr5.md", total=20 - ) + )) assert holder.read_coverage == [("S", "cvr5.md", 1, 14)] # still ONE span assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3646,7 +3660,7 @@ def test_coverage_range_reread_of_full_doc_records_nothing_new( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT + assert llm.requests[2][0][5]["content"].startswith(agent.ALREADY_IN_CONTEXT) assert holder.read_coverage == [("S", "cvr6.md", 1, 5)] # still ONE span assert holder.read_docs == [doc] and holder.tool_calls == 1 @@ -3659,7 +3673,7 @@ def test_unknown_tool_name_refused(monkeypatch: pytest.MonkeyPatch) -> None: ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [] and holder.tool_calls == 0 - assert llm.requests[1][0][3]["content"] == agent.UNKNOWN_TOOL + assert llm.requests[1][0][3]["content"].startswith(agent.UNKNOWN_TOOL) assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -3717,7 +3731,13 @@ def _one_read( kwargs: dict[str, Any] = {"read_max_chars": read_max_chars} if read_max_chars else {} asyncio.run(_run(llm, holder, _settings(**kwargs), db_factory=_git_rows_factory(rows))) assert holder.read_docs == [doc] and holder.tool_calls == 1 - return llm.requests[1][0][3]["content"] + # Phase 160 (task 01): the trailer rides the read result too — + # round 1 of the default 10-round budget reads 9 left. This + # helper returns the RAW result (its contract), with the trailer + # stripped after being asserted present. + content = llm.requests[1][0][3]["content"] + assert content.endswith(agent._budget_note(9)) + return content.removesuffix(agent._budget_note(9)) def test_origin_lines_plain_read_under_cap( @@ -4011,10 +4031,10 @@ def test_grep_whole_kb_grep_style_output(monkeypatch: pytest.MonkeyPatch) -> Non [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Alpha/a/one.md:2: NEEDLE in one\n" "Beta/b/two.md:2: needle in two" - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] # locked A5: a grep adds no context assert llm.requests[1][1] == AGENT_TOOLS # tools stay offered @@ -4034,7 +4054,9 @@ def test_grep_capped_at_20_matches_in_catalog_order( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - lines = llm.requests[1][0][3]["content"].split("\n") + # Phase 160 (task 01): the trailer rides the grep result too — the + # 20 raw match lines are the RAW part (the trailer stripped). + lines = llm.requests[1][0][3]["content"].removesuffix(agent._budget_note(9)).split("\n") assert len(lines) == agent.SEARCH_MAX_MATCHES assert lines[0] == "S/a.md:1: hit-0" assert lines[14] == "S/a.md:15: hit-14" # all of a.md @@ -4053,7 +4075,7 @@ def test_grep_truncates_match_lines_at_200_chars(monkeypatch: pytest.MonkeyPatch [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == f"S/a.md:2: {'x' * agent.SEARCH_LINE_LIMIT}" + assert llm.requests[1][0][3]["content"].startswith(f"S/a.md:2: {'x' * agent.SEARCH_LINE_LIMIT}") assert holder.tool_calls == 1 @@ -4116,15 +4138,15 @@ def test_grep_no_match_regex_pattern_gets_teaching_line( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == agent.NO_MATCHES_REGEX.format( + assert llm.requests[1][0][3]["content"].startswith(agent.NO_MATCHES_REGEX.format( pattern=r"qwen.*3\.8", plain="qwen3.8" - ) - assert llm.requests[1][0][3]["content"] == ( + )) + assert llm.requests[1][0][3]["content"].startswith( "No matches for 'qwen.*3\\.8'. grep matches a plain substring " "(case-insensitive), not a regex — '.*', '\\.' and the like are " "literal text here, so that pattern can never match. Retry with " "the plain text you expect to see (e.g. 'qwen3.8')." - ) + ) assert holder.tool_calls == 1 # a no-match with teaching is still a result assert holder.read_docs == [] # locked A5: a grep adds no context @@ -4152,11 +4174,11 @@ def test_grep_no_match_regex_scoped_gets_teaching_line( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No matches for 'qwen 3\\.8' in Alpha/a/one.md. grep matches a " "plain substring (case-insensitive), not a regex — retry with " "the plain text you expect to see (e.g. 'qwen 3.8')." - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] @@ -4176,9 +4198,9 @@ def test_grep_no_match_plain_pattern_keeps_ordinary_line( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No matches for 'qwen 3.8' in the knowledge base." - ) + ) assert holder.tool_calls == 1 @@ -4196,7 +4218,7 @@ def test_grep_matched_regex_pattern_returns_matches_not_teaching( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "S/a.md:1: the C++ compiler is here" + assert llm.requests[1][0][3]["content"].startswith("S/a.md:1: the C++ compiler is here") assert holder.tool_calls == 1 @@ -4213,9 +4235,9 @@ def test_grep_no_match_regex_reducing_to_empty_falls_back( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No matches for '.*' in the knowledge base." - ) + ) assert holder.tool_calls == 1 @@ -4266,7 +4288,7 @@ def test_grep_scoped_to_one_document(monkeypatch: pytest.MonkeyPatch) -> None: [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "S/a.md:1: needle here" + assert llm.requests[1][0][3]["content"].startswith("S/a.md:1: needle here") assert holder.tool_calls == 1 assert holder.read_docs == [] # grepped doc did not enter the context @@ -4296,7 +4318,7 @@ def test_grep_scoped_combined_path_with_nested_path( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "S/deep/nested/a.md:1: needle here" + assert llm.requests[1][0][3]["content"].startswith("S/deep/nested/a.md:1: needle here") assert holder.tool_calls == 1 @@ -4319,9 +4341,9 @@ def test_grep_scoped_missing_document_refused(monkeypatch: pytest.MonkeyPatch) - [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'S/ghost.md' — check the ls output." - ) + ) assert holder.tool_calls == 0 and holder.read_docs == [] # a refusal assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -4353,7 +4375,7 @@ def test_grep_scoped_missing_path_like_doc_gets_did_you_mean( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( agent.NO_DOCUMENT_DID_YOU_MEAN.format( arg="active/container_caddy/caddy.md", source="Homelab", @@ -4385,9 +4407,9 @@ def test_grep_scoped_bare_source_name_refused_without_db( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No document at 'Homelab' — check the ls output." - ) + ) assert holder.tool_calls == 0 and holder.read_docs == [] @@ -4400,7 +4422,7 @@ def test_grep_scoped_bare_source_name_refused_without_db( ({"pattern": 42}, "non-string pattern"), ({"pattern": None}, "null pattern"), ], -) + ) def test_grep_missing_arguments_refused( monkeypatch: pytest.MonkeyPatch, arguments: dict[str, Any], label: str ) -> None: @@ -4418,7 +4440,7 @@ def test_grep_missing_arguments_refused( [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == agent.MISSING_SEARCH_ARGS + assert llm.requests[1][0][3]["content"].startswith(agent.MISSING_SEARCH_ARGS) assert holder.tool_calls == 0 and holder.read_docs == [] assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered @@ -4435,9 +4457,9 @@ def test_grep_no_matches_whole_kb(monkeypatch: pytest.MonkeyPatch) -> None: [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "No matches for 'zebra' in the knowledge base." - ) + ) assert holder.tool_calls == 1 assert holder.read_docs == [] @@ -4462,7 +4484,7 @@ def test_grep_no_matches_scoped(monkeypatch: pytest.MonkeyPatch) -> None: [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == "No matches for 'zebra' in S/a.md." + assert llm.requests[1][0][3]["content"].startswith("No matches for 'zebra' in S/a.md.") assert holder.tool_calls == 1 assert holder.read_docs == [] @@ -4479,9 +4501,9 @@ def test_grep_no_match_truncates_long_pattern(monkeypatch: pytest.MonkeyPatch) - [StreamPiece("content", "ans")], ) asyncio.run(_run(llm, holder, _settings())) - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( f"No matches for '{'p' * 100}' in the knowledge base." - ) + ) assert holder.tool_calls == 1 @@ -4501,10 +4523,10 @@ def test_grep_counts_but_never_adds_context(monkeypatch: pytest.MonkeyPatch) -> asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 2 # grep + read, both executed assert holder.read_docs == [doc] # only the read added context (A5) - assert llm.requests[1][0][3]["content"] == "S/a.md:1: needle here" - assert llm.requests[2][0][5]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith("S/a.md:1: needle here") + assert llm.requests[2][0][5]["content"].startswith( "Document S/a.md:\ndate: 2024-06-15\nneedle here" - ) + ) # ---------- unlimited calls: re-lists and multi-reads (phase 45) ---------- @@ -4525,9 +4547,11 @@ def test_relist_executes_and_counts(monkeypatch: pytest.MonkeyPatch) -> None: asyncio.run(_run(llm, holder, _settings())) assert holder.tool_calls == 2 # both re-lists executed and counted listing = "2 sources:\n\nDeployments — 1 documents\nHomelab — 1 documents" - # The answer request carries the listing a second time as a tool result. - assert llm.requests[2][0][3]["content"] == listing # first listing - assert llm.requests[2][0][5]["content"] == listing # the re-list + # The answer request carries the listing a second time as a tool + # result. Phase 160 (task 01): the trailer rides both — 9 left + # after round 1, 8 left after round 2 (10-round budget). + assert llm.requests[2][0][3]["content"] == listing + agent.TOOL_BUDGET_REMAINING.format(n=9) + assert llm.requests[2][0][5]["content"] == listing + agent.TOOL_BUDGET_REMAINING.format(n=8) assert llm.requests[2][1] == AGENT_TOOLS # still offered (no budgets) @@ -4549,12 +4573,12 @@ def test_multi_read_executes_without_budgets(monkeypatch: pytest.MonkeyPatch) -> asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [a, b] # both reads appended, in order assert holder.tool_calls == 2 - assert llm.requests[1][0][3]["content"] == ( + assert llm.requests[1][0][3]["content"].startswith( "Document S/a.md:\ndate: 2024-06-15\nA-CONTENT" - ) - assert llm.requests[2][0][5]["content"] == ( + ) + assert llm.requests[2][0][5]["content"].startswith( "Document S/b.md:\ndate: 2024-06-15\nB-CONTENT" - ) + ) assert llm.requests[2][1] == AGENT_TOOLS # the second read was still offered @@ -4593,9 +4617,12 @@ def test_always_ls_bounded_by_round_cap(monkeypatch: pytest.MonkeyPatch) -> None # The final request carries all three executed listings as history. final_msgs = llm.requests[3][0] assert len(final_msgs) == 8 # 2 + 3 rounds × (assistant + tool) - assert final_msgs[3]["content"] == listing - assert final_msgs[5]["content"] == listing - assert final_msgs[7]["content"] == listing + # Phase 160 (task 01): the trailer tells the model how many rounds + # it has LEFT — 2, then 1, then the spent line (the model is + # coached to answer or tell the user to say "continue"). + assert final_msgs[3]["content"] == listing + agent.TOOL_BUDGET_REMAINING.format(n=2) + assert final_msgs[5]["content"] == listing + agent.TOOL_BUDGET_LAST + assert final_msgs[7]["content"] == listing + agent.TOOL_BUDGET_SPENT def test_zero_max_rounds_is_one_request_without_tools() -> None: @@ -4612,6 +4639,9 @@ def test_zero_max_rounds_is_one_request_without_tools() -> None: {"role": "system", "content": "SYSTEM_PROMPT"}, {"role": "user", "content": "QUESTION"}, ] + # Phase 160 (task 01): no tools offered → no tool result → the + # budget-telemetry trailer exists nowhere on this request. + assert not any(m["role"] == "tool" for m in llm.requests[0][0]) assert holder.read_docs == [] and holder.tool_calls == 0 @@ -4637,9 +4667,301 @@ def test_rejected_read_spam_runs_to_round_cap(monkeypatch: pytest.MonkeyPatch) - assert llm.requests[3][1] is None # the forced final request: no tools assert holder.read_docs == [] and holder.tool_calls == 0 # nothing executed refusal = "No document at 'S/ghost.md' — check the ls output." - assert llm.requests[1][0][3]["content"] == refusal - assert llm.requests[2][0][5]["content"] == refusal - assert llm.requests[3][0][7]["content"] == refusal + # Phase 160 (task 01): a rejection consumes a round, so its + # refusal carries the budget line too — 2, then 1, then spent. + assert llm.requests[1][0][3]["content"] == refusal + agent.TOOL_BUDGET_REMAINING.format(n=2) + assert llm.requests[2][0][5]["content"] == refusal + agent.TOOL_BUDGET_LAST + assert llm.requests[3][0][7]["content"] == refusal + agent.TOOL_BUDGET_SPENT + + +# ---------- budget telemetry (phase 160, task 01) ---------- + + +def test_budget_note_boundary_table() -> None: + """The pure helper: ``n ≥ 2`` → the formatted remaining line, + ``1`` → the last-call line, ``≤ 0`` → the spent line.""" + assert agent._budget_note(2) == agent.TOOL_BUDGET_REMAINING.format(n=2) + assert agent._budget_note(3) == agent.TOOL_BUDGET_REMAINING.format(n=3) + assert agent._budget_note(1) == agent.TOOL_BUDGET_LAST + assert agent._budget_note(0) == agent.TOOL_BUDGET_SPENT + assert agent._budget_note(-1) == agent.TOOL_BUDGET_SPENT + + +def test_budget_trailer_decreases_across_rounds(monkeypatch: pytest.MonkeyPatch) -> None: + """The 3-round always-caller: every tool result the model receives + ends with its remaining count — 2, then the 1-call line, then the + spent line (the boundary call still runs; its result says 0).""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [ToolCallPiece(id="call_2", name="ls", arguments={})], + [ToolCallPiece(id="call_3", name="ls", arguments={})], + [StreamPiece("content", "forced answer")], + ) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=3))) + final_msgs = llm.requests[3][0] + assert final_msgs[3]["content"].endswith(agent.TOOL_BUDGET_REMAINING.format(n=2)) + assert final_msgs[5]["content"].endswith(agent.TOOL_BUDGET_LAST) + assert final_msgs[7]["content"].endswith(agent.TOOL_BUDGET_SPENT) + + +def test_single_round_budget_carries_the_spent_line( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """``agent_max_rounds=1``: the one tool result the model receives + says the budget is already spent — the spent line names the user + action (say "continue" to keep researching).""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + listing = "1 sources:\n\nS — 1 documents" + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [StreamPiece("content", "forced answer")], + ) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + assert len(llm.requests) == 2 + assert llm.requests[1][1] is None # the forced final: no tools + assert llm.requests[1][0][3]["content"] == listing + agent.TOOL_BUDGET_SPENT + + +def test_rejected_call_carries_the_budget_note() -> None: + """An unknown tool (a rejection) still consumes a round — its + refusal text ends with the note too, so the budget line is + exactly as true there.""" + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="fly_to_the_moon", arguments={})], + [StreamPiece("content", "the answer")], + ) + asyncio.run(_run(llm, holder, _settings())) + assert llm.requests[1][0][3]["content"] == ( + agent.UNKNOWN_TOOL + agent.TOOL_BUDGET_REMAINING.format(n=9) + ) + + +# ---------- cap path: reject the model's late calls (phase 160, task 02) ---------- + + +def test_cap_path_tool_calls_rejected_and_budget_fallback( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The forced final (``tools=None``) the model answers with + structured ``tool_calls`` (the documented ``lite`` behavior — a + ``tools=None`` request can still come back with calls): the calls + are REJECTED — never executed, never yielded (no ``tool`` frame, no + visible markup), counted in ``holder.rejected_calls`` (``tool_calls`` + stays executed-only) — and the turn settles with the deterministic + ``ROUND_CAP_FALLBACK_ANSWER`` (no exception, no empty answer).""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], # round 1: executed + [ # the forced final: structured calls, no content + ToolCallPiece(id="call_f1", name="ls", arguments={}), + ToolCallPiece(id="call_f2", name="ls", arguments={}), + ], + ) + # No exception — the budget fallback settles the turn. + pieces = asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + # Only the executed round 1 call streams as a tool piece; the forced + # final's two calls are rejected (not yielded at all). + assert [type(p) for p in pieces] == [ToolCallPiece, StreamPiece] + assert pieces[0] == ToolCallPiece(id="call_1", name="ls", arguments={}) + # The fallback is the terminal visible content. + assert pieces[1] == StreamPiece("content", agent.ROUND_CAP_FALLBACK_ANSWER) + # The attempts are counted; the executed call is unchanged. + assert holder.rejected_calls == 2 + assert holder.tool_calls == 1 + # The forced final request carried no tools (the ``tools=None`` pin). + assert len(llm.requests) == 2 + assert llm.requests[1][1] is None + + +def test_cap_path_calls_rejected_but_content_streams( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A forced final that carries BOTH a structured call and real + visible content: the content streams (it is the answer), the call is + still rejected (counted, not yielded), and NO fallback is appended + (there was visible content).""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], # round 1: executed + [ + ToolCallPiece(id="call_f1", name="ls", arguments={}), + StreamPiece("content", "the real answer"), + ], + ) + pieces = asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + # The executed call + the content stream; the rejected call is not + # yielded; no fallback (there was visible content). + assert [type(p) for p in pieces] == [ToolCallPiece, StreamPiece] + assert pieces[1] == StreamPiece("content", "the real answer") + assert agent.ROUND_CAP_FALLBACK_ANSWER not in [getattr(p, "text", "") for p in pieces] + assert holder.rejected_calls == 1 + assert holder.tool_calls == 1 + + +def test_forced_final_carries_the_wrapup_coaching( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The forced final's single system message is + ``system_prompt + "\n" + ROUND_CAP_WRAPUP_INSTRUCTION`` (the + phase-71 :data:`CORRECTION_INSTRUCTION` pattern); the rest of the + history (user + the executed round's assistant/tool pair) is + byte-identical to what the rounds accumulated — only the system + message changes. The round 1 request's system message stays the + plain prompt (the coaching is the cap path only).""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + listing = "1 sources:\n\nS — 1 documents" + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [StreamPiece("content", "forced answer")], + ) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + # The round 1 request: the plain system prompt, tools offered. + assert llm.requests[0][0][0] == {"role": "system", "content": "SYSTEM_PROMPT"} + # The forced final: the coached system message, no tools, and the + # tail (user + the round 1 assistant/tool pair) unchanged. + capped = llm.requests[1][0] + assert llm.requests[1][1] is None + assert capped[0] == { + "role": "system", + "content": "SYSTEM_PROMPT\n" + agent.ROUND_CAP_WRAPUP_INSTRUCTION, + } + assert len(capped) == 4 + assert capped[1] == {"role": "user", "content": "QUESTION"} + assert capped[2]["role"] == "assistant" + assert capped[2]["tool_calls"][0]["function"]["name"] == "ls" + assert capped[3]["role"] == "tool" + assert capped[3]["content"] == listing + agent.TOOL_BUDGET_SPENT + + +def test_cap_path_empty_forced_final_yields_the_budget_fallback( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A forced final that streams NOTHING (no content, no calls, no + scaffolding) still settles with the ``ROUND_CAP_FALLBACK_ANSWER`` — + the ``final_content == 0`` branch, the pre-phase silent-empty-answer + path now gets the honest budget line.""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [], # the forced final: literally nothing + ) + pieces = asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + assert [type(p) for p in pieces] == [ToolCallPiece, StreamPiece] + assert pieces[1] == StreamPiece("content", agent.ROUND_CAP_FALLBACK_ANSWER) + assert holder.rejected_calls == 0 + assert holder.tool_calls == 1 + + +def test_cap_path_logs_the_rejection_and_fallback_warnings( + monkeypatch: pytest.MonkeyPatch, + caplog: pytest.LogCaptureFixture, +) -> None: + """Both cap-path warnings appear: the existing + ``round cap reached (rounds=N)`` (the phase-45 pin, unchanged) and + the new ``tool call(s) attempted after the round cap — rejected, not + shown`` (the rejection, named), plus the ``no visible reply`` + fallback line. The ``rejected`` warning is named by the count.""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [ToolCallPiece(id="call_f1", name="ls", arguments={})], + ) + with caplog.at_level(logging.INFO, logger="app.agent"): + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + msgs = [r.getMessage() for r in caplog.records] + assert any("round cap reached (rounds=1)" in m for m in msgs) + assert any( + "1 tool call(s) attempted after the round cap — rejected, not shown" + in m + for m in msgs + ) + assert any( + "round cap hit with no visible reply (rejected_calls=1, scaffold_stripped=0)" + in m + for m in msgs + ) + + +# ---------- rounds / cap_reached holder state (phase 160, task 03) ---------- + + +def test_holder_rounds_counts_rejections_as_rounds() -> None: + """``holder.rounds`` counts every call the model emitted on a + tools-offered round — an unknown-tool rejection bumps ``rounds`` + but NOT ``tool_calls`` (the executed-only counter); a turn that + answered inside the cap keeps ``cap_reached`` ``False``.""" + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="fly_to_the_moon", arguments={})], + [ToolCallPiece(id="call_2", name="fly_to_the_moon", arguments={})], + [StreamPiece("content", "the answer")], + ) + asyncio.run(_run(llm, holder, _settings())) + assert holder.rounds == 2 # both rounds consumed (both rejected) + assert holder.tool_calls == 0 # nothing executed + assert holder.cap_reached is False # answered inside the cap + + +def test_holder_rounds_and_cap_flags_on_an_in_cap_executed_turn( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """An in-cap turn with executed rounds: ``holder.rounds`` mirrors + the loop's local counter (2 executed rounds), ``tool_calls`` + agrees, and ``cap_reached`` stays ``False``.""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], + [ToolCallPiece(id="call_2", name="ls", arguments={})], + [StreamPiece("content", "inside the cap")], + ) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=10))) + assert holder.rounds == 2 + assert holder.tool_calls == 2 + assert holder.cap_reached is False + + +def test_holder_reports_the_full_round_count_at_the_cap( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A turn that ends at the cap: ``holder.cap_reached`` is + ``True`` and ``holder.rounds`` reports the FULL count (the mirror + before the forced final); the cap-path's rejected call is NOT a + round — it counts in ``rejected_calls``, not ``rounds``.""" + monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) + holder = AgentHolder() + llm = ScriptedLLM( + [ToolCallPiece(id="call_1", name="ls", arguments={})], # executed + [ # the forced final: one late call, no content + ToolCallPiece(id="call_f1", name="ls", arguments={}) + ], + ) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + assert holder.cap_reached is True + assert holder.rounds == 1 # the executed round; the late call is not a round + assert holder.rejected_calls == 1 + assert holder.tool_calls == 1 + + +def test_holder_defaults_on_the_kill_switch_turn() -> None: + """``agent_max_rounds=0``: no rounds are consumed and the cap + never runs — the holder reports ``rounds=0`` / + ``cap_reached=False`` (the turn shape the per-turn log line + renders as ``rounds=0 cap_hit=0``).""" + holder = AgentHolder() + llm = ScriptedLLM([StreamPiece("content", "direct answer")]) + asyncio.run(_run(llm, holder, _settings(agent_max_rounds=0))) + assert holder.rounds == 0 + assert holder.cap_reached is False # ---------- retries inside the agent loop (phase 67, locked A2) ---------- @@ -5011,7 +5333,7 @@ def test_low_prompt_is_byte_identical_and_tool_free() -> None: #: task 05) — a complete span the filter strips in full. _INCIDENT_SPAN = ( "<|tool_call_start|>[read(path='/homelab/backup-notes.md')]<|tool_call_end|>" -) + ) def test_correction_instruction_is_the_harness_constant() -> None: @@ -5226,30 +5548,45 @@ def test_recovery_after_tool_rounds_keeps_the_history( assert recovered[1] == {"role": "user", "content": "QUESTION"} assert len(recovered) == 4 assert recovered[2]["role"] == "assistant" - assert recovered[3] == { - "role": "tool", - "tool_call_id": "call_1", - "content": "1 sources:\n\nS — 1 documents", - } + assert recovered[3]["role"] == "tool" + assert recovered[3]["tool_call_id"] == "call_1" + # Phase 160 (task 01): the tool history behind the recovery carries + # the trailer (round 1 of a 10-round budget reads 9 left). + assert recovered[3]["content"] == ( + "1 sources:\n\nS — 1 documents" + agent.TOOL_BUDGET_REMAINING.format(n=9) + ) assert sum(1 for m in recovered if m["role"] == "system") == 1 assert holder.scaffold_stripped == len(_INCIDENT_SPAN) -def test_forced_final_scaffolding_only_settles_malformed( +def test_forced_final_scaffolding_only_yields_the_budget_fallback( monkeypatch: pytest.MonkeyPatch, ) -> None: """The round-cap forced final (``tools=None``) is filtered too: a - scaffolding-only forced answer never reaches the user raw — the turn - settles with :class:`MalformedReplyError` (the same terminal - semantics; this turn used no recovery, so nothing is doubled up).""" + scaffolding-only forced answer never reaches the user raw, and — + phase 160 (task 02, the recorded revision of the phase-71 cap-path + terminal, AGENTS.md rule 3, owner-confirmed 2026-10-05) — the turn + NO LONGER settles with :class:`MalformedReplyError`; it settles with + the deterministic ``ROUND_CAP_FALLBACK_ANSWER`` (an honest "budget + spent" line — the no-visible-content path, no exception). This turn + used no recovery, so nothing is doubled up; the strip is still + captured in the turn total.""" monkeypatch.setattr(agent, "ls_top", lambda db: [("S", 1, None)]) holder = AgentHolder() llm = ScriptedLLM( [ToolCallPiece(id="call_1", name="ls", arguments={})], [StreamPiece("content", _INCIDENT_SPAN)], ) - with pytest.raises(MalformedReplyError): - asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + # No exception — the budget fallback settles the turn. + pieces = asyncio.run(_run(llm, holder, _settings(agent_max_rounds=1))) + # The executed round 1 call streams; the forced final's scaffolding is + # filtered to nothing; the fallback is the terminal visible content. + assert [type(p) for p in pieces] == [ToolCallPiece, StreamPiece] + assert pieces[1] == StreamPiece("content", agent.ROUND_CAP_FALLBACK_ANSWER) assert len(llm.requests) == 2 assert llm.requests[1][1] is None # the forced final — no recovery after it assert holder.scaffold_stripped == len(_INCIDENT_SPAN) + # The scaffold-only forced final had no visible content and no rejected + # calls — this is the fallback path, not the rejection path. + assert holder.rejected_calls == 0 + assert holder.tool_calls == 1 diff --git a/tests/unit/test_chat_gate.py b/tests/unit/test_chat_gate.py index 1eb93de..40a35e3 100644 --- a/tests/unit/test_chat_gate.py +++ b/tests/unit/test_chat_gate.py @@ -1578,6 +1578,8 @@ def test_endpoint_log_line_records_suggested_after_summary_hits( "sources=", "thinking_chars=", "tool_calls=", + "rounds=", + "cap_hit=", "total_ms=", "retries=", "scaffold_stripped=", diff --git a/tests/unit/test_chat_image_questions.py b/tests/unit/test_chat_image_questions.py index 6fdabb9..ca79749 100644 --- a/tests/unit/test_chat_image_questions.py +++ b/tests/unit/test_chat_image_questions.py @@ -491,7 +491,9 @@ def test_chat_message_image_defaults_to_none_and_is_omitted() -> None: assert m.image is None dumped = m.model_dump() assert "image" not in dumped # ABSENT, never ``null`` (the phase-50 contract) - # and every pre-phase key set is untouched + # and every pre-phase key set is untouched (the phase-160 additive + # ``budget_exhausted`` marker joins as an explicit null — the + # phase-120 ``failed`` / phase-48 ``stopped`` style). assert set(dumped) == { "who", "text", @@ -504,6 +506,7 @@ def test_chat_message_image_defaults_to_none_and_is_omitted() -> None: "stopped", "failed", "error", + "budget_exhausted", } diff --git a/tests/unit/test_chat_message_failed.py b/tests/unit/test_chat_message_failed.py index ac180d2..b7e11a4 100644 --- a/tests/unit/test_chat_message_failed.py +++ b/tests/unit/test_chat_message_failed.py @@ -173,10 +173,14 @@ def test_pre_phase_record_round_trips_byte_identical_on_existing_keys() -> None: # Every pre-phase key survives byte-identical… for key, value in pre_phase.items(): assert dumped[key] == value, f"pre-phase key {key!r} changed: {dumped[key]!r}" - # …and the ONLY additions are the two new keys as explicit nulls. - assert set(dumped) == set(pre_phase) | {"failed", "error"} + # …and the ONLY additions are the new keys as explicit nulls + # (the phase-120/48 ``failed`` / ``error`` pair, and the phase-160 + # additive ``budget_exhausted`` marker — the client's restore + # hint, explicit null where absent). + assert set(dumped) == set(pre_phase) | {"failed", "error", "budget_exhausted"} assert dumped["failed"] is None assert dumped["error"] is None + assert dumped["budget_exhausted"] is None # The stored shape re-validates (round-trip through the DB JSONB). assert ChatMessage.model_validate(dumped).model_dump() == dumped @@ -197,4 +201,5 @@ def test_minimal_user_record_unchanged() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, } diff --git a/tests/unit/test_frontend_edit_message.py b/tests/unit/test_frontend_edit_message.py index 9bd1b4b..6beac46 100644 --- a/tests/unit/test_frontend_edit_message.py +++ b/tests/unit/test_frontend_edit_message.py @@ -796,9 +796,15 @@ def test_retry_last_turn_is_byte_unchanged() -> None: #: task-01 idle-settle ``markLastEditable()`` mark pass (work item 3 #: call sites — the in-flight guard's at-rest landing), and the #: task-03 aborted-settle mark pair (the 300s guard's ``aborted`` -#: catch branch — a timed-out turn must not strand the question). -#: The ``reask`` path is consumed as-is. A diff here is a contract -#: violation for the phase's remaining tasks. +#: catch branch — a timed-out turn must not strand the question); +#: plus the phase-160 (task 04, TODO L4) sanctioned additions: +#: the done settle's ``budget_exhausted`` meta key (the additive +#: done-frame marker — ``undefined`` drops the key, so every non-cap +#: record stays byte-identical) and the flag-gated +#: ``appendContinueButton`` (the "Continue researching" affordance — +#: ABSENT on every other turn). The ``reask`` path is consumed as-is. +#: A diff here is a contract violation for the phase's remaining +#: tasks. PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}) { if (!reask) { // Phase 123 (task 02, TODO L6): the attached image rides the user @@ -1181,6 +1187,12 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {} // row exactly as it looked live. related: ev.related?.length ? ev.related : undefined, suggestions: ev.suggestions, + // Phase 160 (task 04, TODO L4): the cap-reached marker + // persists with the turn — `undefined` drops the key from + // the JSON (the house optional-meta pattern, the + // answer_image line above is the precedent), so every + // non-cap record stays byte-identical to pre-phase. + budget_exhausted: ev.budget_exhausted || undefined, }, leavePartialIndex); lastBrainWrap = wrap; // this bubble is now the last brain answer // Phase 130 (task 02): the row order is the DOM order — the @@ -1191,6 +1203,20 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {} appendTuneButton(wrap); // every completed brain bubble is tunable markLastRetryable(); // phase 49: the Retry button is last-bubble-only markLastEditable(); // phase 159: the done turn's question is editable (the in-flight guard no-ops here — the runTurn idle settle lands it) + // Phase 160 (task 04, TODO L4): a cap-reached answer offers + // the one-click continuation — first, every earlier bubble's + // continue button is cleared (the markLastRetryable + // remove-first pattern: the moment a newer turn lands, an old + // bubble's stale affordance is gone — its "continue" would + // append to a history the conversation already moved past), + // then the button is appended ONLY when the done frame is + // flagged (ABSENT on every other turn, so the button never + // appears elsewhere); the row order stays Tune → Retry → + // Continue → Save-as-doc (the insertion order phase 130 + // relies on — markLastEditable above acts on the user + // bubble, so the brain bubble's row order is unchanged). + messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove()); + if (ev.budget_exhausted) appendContinueButton(wrap); // Phase 59: the RAW persisted markdown (never the rendered // HTML) — exactly the string rememberBrainTurn stores above, // so a reload (the restore path) offers the identical draft. @@ -1397,13 +1423,16 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {} def test_run_turn_is_pinned_at_phase_159_state() -> None: - """``runTurn`` is pinned at exactly the state phase 159 left it - (task 02 consumes its ``reask`` path AS-IS — the phase's "NOT - touched" contract): the full source must match the pin — the - ``reask`` path, the turn-local resets, the SSE contract, the - save points, and the sanctioned additions (the task-01 - ``lastUserWrap`` assignment + the idle-settle mark pass, and the - task-03 aborted-settle mark pair) included.""" + """``runTurn`` is pinned at exactly the state phase 159 left it, + plus phase 160 (task 04)'s sanctioned additions (task 02 consumes + its ``reask`` path AS-IS — the phase's "NOT touched" contract; + the phase-160 additions are the done settle's additive + ``budget_exhausted`` meta key + the flag-gated Continue button): + the full source must match the pin — the ``reask`` path, the + turn-local resets, the SSE contract, the save points, and the + sanctioned additions (the task-01 ``lastUserWrap`` assignment + + the idle-settle mark pass, the task-03 aborted-settle mark pair, + and the phase-160 done-settle pair) included.""" assert _fn_source(_js(), "runTurn") == PINNED_RUN_TURN diff --git a/tests/unit/test_frontend_tool_budget.py b/tests/unit/test_frontend_tool_budget.py new file mode 100644 index 0000000..3493a57 --- /dev/null +++ b/tests/unit/test_frontend_tool_budget.py @@ -0,0 +1,546 @@ +"""Unit: the phase-160 (task 04, TODO L4) "Continue researching" button. + +When a turn ends because its tool budget was spent (the ``done`` +frame's additive ``budget_exhausted`` key, task 03 — ABSENT on every +non-cap turn), the answer bubble offers **Continue researching**: one +click sends the next turn (a plain send — the budget is per-turn, so +the fresh turn has a whole new round budget) and the marker persists +with the record so a page refresh restores the affordance. The shared +page never shows the button (read-only by construction). + +Pins (the house source-pin pattern — no browser): + +* ``CONTINUE_ICON`` / ``CONTINUE_QUESTION`` — the inline aria-hidden + SVG in the RETRY_ICON stroke family (no icon font, no CDN — A11) + and the literal continuation word (what the model is taught to ask + the user for); +* ``appendContinueButton`` — the house pattern through + ``getActionsRow`` (the role=list row joins the button as a + listitem), the one-per-bubble guard, click → ``continueTurn()``; +* ``continueTurn`` — the ``retryLastTurn`` discipline: no-op while a + turn is in flight, every ``.continue-btn`` removed FIRST, then a + PLAIN ``runTurn(CONTINUE_QUESTION)`` (no ``reask``); +* the ``done`` settle — ``budget_exhausted`` rides + ``rememberBrainTurn``'s meta (``undefined`` drops the key, so every + non-cap record stays byte-identical) and the button is appended + ONLY when the frame is flagged, in the row order Tune → Retry → + Continue → Save-as-doc (the insertion order phase 130 relies on); +* the restore branch — keys on ``m.budget_exhausted``; +* ``startNewChat`` needs nothing — its list reset drops the buttons; +* ``retryLastTurn`` / ``markLastRetryable`` / ``rememberBrainTurn`` + are byte-unchanged (the phase-120 shared pin for the first); +* the shared page (work item 8) — ``shared.html`` never loads + ``app.js`` and ``shared.js`` builds no buttons / actions rows, so a + flagged record renders without any Continue control there. + +The browser-level assertions (the live click running a fresh turn, the +reload restoring the button, the shared page's read-only render) are +E2E-gated by tests/e2e/test_tool_budget_telemetry.py (task 05). +""" +from __future__ import annotations + +from pathlib import Path + +from tests.unit.test_frontend_failed_turn import PINNED_RETRY_LAST_TURN + +FRONTEND = Path(__file__).resolve().parents[2] / "frontend" +APP_JS = FRONTEND / "assets" / "app.js" +STYLES_CSS = FRONTEND / "assets" / "styles.css" +SHARED_HTML = FRONTEND / "shared.html" +SHARED_JS = FRONTEND / "assets" / "shared.js" + + +def _js() -> str: + return APP_JS.read_text(encoding="utf-8") + + +def _css() -> str: + return STYLES_CSS.read_text(encoding="utf-8") + + +def _fn_body(js: str, header: str) -> str: + """The full source of the function whose header is *header* — + brace-matched from the first ``{`` OUTSIDE the parameter list + (paren depth 0, the house convention), so destructured parameters + are skipped (the pinned functions carry no string literal with a + stray brace).""" + start = js.index(header) + i = start + paren = 0 + while i < len(js): + c = js[i] + if c == "(": + paren += 1 + elif c == ")": + paren -= 1 + elif c == "{" and paren == 0: + break + i += 1 + depth = 0 + for j in range(i, len(js)): + c = js[j] + if c == "{": + depth += 1 + elif c == "}": + depth -= 1 + if depth == 0: + return js[start : j + 1] + raise AssertionError(f"unbalanced braces in {header!r}") + + +def _done_branch(js: str) -> str: + """The SSE ``done`` branch — from the branch test to the next + ``else if`` (flat block, the slice is exact).""" + start = js.index('ev.type === "done"') + return js[start : js.index('ev.type === "tool_result"', start)] + + +# --------------------------------------------------------------------------- +# the glyph + the continuation word +# --------------------------------------------------------------------------- + + +def test_continue_icon_is_an_inline_svg_in_the_stroke_family() -> None: + """``CONTINUE_ICON`` is the inline ``aria-hidden`` SVG in the + RETRY_ICON stroke family (task 01: 24×24, ``fill="none"``, + ``stroke="currentColor"``, ``stroke-width="1.7"`` — the CSS themes + the stroke) — no icon font, no CDN (A11).""" + js = _js() + start = js.index("const CONTINUE_ICON =") + seg = js[start :] + q1 = seg.index("'") + q2 = seg.index("'", q1 + 1) + literal = seg[q1 + 1 : q2] # the first string literal = the inline SVG + for attr in ( + 'aria-hidden="true"', + 'viewBox="0 0 24 24"', + 'fill="none"', + 'stroke="currentColor"', + 'stroke-width="1.7"', + ): + assert attr in literal, f"the glyph keeps the shared family: {attr!r}" + assert literal.startswith("") + + +def test_continue_question_is_the_literal_word_continue() -> None: + """``CONTINUE_QUESTION`` is the literal word ``continue`` — the + word the model is taught to ask the user for (task 02's + ROUND_CAP_WRAPUP_INSTRUCTION / the spent-budget trailer), so the + visible bubble, the typed equivalent, and the model's own + suggestion all agree.""" + js = _js() + assert 'const CONTINUE_QUESTION = "continue";' in js + + +def test_continue_icon_is_defined() -> None: + """EXACTLY ONE ``CONTINUE_ICON`` constant (the glyph is shared by + the button, never duplicated).""" + js = _js() + assert js.count("const CONTINUE_ICON =") == 1 + assert js.count("const CONTINUE_QUESTION =") == 1 + + +# --------------------------------------------------------------------------- +# appendContinueButton — the house pattern +# --------------------------------------------------------------------------- + + +def test_append_continue_button_joins_the_actions_row() -> None: + """``appendContinueButton(wrap)`` keeps the house pattern (the + appendRetryButton shape, task 02): the getActionsRow lookup (the + dedicated actions row, NEVER the chips row), the null no-op, the + one-per-bubble guard (``.continue-btn``), the role=list → + listitem ARIA join, the icon + the visible "Continue researching" + label (the label carries the accessible name), and the click → + ``continueTurn()`` binding.""" + js = _js() + body = _fn_body(js, "function appendContinueButton(wrap) {") + assert "getActionsRow(wrap)" in body, "the shared get-or-create helper" + assert "if (!meta) return" in body, "no-op without a .msg-body" + assert 'querySelector(".continue-btn")' in body, ( + "the one-per-bubble guard" + ) + assert 'meta.getAttribute("role") === "list"' in body, ( + "the role=list check (the row is always role=list)" + ) + assert 'setAttribute("role", "listitem")' in body, "the listitem join" + assert 'btn.className = "continue-btn"' in body + assert 'btn.innerHTML = CONTINUE_ICON + "Continue researching"' in body, ( + "icon + the visible label (the accessible name)" + ) + assert "btn.addEventListener(\"click\", () => continueTurn())" in body, ( + "the click handler — a plain continueTurn() (no wrap, no args)" + ) + assert "meta.appendChild(btn)" in body + # NOT admin-gated, like Retry (chat is public). + assert "isAdmin" not in body + + +def test_continue_button_lands_after_the_retry_button() -> None: + """``appendContinueButton`` is defined DIRECTLY AFTER + ``appendRetryButton`` (task 02's placement — the sibling in the + actions-row family, before the Edit button's block).""" + js = _js() + retry = js.index("function appendRetryButton(wrap) {") + cont = js.index("function appendContinueButton(wrap) {") + edit = js.index("function appendEditButton(wrap) {") + assert retry < cont < edit + + +# --------------------------------------------------------------------------- +# continueTurn — the retryLastTurn discipline +# --------------------------------------------------------------------------- + + +def test_continue_turn_bails_and_clears_and_sends_plain() -> None: + """``continueTurn()`` (task 03): (1) no-op while a turn is in + flight (the ``retryLastTurn`` guard — one turn at a time, no + double-send); (2) clears EVERY rendered ``.continue-btn`` FIRST + (the markLastRetryable remove-first pattern — a stale button on an + earlier bubble is gone the moment the new turn's answer lands); + (3) a PLAIN send — ``runTurn(CONTINUE_QUESTION)`` with NO + ``reask`` argument (the default ``false``): the user bubble + appears, the record is pushed + saved at the ordinary save + points, and the request carries the full history.""" + js = _js() + body = _fn_body(js, "function continueTurn() {") + assert ( + "if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;" + in body + ), "the in-flight no-op (the retryLastTurn guard)" + assert ( + 'messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());' + in body + ), "every continue button is cleared before the send" + assert "return runTurn(CONTINUE_QUESTION)" in body, ( + "a PLAIN send — the returned promise (the retryLastTurn " + "convention)" + ) + assert "reask" not in body, ( + "NO reask argument — a plain send (runTurn's default false)" + ) + order = [ + body.index("UI_STATE.thinking"), + body.index(".remove()"), + body.index("runTurn(CONTINUE_QUESTION)"), + ] + assert order == sorted(order), ( + "guard → clear-every-button → send, in that order" + ) + + +def test_continue_turn_sends_one_plain_turn_with_full_history() -> None: + """The plain-send contract on the ``runTurn`` side (task 03's + save-point list): with ``reask`` defaulting to false, the turn + appends the user bubble, pushes ``{ who: "user", text }`` (no + image — a text-only record), runs the ordinary save points + (``saveConversation`` + ``persistConversation``), and the request's + ``history`` is the FULL ``conversation.slice(0, -1)`` — the model + resumes with everything it already read.""" + js = _js() + rt = _fn_body(js, "function runTurn(text, { reask = false, image = null } = {}) {") + assert "if (!reask) {" in rt, "the default is a plain send" + assert 'conversation.push(' in rt + assert 'saveConversation();' in rt + assert "persistConversation();" in rt + assert "const history = conversation.slice(0, -1).map(" in rt, ( + "the request carries the full history (everything except the " + "just-pushed question)" + ) + + +# --------------------------------------------------------------------------- +# the done settle — the flag rides the record + the button is gated +# --------------------------------------------------------------------------- + + +def test_done_handler_persists_the_flag_and_appends_the_button() -> None: + """The ``done`` settle (task 04): ``budget_exhausted`` rides + ``rememberBrainTurn``'s meta as + ``ev.budget_exhausted || undefined`` (the ``answer_image`` + precedent — ``undefined`` drops the key from the JSON, so every + non-cap record stays byte-identical), every earlier bubble's + continue button is cleared FIRST (the completion criterion — + "neither does any earlier answer once a newer turn lands" — the + markLastRetryable remove-first pattern), and the button is + appended ONLY when the frame is flagged.""" + js = _js() + branch = _done_branch(js) + assert "budget_exhausted: ev.budget_exhausted || undefined" in branch, ( + "the flag persists with the turn (undefined drops the key)" + ) + assert ( + 'messagesEl.querySelectorAll(".continue-btn").forEach((b) => b.remove());' + in branch + ), "every earlier bubble's continue button is cleared on the settle" + assert "if (ev.budget_exhausted) appendContinueButton(wrap);" in branch, ( + "the button is appended ONLY on a flagged done" + ) + order = [ + branch.index('querySelectorAll(".continue-btn")'), + branch.index("appendContinueButton(wrap)"), + ] + assert order == sorted(order), "clear-every-button → gated append" + + +def test_done_branch_row_order_is_tune_retry_continue_save() -> None: + """The row order on a capped answer (task 04): Tune → Retry (the + markLastRetryable re-append) → Continue (flag-gated) → Save-as-doc + — the INSERTION order phase 130 relies on (the DOM order inside + the row is the visual order).""" + js = _js() + branch = _done_branch(js) + order = [ + branch.index("appendTuneButton(wrap);"), + branch.index("markLastRetryable();"), + branch.index("appendContinueButton(wrap);"), + branch.index('appendSaveAsDocButton(wrap, finalText || acc || "…");'), + ] + assert order == sorted(order), ( + "done: Tune → Retry → Continue → Save-as-doc (the insertion " + "order phase 130 relies on)" + ) + + +def test_continue_button_is_not_added_on_the_other_settle_paths() -> None: + """The Continue button is a CAP-REACHED-DONE affordance only: the + zero-frame fallback, the stop finalize, and the failed-turn funnel + (none of them sees a ``budget_exhausted`` frame — a failed or + stopped turn never hit the cap) append NO continue button, and the + ``done`` branch's flag-gate is the single gate.""" + js = _js() + assert js.count("appendContinueButton(wrap);") == 2, ( + "exactly TWO call sites (with the terminating semicolon — the " + "function header ends in a brace, not a semicolon): the done " + "settle + the restore path" + ) + zero = js.index("if (!aborted && !wrap) {") + block = js[zero : js.index("} catch (err) {", zero)] + assert "appendContinueButton" not in block, ( + "the zero-frame fallback (a failed turn) adds no continue button" + ) + funnel = _fn_body( + js, + "function finalizeFailedTurn(\n" + " detail,\n" + " { acc, thinking, tools, segments, wrap, leavePartialIndex }\n" + ") {", + ) + assert "appendContinueButton" not in funnel, ( + "the failed-turn funnel adds no continue button" + ) + + +# --------------------------------------------------------------------------- +# the restore path + startNewChat +# --------------------------------------------------------------------------- + + +def test_restore_branch_keys_on_the_persisted_marker() -> None: + """The phase-14 restore (``renderStoredMessage``'s brain branch, + task 05): the Continue affordance restores from the persisted + ``m.budget_exhausted`` marker — a reload keeps the button on a + flagged answer (a restored ``history`` is exactly what the + continuation turn needs), and a pre-phase / non-cap record (no + key) restores byte-identically (no button). The append lands + BEFORE the ``lastBrainWrap`` assignment, after the per-message + appends (the actions row is the bubble's LAST row, as on the + live done path).""" + js = _js() + body = _fn_body(js, "function renderStoredMessage(m) {") + assert "if (m.budget_exhausted) appendContinueButton(wrap);" in body, ( + "the restore keys on the persisted marker" + ) + order = [ + body.index("appendFailedNote(wrap, m.error);"), + body.index("if (m.budget_exhausted) appendContinueButton(wrap);"), + body.index("lastBrainWrap = wrap"), + ] + assert order == sorted(order), ( + "restore: the per-message appends → the continue button → " + "lastBrainWrap (the after-the-loop markLastRetryable lands the " + "Retry in the same actions row, as on the live path)" + ) + + +def test_start_new_chat_unchanged_drops_the_buttons_with_the_list() -> None: + """``startNewChat`` (task 05: needs nothing) — its list reset + removes every ``.msg`` bubble, so the continue buttons die with + the list (the phase-49 reasoning). The reset + the in-flight + guard stay byte-pinned.""" + js = _js() + body = _fn_body(js, "function startNewChat() {") + assert ( + "if (uiState === UI_STATE.thinking || uiState === UI_STATE.streaming) return;" + in body + ), "the in-flight guard stays" + assert 'messagesEl.querySelectorAll(".msg").forEach((el) => el.remove());' in body, ( + "the list reset drops the buttons along with the list" + ) + assert "appendContinueButton" not in body, ( + "no continue-button wiring was added to startNewChat" + ) + + +# --------------------------------------------------------------------------- +# NOT touched — the phase's explicit contract (byte-pinned sources) +# --------------------------------------------------------------------------- + +#: The FULL source of ``markLastRetryable`` as of pre-phase-160 task +#: 04 — the remove-then-append shape (remove every rendered +#: .retry-btn FIRST, then re-append on the last brain bubble) is +#: UNCHANGED: the continue button has its OWN clear pass in +#: continueTurn, and markLastRetryable's targeting is untouched. +PINNED_MARK_LAST_RETRYABLE = """function markLastRetryable() { + messagesEl.querySelectorAll(".retry-btn").forEach((b) => b.remove()); + if (!lastBrainWrap) return; + let lastIdx = -1; + for (let i = conversation.length - 1; i >= 0; i -= 1) { + if (conversation[i].who === "brain") { + lastIdx = i; + break; + } + } + const prev = lastIdx > 0 ? conversation[lastIdx - 1] : null; + if (!prev || prev.who !== "user") return; + appendRetryButton(lastBrainWrap); +}""" + +#: The FULL source of ``rememberBrainTurn`` as of pre-phase-160 task +#: 04 — the ``...meta`` spread means the new ``budget_exhausted`` +#: optional key rides the record with ZERO change to this helper +#: (``undefined`` drops the key from the JSON, the house +#: optional-meta pattern). +PINNED_REMEMBER_BRAIN_TURN = """function rememberBrainTurn(rawText, meta, replaceIndex = -1) { + const rec = { who: "brain", text: rawText || "…", ...meta }; + if (replaceIndex >= 0 && conversation[replaceIndex]?.who === "brain") { + conversation[replaceIndex] = rec; + } else { + conversation.push(rec); + } + saveConversation(); + // Phase 55 (A2): the auto-save rides the brain save point — the row + // updates with the (possibly replaced) brain turn + metadata. The + // pagehide partial reuses this helper, so it rides the same path (no + // extra wiring). + persistConversation(); +}""" + + +def test_mark_last_retryable_is_byte_unchanged() -> None: + """``markLastRetryable`` is byte-unchanged by phase 160 (task 04 + — the redo-in-place targeting is REUSED, not extended): the full + source must match the pin (a diff is a contract violation).""" + assert _fn_body(_js(), "function markLastRetryable() {") == PINNED_MARK_LAST_RETRYABLE + + +def test_remember_brain_turn_is_byte_unchanged() -> None: + """``rememberBrainTurn`` is byte-unchanged by phase 160 (task 04 — + the ``budget_exhausted`` meta key rides the existing ``...meta`` + spread, so the save point is consumed as-is): the full source must + match the pin.""" + assert _fn_body(_js(), "function rememberBrainTurn(rawText, meta, replaceIndex = -1) {") == ( + PINNED_REMEMBER_BRAIN_TURN + ) + + +def test_retry_last_turn_is_byte_unchanged() -> None: + """``retryLastTurn`` is byte-unchanged by phase 160 (task 04 — the + continue button REUSES its discipline, the function is consumed + as-is): the full source must match the phase-120 pin (the shared + constant — one pin, no drift).""" + assert _fn_body(_js(), "function retryLastTurn(wrap) {") == PINNED_RETRY_LAST_TURN + + +# --------------------------------------------------------------------------- +# the shared page (work item 8) — the Continue control never renders there +# --------------------------------------------------------------------------- + + +def test_shared_page_never_shows_the_continue_control() -> None: + """Work item 8 (no code change expected): the shared page + (``/shared/``, served server-side — ``shared.html``) + renders the conversation through ``shared.js``'s OWN read-only + renderer, never through ``app.js`` — so the ``budget_exhausted`` + key is IGNORED there by construction: the page loads no + ``app.js`` (no ``appendContinueButton`` in its script graph), + and ``shared.js`` builds NO buttons, no forms, and no + ``.msg-actions`` row (owner-locked: zero interactive controls — + the "no buttons, no forms, no links, no click handlers" + contract), so a flagged record renders without any Continue + affordance.""" + html = SHARED_HTML.read_text(encoding="utf-8") + assert "app.js" not in html, ( + "the shared page never loads app.js — its action buttons " + "(incl. Continue) are not in its script graph" + ) + shared = SHARED_JS.read_text(encoding="utf-8") + assert "continue-btn" not in shared + assert "msg-actions" not in shared, ( + "shared.js never builds the actions row — zero controls" + ) + assert 'createElement("button")' not in shared, ( + "owner-locked: NO buttons are ever created on the shared page" + ) + assert 'addEventListener("click"' not in shared, ( + "owner-locked: NO click handlers on the shared page (the img " + "load-failure degradation handler is not a control — the " + "zero-interactive-controls contract)" + ) + + +# --------------------------------------------------------------------------- +# styles.css — the .continue-btn family +# --------------------------------------------------------------------------- + + +def test_continue_btn_css_is_the_retry_family() -> None: + """``.continue-btn`` is the exact visual family of ``.retry-btn`` + (task 07): the same pill size/spacing (``min-height: 44px``, + ``gap: 0.35rem``, ``padding: 0.35rem 0.8rem``, + ``border-radius: 999px``, ``border: 1px solid var(--line)``), + the same neutral color (``--ink-soft`` → ``--ink`` on hover over + ``--brand-soft``), the 14px currentColor glyph, and the shared + GLOBAL ``:focus-visible`` ring (styles.css L162 — the ring rule is + not per-class, so no selector-list extension is needed). No + animation of its own → the ``prefers-reduced-motion`` blocks need + nothing for it (the family's parity — .retry-btn / .edit-btn have + none either).""" + css = _css() + start = css.index(".continue-btn {") + end = css.index("}", start) + block = css[start : end + 1] + for decl in ( + "display: inline-flex;", + "gap: 0.35rem;", + "min-height: 44px;", + "padding: 0.35rem 0.8rem;", + "border-radius: 999px;", + "border: 1px solid var(--line);", + "background: transparent;", + "color: var(--ink-soft);", + "font-weight: 600;", + "font-size: 0.82rem;", + "white-space: nowrap;", + "cursor: pointer;", + ): + assert decl in block, f"the pill family keeps {decl!r}" + assert ".continue-btn svg { width: 14px; height: 14px; display: block; }" in css + assert ".continue-btn:hover { background: var(--brand-soft); color: var(--ink); }" in css + + +def test_continue_btn_mobile_block_min_height() -> None: + """The mobile block (task 07): ``.continue-btn`` joins the + ``.tune-btn`` / ``.retry-btn`` / ``.edit-btn`` 44px min-height + lockstep next to ``.retry-btn`` — EXACTLY ONE such single-line + rule (the base rule is a multi-line block, so the one-line form + occurs only in the mobile media block).""" + css = _css() + assert css.count(".continue-btn { min-height: 44px; }") == 1 + idx = css.index(".continue-btn { min-height: 44px; }") + # It sits beside the family's other lockstep lines (near them in + # the same mobile block). + retry = css.index(".retry-btn { min-height: 44px; }") + assert abs(idx - retry) < 200, ( + "the continue line is beside .retry-btn in the mobile block" + ) diff --git a/tests/unit/test_image_chips_and_answers.py b/tests/unit/test_image_chips_and_answers.py index 9a0bc5f..002825a 100644 --- a/tests/unit/test_image_chips_and_answers.py +++ b/tests/unit/test_image_chips_and_answers.py @@ -565,6 +565,88 @@ def test_done_event_answer_image_ref_drops_its_own_null_image_url() -> None: assert "image_url" not in dump["answer_image"] +# --- the ``budget_exhausted`` done key (phase 160, task 03) --- + + +def test_done_event_without_budget_exhausted_drops_the_key() -> None: + """``budget_exhausted=False`` (the default — every deflected / + in-cap / pre-phase turn) → ``model_dump()`` has NO + ``budget_exhausted`` key — ABSENT, never ``false`` — and a full + done frame (sources + related + suggestions, no flag) serializes + byte-identically to the pre-phase expectation (the + byte-identical-off contract, PLAN §13).""" + event = ChatDoneEvent( + deflected=False, + sources=[SourceRef(source="docs", path="a.md", title="A")], + related=[SourceRef(source="docs", path="b.md", title="B")], + suggestions=["Maybe try X?"], + ) + dump = event.model_dump() + assert "budget_exhausted" not in dump + # The pre-phase-160 key set, exactly (answer_image absent too). + assert set(dump) == {"type", "deflected", "sources", "related", "suggestions"} + assert _json_dumps(dump, sort_keys=True) == _json_dumps( + { + "type": "done", + "deflected": False, + "sources": [{"source": "docs", "path": "a.md", "title": "A"}], + "related": [{"source": "docs", "path": "b.md", "title": "B"}], + "suggestions": ["Maybe try X?"], + }, + sort_keys=True, + ) + raw = _json_dumps(dump) + assert "budget_exhausted" not in raw + + +def test_done_event_with_budget_exhausted_carries_the_key() -> None: + """``budget_exhausted=True`` (a cap-reached turn) → the key is + present and ``true``, the rest of the frame untouched; an + explicit ``False`` behaves exactly like the default (absent).""" + dump = ChatDoneEvent(deflected=False, sources=[], budget_exhausted=True).model_dump() + assert dump["budget_exhausted"] is True + assert dump["type"] == "done" + assert dump["deflected"] is False + assert dump["related"] == [] + assert dump["suggestions"] == [] + explicit_false = ChatDoneEvent( + deflected=False, sources=[], budget_exhausted=False + ).model_dump() + assert "budget_exhausted" not in explicit_false + + +def test_done_event_budget_exhausted_and_answer_image_are_independent() -> None: + """Both optional keys shown on the same turn: the two omission + rules are independent — a cap-reached turn with a shown answer + image carries BOTH keys; the answer-image ref's own serializer + still applies inside the frame.""" + dump = ChatDoneEvent( + deflected=False, + sources=[], + answer_image=SourceRef(**_SHOWN_REF), + budget_exhausted=True, + ).model_dump() + assert dump["budget_exhausted"] is True + assert dump["answer_image"] == _SHOWN_REF + + +def test_done_event_without_budget_exhausted_parses_a_pre_phase_frame() -> None: + """Back-compat: a pre-phase-160 done frame (no + ``budget_exhausted`` key) parses with the default ``False`` and + re-dumps byte-identically (PLAN §4 — the field is additive in + both directions).""" + old_payload = { + "type": "done", + "deflected": True, + "sources": [], + "related": [], + "suggestions": [], + } + event = ChatDoneEvent(**old_payload) + assert event.budget_exhausted is False + assert event.model_dump() == old_payload + + # --- the ``ChatMessage`` boundary (the saved-chat record key) --- diff --git a/tests/unit/test_prompts_dates.py b/tests/unit/test_prompts_dates.py index e00cebc..e652788 100644 --- a/tests/unit/test_prompts_dates.py +++ b/tests/unit/test_prompts_dates.py @@ -259,7 +259,9 @@ def test_read_result_plain_carries_date_second_line( ) asyncio.run(_run(llm, holder, _settings())) assert holder.read_docs == [doc] and holder.tool_calls == 1 - content = llm.requests[1][0][3]["content"] + # Phase 160 (task 01): the budget-telemetry trailer rides the read + # result — the raw result is the RAW part (the trailer stripped). + content = llm.requests[1][0][3]["content"].removesuffix(agent._budget_note(9)) lines = content.splitlines() assert lines[0] == "Document S/a.md:" # byte-identical first line assert lines[1] == f"date: {DATE}" # the D5 second line @@ -291,7 +293,9 @@ def test_read_result_truncated_carries_date_then_marker_and_notice( + "\n" + READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content_body)) ) - assert llm.requests[1][0][3]["content"] == expected + # Phase 160 (task 01): the trailer rides the truncated result too — + # the truncation contract (phase 95) is the RAW part. + assert llm.requests[1][0][3]["content"].startswith(expected) # The truncation still records on the holder (the ToolResultPiece # signal) — the date line adds nothing to the counters. assert holder.read_truncations == [("S/big.md", cap, len(content_body))] diff --git a/tests/unit/test_schemas.py b/tests/unit/test_schemas.py index 567fe07..8029219 100644 --- a/tests/unit/test_schemas.py +++ b/tests/unit/test_schemas.py @@ -386,6 +386,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, }, { "who": "brain", @@ -434,6 +435,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, }, { "who": "user", @@ -447,6 +449,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, }, { "who": "brain", @@ -460,6 +463,7 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: "stopped": True, # the owner stopped the generation mid-answer "failed": None, "error": None, + "budget_exhausted": None, }, ] @@ -468,7 +472,10 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: dumped = payload.model_dump() # the stored shape is byte-identical: every message dict — None-keys # and all — round-trips unchanged (and the top-level record keeps its - # ``share`` default). + # ``share`` default). The phase-160 additive ``budget_exhausted`` + # marker joins as an explicit null (the phase-120 ``failed`` / + # phase-48 ``stopped`` style — ``None`` flows as null, the + # phase-50 byte-identical contract covers the new key). assert dumped["messages"] == messages assert dumped["share"] is False @@ -488,8 +495,11 @@ def test_realistic_payload_round_trips_through_update_model() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, } payload = SavedChatUpdate.model_validate({"messages": [msg]}) + # Identical stored shape (the phase-160 additive ``budget_exhausted`` + # marker rides as an explicit null — the phase-120/48 style). assert payload.model_dump()["messages"] == [msg] @@ -708,8 +718,11 @@ def test_segments_absent_by_default_and_omitted_in_dump() -> None: dumped = msg.model_dump() assert "segments" not in dumped # Byte-identical to the pre-phase stored shape: every bor.chat.v1 - # optional key present (null), the three serializer-popped keys - # (image / answer_image / segments) absent. + # optional key present (null) — the phase-160 additive + # ``budget_exhausted`` marker included (the phase-120 ``failed`` / + # phase-48 ``stopped`` style — explicit null where absent) — and + # the three serializer-popped keys (image / answer_image / + # segments) absent. assert dumped == { "who": "brain", "text": "ok", @@ -722,6 +735,7 @@ def test_segments_absent_by_default_and_omitted_in_dump() -> None: "stopped": None, "failed": None, "error": None, + "budget_exhausted": None, } # An explicit empty list is a different thing: [] is a value, so it # round-trips as [] (present, empty — the pop is None-gated only).