phase: 192_verbatim_inline_midturn
Phase 192 — final verification pass complete. - **Verified (all 4 tasks already complete):** additive `verbatim_block` SSE frame (`VerbatimBlockPiece` → `ChatVerbatimBlockEvent` → chat dispatch, after the round's `tool`/`tool_result`, before the next round); `hydrateVerbatimNow` matched-only mid-turn pass + app.js branch; `dropPlacedTwin` per-locator done/restore dedupe; the dedicated E2E suite. - **Root cause of the rejected attempt:** my own concurrent background `pytest` run deadlocked the harness validation run on the shared Postgres (`psycopg.errors.DeadlockDetected`). No code defect — re-ran every gate **exclusively**. - `uv run pytest -q --cov=app --cov-report=term-missing` → **exit 0**, `TOTAL 7181 16 99%` (>90%) - `uv run pytest tests/e2e/test_verbatim_inline_midturn.py --no-cov` → **6 passed** (isolated) - `uv run pytest tests/e2e/test_verbatim_show_text.py tests/e2e/test_chat_history.py --no-cov` → **18 passed** (`test_chat_history.py` byte-unedited; the phase-182 suite's "no eighth type" pin was tightened, not loosened, to seven + `verbatim_block` with frame-order asserts) - `uv run ruff check .` → All checks passed (exit 0) · `uv run pyright` → **0 errors** - **Completion criteria:** single frame per success + zero per refusal ✅ · `BOR_ANSWER_TEXT=0` wire byte-identical (integration spelled baseline + E2E App B) ✅ · fence hydrates in place / unfenced lands at the call moment, before `done` ✅ · append+fence → 1 panel, two ranges → 2 panels ✅ · restore 1 panel/locator, `done.answer_text` + record key + `verbatim_blocks=N` unchanged ✅ · suite green, coverage 99%, lint/types clean ✅ · no-show_text turn byte-identical ✅ · commit → left to the harness (no git index/history commands run). - **Docs:** README reviewed line-by-line (2399 lines) + AGENTS.md in full; mechanically verified all 86 `BOR_*` settings, every `.env.example` var, table defaults, internal anchors (0 broken), ports 8123/8901, tree depth 1–8, pagination 25/1–100, 17 palette colors, `0029`/`0030` migrations, and every cited test file. Fixed 1 inconsistency: the verbatim "Parity" bullet claimed the live turn/reload/shared page share "the same three globals" — now names the settle-path trio and states that `hydrateVerbatimNow` is the ONE chat-only addition (a shared page has no stream). - **Notable:** `run_agent`'s four repeated piece unions were collapsed into the ONE exported `AgentPiece` alias, so a future piece type is spelled in one place; `test_frontend_edit_message.py`'s `runTurn` byte-pin was regenerated, not weakened. - **Next pending phase:** `193_sync_remaining_count`.
This commit is contained in:
+1
@@ -0,0 +1 @@
|
||||
All 4 tasks are complete. This is the final verification pass. Let me review the reports and completions.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+13026
File diff suppressed because it is too large
Load Diff
+12
@@ -0,0 +1,12 @@
|
||||
Phase 192 — final verification pass complete.
|
||||
|
||||
- **Verified (all 4 tasks already complete):** additive `verbatim_block` SSE frame (`VerbatimBlockPiece` → `ChatVerbatimBlockEvent` → chat dispatch, after the round's `tool`/`tool_result`, before the next round); `hydrateVerbatimNow` matched-only mid-turn pass + app.js branch; `dropPlacedTwin` per-locator done/restore dedupe; the dedicated E2E suite.
|
||||
- **Root cause of the rejected attempt:** my own concurrent background `pytest` run deadlocked the harness validation run on the shared Postgres (`psycopg.errors.DeadlockDetected`). No code defect — re-ran every gate **exclusively**.
|
||||
- `uv run pytest -q --cov=app --cov-report=term-missing` → **exit 0**, `TOTAL 7181 16 99%` (>90%)
|
||||
- `uv run pytest tests/e2e/test_verbatim_inline_midturn.py --no-cov` → **6 passed** (isolated)
|
||||
- `uv run pytest tests/e2e/test_verbatim_show_text.py tests/e2e/test_chat_history.py --no-cov` → **18 passed** (`test_chat_history.py` byte-unedited; the phase-182 suite's "no eighth type" pin was tightened, not loosened, to seven + `verbatim_block` with frame-order asserts)
|
||||
- `uv run ruff check .` → All checks passed (exit 0) · `uv run pyright` → **0 errors**
|
||||
- **Completion criteria:** single frame per success + zero per refusal ✅ · `BOR_ANSWER_TEXT=0` wire byte-identical (integration spelled baseline + E2E App B) ✅ · fence hydrates in place / unfenced lands at the call moment, before `done` ✅ · append+fence → 1 panel, two ranges → 2 panels ✅ · restore 1 panel/locator, `done.answer_text` + record key + `verbatim_blocks=N` unchanged ✅ · suite green, coverage 99%, lint/types clean ✅ · no-show_text turn byte-identical ✅ · commit → left to the harness (no git index/history commands run).
|
||||
- **Docs:** README reviewed line-by-line (2399 lines) + AGENTS.md in full; mechanically verified all 86 `BOR_*` settings, every `.env.example` var, table defaults, internal anchors (0 broken), ports 8123/8901, tree depth 1–8, pagination 25/1–100, 17 palette colors, `0029`/`0030` migrations, and every cited test file. Fixed 1 inconsistency: the verbatim "Parity" bullet claimed the live turn/reload/shared page share "the same three globals" — now names the settle-path trio and states that `hydrateVerbatimNow` is the ONE chat-only addition (a shared page has no stream).
|
||||
- **Notable:** `run_agent`'s four repeated piece unions were collapsed into the ONE exported `AgentPiece` alias, so a future piece type is spelled in one place; `test_frontend_edit_message.py`'s `runTurn` byte-pin was regenerated, not weakened.
|
||||
- **Next pending phase:** `193_sync_remaining_count`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
...................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 301 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/source_labels.py 52 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 231 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 39 0 100%
|
||||
app/config.py 332 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/memory.py 32 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 117 0 100%
|
||||
app/core/theming.py 45 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 107 0 100%
|
||||
app/models.py 148 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 501 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 865 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 82 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 178 0 100%
|
||||
app/rag/retriever.py 270 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_labels.py 87 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 587 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 7181 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
All gates green. Report:
|
||||
|
||||
**Implemented (task 01 — the additive `verbatim_block` SSE frame, A1)**
|
||||
- `app/rag/agent.py`: new frozen `@dataclass VerbatimBlockPiece` (`block: VerbatimBlock`); the round loop snapshots `shown_before = len(holder.shown_texts)` before `_execute_tool` and yields one piece per appended block, AFTER the round's `tool`/`tool_result` pieces and BEFORE the next model round (success-only — every refusal returns before the `shown_texts.append`); module + `run_agent` docstrings carry the emission-order contract.
|
||||
- Named the widened generator union once as `type AgentPiece = StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece | VerbatimBlockPiece` (task text spelled it literally; the alias keeps `run_agent`, the chat pump and 6 test harnesses on one spelling — the literal widening broke E501/pyright in 19 annotation sites).
|
||||
- `app/schemas.py`: `ChatVerbatimBlockEvent` right after `ChatToolResultEvent` — `type: Literal["verbatim_block"]` + the exact `VerbatimBlockRef` field set/caps, no `model_config` (sibling convention mirrored).
|
||||
- `app/api/chat.py`: the `VerbatimBlockPiece` dispatch branch after the `ToolResultPiece` branch; done-frame `answer_text` and the `verbatim_blocks=N` log line untouched.
|
||||
- Tests: 10 unit cases in `tests/unit/test_agent.py` (piece shape, success position + holder-object identity, order after a truncation piece, 5 refusal params, already-shown dedupe, kill switch), 2 in `tests/unit/test_schemas.py` (wire shape + a field/caps drift pin against `VerbatimBlockRef`), 2 in `tests/integration/test_chat_api.py` (`verbatim_block` between `tool` and `delta`/`done` with the exact key set + same block still on `done`, `verbatim_blocks=1`; kill-switch frame sequence byte-identical to the spelled-out pre-phase baseline).
|
||||
|
||||
**Gates** — `uv run pytest --cov=app --cov-report=term-missing` → 4777 tests, exit 0, TOTAL 99% (chat.py 99%, agent.py 99%, schemas.py 100%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. E2E in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed, `test_chat_history.py` + `test_answer_images.py` 10 passed, `test_source_name_labels.py` 7 passed.
|
||||
|
||||
**Docs** — README verbatim section: new "The panel lands while the answer streams" bullet, the `done`-frame bullet's now-false "no eighth SSE event type" corrected, `BOR_ANSWER_TEXT` row notes no frame is sent; AGENTS.md rule 5 phase-182 clause + rule 10 `verbatim_blocks=N` clause updated.
|
||||
|
||||
**Deviations / notes**
|
||||
- `tests/e2e/test_verbatim_show_text.py` pinned "SSE type set stays inside the pre-phase seven (no eighth type)" — invalidated by this phase's design; updated to "the seven + `verbatim_block`" and extended with the mid-turn ordering pin (that suite is otherwise green unedited).
|
||||
- `PLAN.md` §2 A15 / §4 still list seven event types and "no eighth SSE event type"; PLAN.md is out of my edit scope — flag for the phase-04 docs pass / owner revision note.
|
||||
- No live-infra work in this task; all changes left unstaged for the harness.
|
||||
|
||||
**Next pending task:** `192_verbatim_inline_midturn/02_midturn_hydration_and_append.md` (`hydrateVerbatimNow` + the `app.js` `verbatim_block` branch).
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
......................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 301 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/source_labels.py 52 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 231 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 39 0 100%
|
||||
app/config.py 332 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/memory.py 32 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 117 0 100%
|
||||
app/core/theming.py 45 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 107 0 100%
|
||||
app/models.py 148 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 501 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 865 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 82 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 178 0 100%
|
||||
app/rag/retriever.py 270 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_labels.py 87 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 587 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 7181 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
All gates green. Report:
|
||||
|
||||
**Implemented (task 02 — mid-turn landing, A2)**
|
||||
- `frontend/assets/verbatim.js`: new `hydrateVerbatimNow(root, block, opts)` — walks the done path's pending set (`.verbatim-pending[data-bor-locator]`), matches through the ONE `matchVerbatimBlock` rule with a one-block list, replaces ONLY matched nodes with `buildVerbatimPanel`, leaves every unmatched fence `.verbatim-pending`, returns the settle count, total (no `try`) via `isVerbatimBlock` + root guards.
|
||||
- `frontend/assets/app.js`: the `ev.type === "verbatim_block"` branch beside `tool_result` — hydrate in place, `if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS)`; no client-side kill-switch gate (server-only). No import line existed/changed: `verbatim.js` is a classic script whose top-level functions are globals (task text's "import site" = the existing global call sites).
|
||||
- **Pulled forward phase design A3** (`verbatimPlacedKey` + `dropPlacedTwin`, called from `hydrateVerbatim`): without it, a mid-turn-appended block that the model fences in a later round painted TWO panels and the phase-182 E2E suite (`test_verbatim_show_text.py`, 5 tests) went red — the "one locator, one panel" rule (per canonical locator, `.msg`-bounded, pending/unresolved never a placement) was the only way to leave the repo green.
|
||||
- Pins in `tests/unit/test_frontend_verbatim_blocks.py`: marker pins (`hydrateVerbatimNow` matched-only contract, never-throw, the app.js zero-match-append shape + asserted absence of any client gate, the twin-scan shape) plus 4 new node-driven behavioural tests on a minimal DOM (in-place settle + untouched sibling fence, server-extent fallback, malformed/no-match/null-root arms, mid-turn panel survives the done pass, one-locator-one-panel across passes incl. two ranges → two panels).
|
||||
- `tests/unit/test_frontend_edit_message.py`: regenerated `PINNED_RUN_TURN` to the sanctioned phase-192 branch (verified the old pin == HEAD `runTurn` first) + docstring clause; `verbatimPlacedKey` now the single key derivation shared with `appendUnlocatedVerbatim`.
|
||||
|
||||
**Gates** — `uv run pytest --cov=app --cov-report=term-missing` → **4784 passed**, TOTAL **99%**; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors; E2E in isolation `--no-cov`: `test_verbatim_show_text.py` + `test_chat_history.py` + `test_answer_images.py` + `test_share_chat.py` + `test_chat_persistence.py` → 31 passed (the phase-182 suite green again, unedited assertions).
|
||||
|
||||
**Docs** — README verbatim section: the client half of the mid-turn landing (in-place vs answer-position, twin dropped, one locator one panel, no client flag); AGENTS.md rule 5 phase-182 clause extended with the mid-turn matched-only contract + the per-locator twin scan, re-tagged "phase 182 task 07, phase 192 tasks 01–02".
|
||||
|
||||
**Decisions / deviations**
|
||||
- Task 02's checkbox "hydrateVerbatim untouched" is unsatisfiable together with a green suite (the duplicate is structural); resolved by implementing A3 exactly as the phase overview specifies. **Task 03 should verify/extend, not re-implement** — remaining there: the restore/shared-parity pin and its own marker set.
|
||||
- Side effect, documented + pinned: a fence the model repeats for the SAME locator now shows one panel (at the last fence) instead of two.
|
||||
- No live-infra work; all changes left unstaged for the harness.
|
||||
|
||||
**Next pending task:** `192_verbatim_inline_midturn/03_done_and_restore_dedupe.md`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 301 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/source_labels.py 52 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 231 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 39 0 100%
|
||||
app/config.py 332 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/memory.py 32 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 117 0 100%
|
||||
app/core/theming.py 45 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 107 0 100%
|
||||
app/models.py 148 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 501 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 865 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 82 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 178 0 100%
|
||||
app/rag/retriever.py 270 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_labels.py 87 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 587 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 7181 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
**Task 03 complete — all gates green.**
|
||||
|
||||
Implemented (task 03, A3/A4):
|
||||
- `frontend/assets/verbatim.js`: `hydrateVerbatim`'s twin-scan host is now `root.closest(".msg") || root` — a missing `.msg` wrap narrows the scan to the root instead of switching the per-locator dedupe off (`dropPlacedTwin` itself, `appendUnlocatedVerbatim`, `settleVerbatimPlaceholders` and the task-02 functions untouched).
|
||||
- `tests/unit/test_frontend_verbatim_blocks.py`: new `_doc`/`_flow` docstring-pin helpers + `bareFence` DOM fixture; pins `test_the_twin_dedupe_docstring_keeps_the_per_locator_clause` (per-locator-not-per-identity + two-ranges-stay + in-text-wins prose), `test_the_twin_scan_bounds_to_the_msg_wrap_or_the_root`, node arms `test_the_twin_scan_bounds_to_the_message_it_hydrates` (bare-bubble dedupe, two ranges stay two, a sibling message is out of bounds) and `test_a_stored_record_restores_one_panel_per_locator` (restore/shared parity behaviour).
|
||||
- `tests/integration/test_chat_api.py`: `test_done_frame_and_log_line_unchanged_by_midturn_frames` — two `show_text` rounds → two `verbatim_block` frames each inside its own round, `done.answer_text` byte-equal to the phase-182 resolution spelled from the row (new `_BACKUPS`/`_doc_title` helpers), done key set unchanged, and the ONE log line reads `verbatim_blocks=2` (never 4 — mid-turn frames counted nowhere).
|
||||
- `tests/unit/test_schemas.py`: `test_chat_verbatim_block_event_dump_keys_are_the_client_contract` (dump key set == `VerbatimBlockRef` keys + `type`, client-guard keys present, ref round-trip).
|
||||
- Verified the A3 code and several pins already existed from task 02; I closed only the genuine gaps (fallback, docstring pin, bound arms, the named integration/log pin, the schema key-set pin) rather than duplicating, and confirmed `app.js` restore lines and `shared.js` are untouched (`git diff`: app.js +24 lines = the new branch only).
|
||||
|
||||
Gates (exact commands):
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → **4790 passed**, `TOTAL 7181 16 99%` (>90%)
|
||||
- `uv run pytest tests/unit/test_frontend_verbatim_blocks.py tests/integration/test_chat_api.py -v --no-cov` → **124 passed**
|
||||
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
|
||||
- `uv run pytest tests/e2e/test_verbatim_show_text.py --no-cov` → 13 passed; `tests/e2e/test_chat_history.py` → 5 passed
|
||||
|
||||
Docs: `README.md` verbatim section now states the dedupe's bound (`.msg` wrap, root fallback, never crosses into the next answer); `AGENTS.md` rule 5 `dropPlacedTwin` clause gained the bound and the attribution is now "phase 192 tasks 01–03".
|
||||
|
||||
Next pending task: `.agents/phases/todo/192_verbatim_inline_midturn/04_e2e_and_gates.md`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
...................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 301 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/source_labels.py 52 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 231 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 39 0 100%
|
||||
app/config.py 332 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/memory.py 32 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 117 0 100%
|
||||
app/core/theming.py 45 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 107 0 100%
|
||||
app/models.py 148 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 501 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 865 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 82 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 178 0 100%
|
||||
app/rag/retriever.py 270 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_labels.py 87 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 587 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 7181 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,28 @@
|
||||
**Task 04 complete — all gates green.**
|
||||
|
||||
Added (`tests/e2e/test_verbatim_inline_midturn.py`, NEW — 6 tests, module docstring names TODO.md:6 and the LOCKED-A4 unchanged-contract clause; ports 8209/8210, the phase-182 module-app/KB/cleanup conventions):
|
||||
|
||||
| Test | Arm | Pins |
|
||||
|---|---|---|
|
||||
| `test_midturn_panel_lands_before_the_answer_streams` | (a) | panel in the DOM **before `done`** (in-page `MutationObserver` witness: `doneSeen=false`, `sentinelInDom=false`, pill `"Stop"`, `forLocator=1`), below the intro the model wrote pre-call, server caption/gutter/body/badge, clipboard == the server string, band order bubble → verbatim → `.msg-meta`, `done` adds nothing (count 1), frame order `delta* → tool → verbatim_block → delta* → done` with each delta segment byte-equal to the mock's text |
|
||||
| `test_midturn_fence_hydrates_in_place_not_at_the_answer_tail` | (b) | placeholder exists first → `hydrateVerbatimNow` settles it **in place** mid-turn (inside `.bubble`, nothing appended), one panel after `done`, intro < panel < sentinel, no model bytes |
|
||||
| `test_midturn_append_and_a_later_fence_leave_one_panel` | (c) | mid-turn append at the answer position, `done` re-homes it onto the same-locator fence and drops the twin — exactly ONE panel at the fence |
|
||||
| `test_two_ranges_of_one_file_land_two_panels` | (d) | `tool → block → tool → block`, two panels in call order, own captions/gutter starts, `done.answer_text` = both |
|
||||
| `test_kill_switch_off_ships_the_pre_phase_wire` | (e) | App B: `Unknown tool.` refusal, **no** `verbatim_block`, no `answer_text`, tool frame an exact dict, delta bytes == App A's answer bytes, zero panels/placeholders witnessed |
|
||||
| `test_restore_repaints_one_panel_for_the_midturn_locator` | (f) | record `answer_text` unchanged, reload + server-record path (localStorage cleared) → ONE panel, caption/bytes/badge/gutter, no turn re-run |
|
||||
|
||||
Mock (`tests/e2e/mock_llm.py`, additive): the `MIDTURN_TRIGGER` flow + `MIDTURN_*` constants and answer strings, `MIDTURN_GAP_S` silence between the frame and the answer's first delta, arm markers, the two-range stateless branch, the module-docstring bullet. Phase-182's pins and every other flow untouched.
|
||||
|
||||
Docs: `README.md` — the mid-turn landing clause (already carried tasks 01–03) re-wrapped; `AGENTS.md` rule 5 attribution now `phase 192 tasks 01–04` and names `tests/e2e/test_verbatim_inline_midturn.py` as the owner-visible pin. `.agents/PLAN.md` needed no correction (it never claimed `done` was the only landing point).
|
||||
|
||||
Gates (exact commands):
|
||||
- `uv run pytest -q --cov=app --cov-report=term-missing` → **4790 passed, exit 0**, `TOTAL 7181 16 99%` (>90%)
|
||||
- `uv run pytest tests/e2e/test_verbatim_inline_midturn.py -v --no-cov` → **6 passed**, exit 0 (isolated)
|
||||
- `uv run pytest tests/e2e/test_verbatim_show_text.py tests/e2e/test_chat_history.py -v --no-cov` → **18 passed**, exit 0 (unedited)
|
||||
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
|
||||
|
||||
Notes / deviations:
|
||||
1. Arm (a)'s literal "ABOVE the final sentinel delta" cannot hold as sibling order — an unfenced block has no fence to return to, so `appendUnlocatedVerbatim` parks it at the phase-182 answer band (below the answer text, above the chips). The suite pins the stronger mid-turn claim (the sentinel did not exist when the panel landed) and asserts the literal between-the-two position in the fenced arms (b)/(c), where intro < panel < sentinel inside the bubble. Documented in the test docstring.
|
||||
2. Two mock corrections the arms needed (documented in place): `show_text`'s `path` argument must be the COMBINED `source/path` (a bare path is the phase-72 refusal), and a round-1 fence must be padded with a blank line, because the client concatenates rounds with no separator — an unpadded closing ` ``` ` glues to the next round's first word and stops being a fence.
|
||||
3. Owner-visible observation, NOT changed (out of task scope): when the fence is written *before* the call, the next `delta` re-renders the bubble from `acc` (which still holds the fence), so the node returns to `.verbatim-pending` until `done` settles it again. Both ends are asserted and the interim is the honest phase-182 state (never the model's bytes); making the per-delta renderer preserve hydrated panels would remove the interim and deserves its own phase/task.
|
||||
4. Left to the harness (no index/history-mutating commands run): moving `todo/192_verbatim_inline_midturn/04_e2e.md` → `complete/` and the single atomic `--no-gpg-sign` commit.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 13%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 37%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 42%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 85%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
...................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 301 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/source_labels.py 52 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 231 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 39 0 100%
|
||||
app/config.py 332 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/memory.py 32 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 117 0 100%
|
||||
app/core/theming.py 45 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 107 0 100%
|
||||
app/models.py 148 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 501 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 865 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 82 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 178 0 100%
|
||||
app/rag/retriever.py 270 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_labels.py 87 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 587 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 7181 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -59,7 +59,13 @@
|
||||
caption and a `user-select: none` line-number gutter, while the
|
||||
plain `.code-block` wrapper carries a Copy button and no badge
|
||||
(phase 182); those placeholders hydrate against the frame's
|
||||
`answer_text` through the ONE panel client
|
||||
`answer_text` — or, since phase 192, against the additive
|
||||
`verbatim_block` SSE frame (the eighth event type: one
|
||||
`done.answer_text` item plus `type`) that carries a successful
|
||||
`show_text` block the MOMENT the call succeeds, AFTER that round's
|
||||
`tool` / `tool_result` frames and BEFORE the next model round, so the
|
||||
panel lands where the model introduced the bytes instead of popping in
|
||||
after the answer — through the ONE panel client
|
||||
(`frontend/assets/verbatim.js` — a classic script loaded AFTER
|
||||
`markdown.js` on the chat AND shared pages, no second locator grammar,
|
||||
a plain string compare against the server's canonical `locator`, the
|
||||
@@ -68,10 +74,23 @@
|
||||
knowledge base` with no badge, no body and none of the model's bytes,
|
||||
no panel left at "Retrieving the exact text…" after a terminal frame
|
||||
(the `error` frame, the 300 s guard, Stop and the failure funnel all
|
||||
settle it), the Copy button's clipboard content equal to the server
|
||||
string exactly, and the shared page's panel Copy button + plain
|
||||
`/document.html` link the ONE owner-directed exception to that page's
|
||||
zero-controls rule) — phase 182 task 07), and the sync's peak-memory
|
||||
settle it), a MID-TURN pass (`hydrateVerbatimNow`) that settles ONLY the
|
||||
pending placeholders its ONE block matches — every other fence STAYS
|
||||
`.verbatim-pending`, because burning one is how a still-resolvable fence
|
||||
paints the honest sentence — and ONE canonical locator painting ONE panel
|
||||
(`dropPlacedTwin`: the fence wins over the mid-turn appended twin, the scan
|
||||
is PER LOCATOR so two honest ranges of one file stay two panels, it is
|
||||
bounded by the `.msg` wrap (or the root when there is no wrap — a missing
|
||||
wrap narrows the scan, it never switches the dedupe off), and a
|
||||
pending/unresolved node is never a placement), the Copy button's clipboard
|
||||
content equal to the server string exactly, and the shared page's panel Copy
|
||||
button + plain `/document.html` link the ONE owner-directed exception to that
|
||||
page's zero-controls rule) — phase 182 task 07, phase 192 tasks 01–04, the
|
||||
mid-turn landing (panel on screen BEFORE `done`, the fence winning the
|
||||
same-locator race, two ranges staying two panels, the `answer_text=False`
|
||||
wire byte-identical) pinned through the UI by
|
||||
`tests/e2e/test_verbatim_inline_midturn.py`), and
|
||||
the sync's peak-memory
|
||||
clause is TRAILING TEXT on the result line that already exists
|
||||
(` · peak <N> MB` on the RAG page's `#sync-progress` counts line
|
||||
or "Sync cancelled", written by the ONE `fmtSyncPeakClause` off
|
||||
@@ -105,7 +124,9 @@
|
||||
10. **Amply log, visibly feed back:** server-side, log the required
|
||||
per-turn line (PLAN §9 — phase 182 appends `verbatim_blocks=N` after
|
||||
`scaffold_stripped=N`: the verbatim display blocks actually SENT on
|
||||
the `done` frame, `0` on every turn without them, deflected included)
|
||||
the `done` frame, `0` on every turn without them, deflected included —
|
||||
never the phase-192 mid-turn `verbatim_block` frames, which are not
|
||||
counted anywhere else)
|
||||
**and** the per-response byte-budget line
|
||||
(`app/core/response_budget.py` — one
|
||||
`api method=… path=… status=… bytes=… enc=… duration_ms=…` line per
|
||||
|
||||
@@ -1327,6 +1327,27 @@ The model never retypes quoted text, so a displayed quote cannot drift.
|
||||
each block is capped at `BOR_SHOW_TEXT_MAX_CHARS`, cut at a line boundary
|
||||
with a truncation marker. Image documents are refused with a pointer to
|
||||
`show_image` — the two display tools never convert into each other.
|
||||
- **The panel lands while the answer streams.** A successful `show_text`
|
||||
sends its block TWICE on purpose (phase 192): once as the additive
|
||||
`verbatim_block` SSE frame — the eighth event type,
|
||||
`{type, source, path, title, start_line, end_line, total_lines, text,
|
||||
truncated, locator}`, i.e. exactly one `done.answer_text` item — emitted
|
||||
AFTER that round's `tool` / `tool_result` frames and BEFORE the next
|
||||
model round, so the panel appears where the model introduced the bytes
|
||||
instead of after it finished talking; and once on the `done` frame below.
|
||||
A refused call sends nothing, and `BOR_ANSWER_TEXT=0` offers no tool at
|
||||
all, so such a turn's frame sequence is byte-identical to pre-phase
|
||||
(pinned on the wire). Client-side, `hydrateVerbatimNow` settles the pending
|
||||
fence that block names IN PLACE the moment the frame arrives and leaves every
|
||||
other placeholder pending; when the answer has not fenced it (yet), the panel
|
||||
lands at the `answer_image` position then — not at the end of the answer. If
|
||||
the model fences that same locator in a later round, hydration drops the
|
||||
appended twin, so **one canonical locator paints exactly one panel**, at the
|
||||
fence (two honest ranges of one file stay two panels: the scan is per
|
||||
locator, bounded by the message's `.msg` wrap — or by the root when there is
|
||||
no wrap, so a missing wrap narrows the scan instead of switching the dedupe
|
||||
off, and one answer never reaches into the next one's panels). There is no
|
||||
client-side flag check — the server's gate is the only one.
|
||||
- **The bytes ride the `done` frame.** A turn that displayed text ends
|
||||
with one extra, additive SSE key: `done.answer_text`, the list of blocks
|
||||
in **display order** — answer order first (one per ` ```bor ` fence, the
|
||||
@@ -1338,10 +1359,13 @@ The model never retypes quoted text, so a displayed quote cannot drift.
|
||||
1-based inclusive extent — the numbers a caption shows can never be the
|
||||
model's claim. `locator` is the canonical matching key
|
||||
(`source/path`, or `source/path#12-40` for a range). There is no new
|
||||
bytes route and no eighth SSE event type: the bytes travel inline, and a
|
||||
saved or shared chat renders its panels from its own record. The same
|
||||
array persists on the saved brain record (`bor.chat.v1` `answer_text`),
|
||||
so a reload and an anonymous shared page show the same blocks.
|
||||
bytes route: the bytes travel inline (the only extra frame is the
|
||||
phase-192 `verbatim_block` above, carrying that same item), and a saved
|
||||
or shared chat renders its panels from its own record — the `done`
|
||||
frame, the saved record and the restore path are UNCHANGED by phase 192.
|
||||
The same array persists on the saved brain record (`bor.chat.v1`
|
||||
`answer_text`), so a reload and an anonymous shared page show the same
|
||||
blocks.
|
||||
- **Deflected turns resolve nothing.** A ` ```bor ` fence written into a
|
||||
deflected answer leaks no KB text — the disclaimer turn resolves no
|
||||
block and its frame carries no `answer_text` key.
|
||||
@@ -1379,8 +1403,10 @@ The model never retypes quoted text, so a displayed quote cannot drift.
|
||||
|
||||
- **Hydration paints the panel, and the panel is the file.** The client half
|
||||
is one classic script, `frontend/assets/verbatim.js`, loaded after
|
||||
`markdown.js` on the chat page and the shared page. At `done` (and on every
|
||||
restore) it matches each placeholder's normalized `data-bor-locator`
|
||||
`markdown.js` on the chat page and the shared page. Mid-turn it settles the
|
||||
one block a `verbatim_block` frame carries (`hydrateVerbatimNow`); at `done`
|
||||
(and on every restore) it matches each placeholder's normalized
|
||||
`data-bor-locator`
|
||||
against the frame's `answer_text[].locator` with a plain string compare —
|
||||
`markdown.js::borLocator` stays the ONLY locator grammar in the client, so
|
||||
the two sides cannot drift — and paints:
|
||||
@@ -1426,8 +1452,14 @@ The model never retypes quoted text, so a displayed quote cannot drift.
|
||||
exists once (the owner-locked share strings are unchanged; phase 189 added
|
||||
the save-before-the-token upsert in front of the same success paths).
|
||||
- **Parity, and what stays untouched.** The live turn, a reload and an
|
||||
anonymous shared page call the same three globals in the same order, so one
|
||||
answer renders one way. Only the two answer renderers (plus the shared
|
||||
anonymous shared page call the same three **settle-path** globals in the same
|
||||
order — `hydrateVerbatim` → `appendUnlocatedVerbatim` →
|
||||
`enhanceAnswerCodeBlocks` — so one answer renders one way. The phase-192
|
||||
mid-turn pass (`hydrateVerbatimNow`) is the ONE chat-only addition: a shared
|
||||
page has no stream to land a panel mid-turn, so it hydrates from its record
|
||||
through the same trio (which is why a reload and a shared page show the same
|
||||
panel the live turn showed at the `show_text` call). Only the two answer
|
||||
renderers (plus the shared
|
||||
page's) opt in: a question, a thinking run and the document viewer stay
|
||||
one-argument calls, and `document.html` does not even load the panel client.
|
||||
|
||||
@@ -1886,7 +1918,7 @@ startup (the DB rows shadow the env, `.env` is never rewritten).
|
||||
|-----|---------|---------|
|
||||
| `BOR_AGENT_MAX_ROUNDS` | `10` | Hard cap on agent tool rounds per grounded turn (0 = no tools); every tool result carries the model's remaining count (phase 160) |
|
||||
| `BOR_READ_MAX_CHARS` | `128000` | Cap on a `read` tool result (chars); over-cap reads truncate + a notice that teaches the grep → `read(offset, limit)` workflow (phase 95, rewritten phase 131) |
|
||||
| `BOR_ANSWER_TEXT` | `1` | Verbatim file display (phase 182): the **master switch, ON by default** — an answer's code block is typed by the server from the indexed bytes, never retyped by the model. `0` = the escape hatch: the `show_text` tool is not offered, its `<tools>` teaching clause is absent, ` ```bor ` locator fences are not resolved, and the `done` frame carries no `answer_text` key (the server surface is exactly pre-phase; a stray fence on the client still settles to the honest unresolved sentence rather than the model's copy) |
|
||||
| `BOR_ANSWER_TEXT` | `1` | Verbatim file display (phase 182): the **master switch, ON by default** — an answer's code block is typed by the server from the indexed bytes, never retyped by the model. `0` = the escape hatch: the `show_text` tool is not offered, its `<tools>` teaching clause is absent, ` ```bor ` locator fences are not resolved, no `verbatim_block` frame is sent, and the `done` frame carries no `answer_text` key (the server surface — frame sequence included — is exactly pre-phase; a stray fence on the client still settles to the honest unresolved sentence rather than the model's copy) |
|
||||
| `BOR_SHOW_TEXT_MAX_CHARS` | `12000` | Per-block character budget for ONE displayed verbatim slice (phase 182): the slice is cut at a **line boundary**, the shared `[…truncated…]` marker is appended and the caption's line range shrinks to what is actually shown. `0` = inert (nothing displays); a negative value refuses startup naming the field |
|
||||
| `BOR_SHOW_TEXT_MAX_BLOCKS` | `5` | Per-answer cap on displayed verbatim blocks (phase 182), counted in answer order. `0` = inert (nothing displays); a negative value refuses startup naming the field |
|
||||
|
||||
|
||||
+59
-9
@@ -251,6 +251,24 @@ keeps its exact pre-phase bytes). The per-turn log line records
|
||||
ACTUALLY SENT (after dedupe and the ``show_text_max_blocks`` cut), 0 on
|
||||
every turn without them including deflected ones (the PLAN §9 additive
|
||||
revision).
|
||||
|
||||
Mid-turn verbatim frame (phase 192, task 01, A1; ``TODO.md`` L5): the
|
||||
same block ALSO reaches the client the moment the ``show_text`` call
|
||||
succeeds, as the additive ``verbatim_block`` event —
|
||||
``{"type": "verbatim_block", "source", "path", "title", "start_line",
|
||||
"end_line", "total_lines", "text", "truncated", "locator"}`` (the exact
|
||||
:class:`~app.schemas.VerbatimBlockRef` field set, so the one panel client
|
||||
accepts it as-is). It lands AFTER the round's ``tool`` / ``tool_result`` frames and
|
||||
BEFORE the next model round — while the answer is still streaming, so the
|
||||
panel appears where the model introduced the bytes instead of popping in
|
||||
after it finished talking. Success-only: every refusal (missing ``path``,
|
||||
no document, an image row, the block cap, the already-shown dedupe) emits
|
||||
nothing, and a ``BOR_ANSWER_TEXT=0`` turn offers no tool, so its frame
|
||||
sequence is byte-identical to pre-phase. Everything downstream of the
|
||||
turn keeps the phase-182 contract EXACTLY: the ``done`` frame's
|
||||
``answer_text`` list, the saved record key, the restore path and the
|
||||
shared page are untouched, and ``verbatim_blocks=N`` still counts the
|
||||
``done``-frame blocks (never the mid-turn frames).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -275,7 +293,9 @@ from app.models import Document, QueryLog
|
||||
from app.rag.agent import (
|
||||
CORRECTION_INSTRUCTION, # phase 71: the harness-owned recovery line
|
||||
AgentHolder,
|
||||
AgentPiece, # the ONE union run_agent yields (phase 192: + VerbatimBlockPiece)
|
||||
MalformedReplyError, # phase 71: the recovery policy's terminal signal
|
||||
VerbatimBlockPiece, # phase 192: one shown verbatim block (mid-turn frame)
|
||||
run_agent,
|
||||
suggested_folder_lines, # phase 119 (D3): the suggested-folder context lines
|
||||
)
|
||||
@@ -285,7 +305,6 @@ from app.rag.llm import (
|
||||
LLMClient,
|
||||
LLMError,
|
||||
RetryPiece, # phase 67: one LLM request restart (an SSE retry frame)
|
||||
StreamPiece, # type of the answer pieces streamed by the agent loop
|
||||
ToolCallPiece, # phase 37: one model-requested tool call
|
||||
ToolResultPiece, # phase 95: one truncated tool result (SSE frame = task 02)
|
||||
chat_stream_retried, # phase 67: the retry-before-first-piece primitive
|
||||
@@ -323,6 +342,7 @@ from app.schemas import (
|
||||
ChatThinkingEvent,
|
||||
ChatToolEvent,
|
||||
ChatToolResultEvent,
|
||||
ChatVerbatimBlockEvent,
|
||||
SourceRef,
|
||||
VerbatimBlockRef,
|
||||
)
|
||||
@@ -974,9 +994,11 @@ async def chat(
|
||||
# ``agent_max_rounds=0`` ``run_agent`` is a single
|
||||
# ``tools=None`` request anyway (the kill switch).
|
||||
holder = AgentHolder()
|
||||
answer_stream: AsyncIterator[
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
|
||||
]
|
||||
# Phase 192: ``AgentPiece`` is the ONE union ``run_agent``
|
||||
# yields (the phase-95 truncation piece plus the new
|
||||
# ``VerbatimBlockPiece``); the deflected path's narrower
|
||||
# ``chat_stream_retried`` union flows into it unchanged.
|
||||
answer_stream: AsyncIterator[AgentPiece]
|
||||
deflected_filter: ScaffoldingFilter | None = None
|
||||
if plan.deflected:
|
||||
# Phase 67: the deflected stream goes through the retry
|
||||
@@ -1038,9 +1060,7 @@ async def chat(
|
||||
tools_offered = not plan.deflected
|
||||
|
||||
async def _pump(
|
||||
pieces: AsyncIterator[
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
|
||||
],
|
||||
pieces: AsyncIterator[AgentPiece],
|
||||
tools_offered: bool = True,
|
||||
) -> AsyncIterator[str]:
|
||||
"""One request's piece loop (phase 71 extraction): the
|
||||
@@ -1058,10 +1078,14 @@ async def chat(
|
||||
frame, no visible markup) and counted in a local so one
|
||||
debug line can name it. The grounded first pass keeps its
|
||||
current behavior (``tools_offered=True``; ``run_agent``
|
||||
suppresses the cap-path calls internally)."""
|
||||
suppresses the cap-path calls internally). Phase 192
|
||||
(task 01): the ``VerbatimBlockPiece`` branch emits the
|
||||
additive ``verbatim_block`` frame — the verbatim block a
|
||||
successful ``show_text`` just displayed, so its panel
|
||||
lands mid-stream instead of at ``done``."""
|
||||
nonlocal thinking_chars, content_chars, retries_used
|
||||
dropped_calls = 0 # phase 160: tools=None requests only
|
||||
async for piece in pieces: # StreamPiece | ToolCallPiece | RetryPiece
|
||||
async for piece in pieces: # app.rag.agent.AgentPiece
|
||||
if isinstance(piece, ToolCallPiece):
|
||||
if not tools_offered:
|
||||
# Phase 160 (task 02): a tools=None request
|
||||
@@ -1170,6 +1194,32 @@ async def chat(
|
||||
).model_dump()
|
||||
)
|
||||
continue
|
||||
if isinstance(piece, VerbatimBlockPiece):
|
||||
# Phase 192 (task 01, A1): the mid-turn verbatim
|
||||
# frame — the block the ``show_text`` call just
|
||||
# displayed, sent at the moment it exists (the
|
||||
# panel lands while the answer is still
|
||||
# streaming instead of at ``done``). Additive:
|
||||
# a refused call yields no piece (no frame), a
|
||||
# ``answer_text=False`` turn offers no tool (no
|
||||
# frame) — the frame sequence is byte-identical
|
||||
# to pre-phase in both cases, and the ``done``
|
||||
# frame's ``answer_text`` list and its
|
||||
# ``verbatim_blocks=N`` count are untouched.
|
||||
yield sse_event(
|
||||
ChatVerbatimBlockEvent(
|
||||
source=piece.block.source,
|
||||
path=piece.block.path,
|
||||
title=piece.block.title,
|
||||
start_line=piece.block.start_line,
|
||||
end_line=piece.block.end_line,
|
||||
total_lines=piece.block.total_lines,
|
||||
text=piece.block.text,
|
||||
truncated=piece.block.truncated,
|
||||
locator=piece.block.locator,
|
||||
).model_dump()
|
||||
)
|
||||
continue
|
||||
if piece.kind == "thinking":
|
||||
thinking_chars += len(piece.text)
|
||||
if settings.stream_thinking:
|
||||
|
||||
+88
-3
@@ -233,7 +233,17 @@ task 04):
|
||||
:data:`SHOW_TEXT_NOT_A_TEXT_DOC` refusal, the exact mirror of
|
||||
:data:`NOT_AN_IMAGE_DOC` — never a silent conversion), and the inline
|
||||
``` ```bor ``` locator-fence form of the same contract (phase 182 D2)
|
||||
is resolved at the API layer (task 05), NOT here.
|
||||
is resolved at the API layer (task 05), NOT here. **Phase 192 (task
|
||||
01, A1): the block reaches the user at the CALL moment, not at the
|
||||
end of the answer** — the round loop snapshots
|
||||
``len(holder.shown_texts)`` before the execution and yields one
|
||||
:class:`VerbatimBlockPiece` for whatever the call appended, which the
|
||||
API layer turns into the additive ``verbatim_block`` SSE frame (the
|
||||
panel lands while the model is still talking). Success-only, like the
|
||||
holder entry: a refusal appends nothing and yields nothing. The
|
||||
``done`` frame's ``answer_text`` list and the per-turn
|
||||
``verbatim_blocks=N`` count (the done-frame blocks) are UNCHANGED —
|
||||
that log field never counts the mid-turn frames.
|
||||
4. Every call the model emits is appended back to the message history as
|
||||
the assistant tool-call message + the tool result (refusals included),
|
||||
consumes one round, and the model is called again. Every tool result
|
||||
@@ -344,6 +354,55 @@ from app.rag.verbatim import VerbatimBlock, block_for
|
||||
|
||||
logger = logging.getLogger("app.agent")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VerbatimBlockPiece:
|
||||
"""One verbatim block a ``show_text`` call JUST displayed (phase 192).
|
||||
|
||||
The mid-turn sibling of :class:`app.rag.llm.ToolResultPiece` (phase
|
||||
95 — the other piece the agent loop yields that does NOT come from
|
||||
the model stream: it is derived from the executed call). Phase 182
|
||||
handed the block to the client only on the ``done`` frame's
|
||||
``answer_text`` list, so every panel popped in AFTER the model had
|
||||
finished talking about the bytes; phase 192 (task 01, A1) sends the
|
||||
very same block the moment it exists, as the additive
|
||||
``verbatim_block`` SSE frame — AFTER the round's ``tool`` /
|
||||
``tool_result`` frames and BEFORE the next model round.
|
||||
|
||||
SUCCESS-ONLY by construction: the single gate is the
|
||||
``holder.shown_texts.append`` in the ``show_text`` branch — every
|
||||
refusal (missing args, the ``answer_text=False`` posture, no
|
||||
document, an image row, the block cap, the already-shown dedupe)
|
||||
returns before it, so a refused call yields NO piece. :attr:`block`
|
||||
is the exact :class:`app.rag.verbatim.VerbatimBlock` the tool put on
|
||||
the holder (the server's bytes and the server's extent — LOCKED
|
||||
A1/A3), never a re-derivation. Frozen like its siblings: an
|
||||
immutable wire value. The ``done`` frame's ``answer_text``
|
||||
list and its ``verbatim_blocks=N`` count are UNCHANGED by this
|
||||
piece (task 05's resolution still runs over the finished answer);
|
||||
the mid-turn frames are not counted there.
|
||||
"""
|
||||
|
||||
#: The block the call displayed — the same object that rode
|
||||
#: ``holder.shown_texts`` (the phase-182 task-03 lineage: the
|
||||
#: server resolved it, the tool appended it on success only).
|
||||
block: VerbatimBlock
|
||||
|
||||
|
||||
#: The ONE union of every piece :func:`run_agent` yields — the model
|
||||
#: stream's pieces (:class:`app.rag.llm.StreamPiece` /
|
||||
#: :class:`~app.rag.llm.ToolCallPiece` / :class:`~app.rag.llm.RetryPiece`)
|
||||
#: plus the pieces DERIVED from an executed call: the phase-95
|
||||
#: :class:`~app.rag.llm.ToolResultPiece` (a truncated ``read``) and the
|
||||
#: phase-192 :class:`VerbatimBlockPiece` (a successful ``show_text``, the
|
||||
#: mid-turn ``verbatim_block`` frame). Named so the wire consumers — the
|
||||
#: ``app.api.chat`` pump and the test harnesses — spell the SAME union and
|
||||
#: a future piece type is added in exactly one place.
|
||||
type AgentPiece = (
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece | VerbatimBlockPiece
|
||||
)
|
||||
|
||||
|
||||
#: The three agent tools (phase 70: the harness-aligned surface —
|
||||
#: ``ls`` / ``read`` / ``grep``, the pi.dev tool shapes the model was
|
||||
#: trained on, replacing the phase-37 list/read and phase-68 search
|
||||
@@ -2413,7 +2472,7 @@ async def run_agent(
|
||||
settings: Settings,
|
||||
holder: AgentHolder,
|
||||
history: Sequence[dict[str, Any]] = (),
|
||||
) -> AsyncIterator[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> AsyncIterator[AgentPiece]:
|
||||
"""Run the grounded-turn tool loop, yielding every stream piece.
|
||||
|
||||
Every piece (``thinking`` / ``content`` / tool calls /
|
||||
@@ -2423,7 +2482,18 @@ async def run_agent(
|
||||
longer than ``settings.read_max_chars``) additionally yields one
|
||||
:class:`ToolResultPiece` per truncated call, AFTER the round's
|
||||
``tool`` frame and BEFORE the next model round (the API layer turns it
|
||||
into an SSE ``tool_result`` frame, task 02). After the loop finishes,
|
||||
into an SSE ``tool_result`` frame, task 02). A successful
|
||||
``show_text`` (phase 182) additionally yields exactly ONE
|
||||
:class:`VerbatimBlockPiece` per block it appended to
|
||||
``holder.shown_texts``, with the SAME emission-order contract —
|
||||
AFTER the round's ``tool`` / ``tool_result`` frames and BEFORE the
|
||||
next model round — so the API layer can put the verbatim panel on
|
||||
screen the moment the call succeeds instead of at ``done`` (phase
|
||||
192, task 01, A1). Emission rides the ``shown_texts`` append, which
|
||||
is the branch's SINGLE success gate: every refusal (missing ``path``,
|
||||
the ``answer_text=False`` posture, no document, an ``is_image`` row,
|
||||
the ``show_text_max_blocks`` cap, the already-shown dedupe) returns
|
||||
before it and emits nothing. After the loop finishes,
|
||||
*holder* carries the read documents, the executed tool-call count
|
||||
(re-lists included), the ``read_truncations`` signal list, and the
|
||||
``read_coverage`` spans (phase 141, task 01 — the lines each
|
||||
@@ -2640,6 +2710,12 @@ async def run_agent(
|
||||
# round, so at most one new entry — the loop still iterates the
|
||||
# tail, so a future multi-call round stays correct).
|
||||
trunc_before = len(holder.read_truncations)
|
||||
# Phase 192 (task 01, A1): the same snapshot-then-yield shape for
|
||||
# the verbatim blocks — only what THIS call appended becomes a
|
||||
# ``VerbatimBlockPiece`` (a round adds at most one, the
|
||||
# already-shown dedupe being a refusal, but the tail loop keeps a
|
||||
# future multi-add correct).
|
||||
shown_before = len(holder.shown_texts)
|
||||
with db_factory() as tool_db:
|
||||
result = _execute_tool(tool_db, call, seed_docs, holder, settings)
|
||||
rounds += 1 # every call the model emits consumes a round
|
||||
@@ -2694,6 +2770,15 @@ async def run_agent(
|
||||
chars_shown=shown,
|
||||
chars_total=total,
|
||||
)
|
||||
# Phase 192 (task 01, A1): surface this round's verbatim block(s)
|
||||
# to the API layer — the block the ``show_text`` call JUST
|
||||
# displayed, sent at the moment it exists instead of at ``done``.
|
||||
# Same emission order as the truncation piece above (AFTER the
|
||||
# round's ``tool`` / ``tool_result`` frames, BEFORE the next model
|
||||
# round); task 02 turns it into the additive ``verbatim_block``
|
||||
# SSE frame. A refused call appended nothing, so nothing is sent.
|
||||
for block in holder.shown_texts[shown_before:]:
|
||||
yield VerbatimBlockPiece(block=block)
|
||||
if rounds >= max_rounds:
|
||||
logger.warning(
|
||||
"agent round cap reached (rounds=%d) — forcing a final "
|
||||
|
||||
@@ -347,6 +347,46 @@ class ChatToolResultEvent(BaseModel):
|
||||
chars_total: int = Field(ge=0) # the document's true length
|
||||
|
||||
|
||||
class ChatVerbatimBlockEvent(BaseModel):
|
||||
"""SSE frame for one verbatim block a ``show_text`` call JUST displayed
|
||||
(phase 192, task 01, A1).
|
||||
|
||||
The MID-TURN twin of one :class:`VerbatimBlockRef` item of the ``done``
|
||||
frame's ``answer_text`` list — the same field set with the same caps,
|
||||
sent the moment the block exists instead of at the end of the answer
|
||||
(``TODO.md`` L5: "the verbatim tag appear[s] at the end and only when
|
||||
the llm is done answering which makes the chat read strangely"). The
|
||||
frame is strictly ADDITIVE (the phase-95 ``tool_result`` precedent):
|
||||
it lands AFTER the round's ``tool`` / ``tool_result`` frames and
|
||||
BEFORE the next model round, clients that do not know the type ignore
|
||||
it, a REFUSED ``show_text`` streams NO frame (the holder append is the
|
||||
single success gate), and a ``BOR_ANSWER_TEXT=0`` turn offers no tool
|
||||
at all — its frame sequence is byte-identical to pre-phase. The
|
||||
``done`` frame's ``answer_text`` list, the saved record and the
|
||||
restore path keep the exact phase-182 contract (the client's
|
||||
idempotence keeps the same block from painting twice).
|
||||
|
||||
``isVerbatimBlock`` (``frontend/assets/verbatim.js``) accepts the
|
||||
event as-is: the key set is the done-frame item's key set, so the one
|
||||
panel client hydrates a mid-turn frame with no new shape. The field
|
||||
caps are :class:`VerbatimBlockRef`'s verbatim — a unit pin fails the
|
||||
day the two drift.
|
||||
"""
|
||||
|
||||
type: Literal["verbatim_block"] = "verbatim_block"
|
||||
source: str = Field(max_length=120)
|
||||
path: str = Field(max_length=1000)
|
||||
title: str = Field(max_length=500)
|
||||
start_line: int = Field(ge=0)
|
||||
end_line: int = Field(ge=0)
|
||||
total_lines: int = Field(ge=0)
|
||||
#: The verbatim bytes themselves — SERVER-supplied, never model output
|
||||
#: (the :attr:`VerbatimBlockRef.text` contract, same cap).
|
||||
text: str = Field(max_length=600_000)
|
||||
truncated: bool = False
|
||||
locator: str = Field(max_length=2100)
|
||||
|
||||
|
||||
class ChatDoneEvent(BaseModel):
|
||||
"""Final SSE event of a chat turn: metadata for the finished answer.
|
||||
|
||||
|
||||
@@ -4640,6 +4640,30 @@ async function runTurn(text, { reask = false, image = null } = {}) {
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (ev.type === "verbatim_block") {
|
||||
// Phase 192 (task 02, A2): the show_text panel lands at the moment
|
||||
// the call succeeds — the fence it names hydrates IN PLACE; an
|
||||
// unfenced block lands at the answer position now, not at done.
|
||||
// The done pass re-runs the same functions and can never double this
|
||||
// panel: hydrateVerbatim is idempotent (only .verbatim-pending is
|
||||
// touched), it DROPS this appended twin when the answer fences the
|
||||
// same locator later (dropPlacedTwin, phase 192 A3), and
|
||||
// appendUnlocatedVerbatim skips an already-placed locator. NO
|
||||
// client-side answer_text gate: the server's gate is the only one
|
||||
// (a BOR_ANSWER_TEXT=0 turn sends no frame, so this branch never
|
||||
// fires — a second, client-side gate would only drift).
|
||||
// `wrap` always exists here (the round's `tool` frame created the
|
||||
// live bubble) but the optional chain keeps a frame that arrives
|
||||
// without one a silent no-op, like the `tool_result` branch.
|
||||
const n = hydrateVerbatimNow(
|
||||
wrap?.querySelector(".bubble"),
|
||||
ev,
|
||||
VERBATIM_LINK_OPTS
|
||||
);
|
||||
// The append is the ZERO-match path only: a fence already on screen
|
||||
// keeps its panel IN PLACE (the model's order), and only a block no
|
||||
// placeholder names lands at the answer position.
|
||||
if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);
|
||||
} else if (ev.type === "error") {
|
||||
// Phase 182 (task 07): an `error` frame is terminal — the `done`
|
||||
// hydrate will never run, so a placeholder that streamed in stays at
|
||||
|
||||
+110
-11
@@ -6,11 +6,13 @@
|
||||
* locator, and the SERVER (`app.rag.verbatim.resolve_answer_blocks`,
|
||||
* task 05) puts the real bytes on the `done` frame as `answer_text`. This
|
||||
* file is the other half of that handshake: it RESOLVES the placeholders
|
||||
* against the frame's blocks (`hydrateVerbatim`), places the blocks the
|
||||
* answer never located (`appendUnlocatedVerbatim`), settles leftovers when
|
||||
* a turn ends without a `done` (`settleVerbatimPlaceholders`), gives every
|
||||
* settled answer code block a Copy button (`enhanceAnswerCodeBlocks`) and
|
||||
* owns the ONE clipboard path (`copyTextWithFallback`).
|
||||
* against the frame's blocks (`hydrateVerbatim`), settles the ONE block a
|
||||
* mid-turn `verbatim_block` frame carries (`hydrateVerbatimNow`, phase 192),
|
||||
* places the blocks the answer never located (`appendUnlocatedVerbatim`),
|
||||
* settles leftovers when a turn ends without a `done`
|
||||
* (`settleVerbatimPlaceholders`), gives every settled answer code block a
|
||||
* Copy button (`enhanceAnswerCodeBlocks`) and owns the ONE clipboard path
|
||||
* (`copyTextWithFallback`).
|
||||
*
|
||||
* WHY IT IS A CLASSIC SCRIPT, NOT A MODULE (the `markdown.js` pattern):
|
||||
* `index.html` and `shared.html` load it as a plain `<script>` right after
|
||||
@@ -38,7 +40,13 @@
|
||||
* whose range the model got wrong shows what the server actually read
|
||||
* (A2 made visible);
|
||||
* • a locator the server could not resolve says so — the unresolved node
|
||||
* carries NO badge, NO body and NONE of the model's bytes.
|
||||
* carries NO badge, NO body and NONE of the model's bytes;
|
||||
*
|
||||
* ONE CANONICAL LOCATOR, ONE PANEL (phase 192): the mid-turn frame can place
|
||||
* a block BEFORE the answer fences it, and hydration drops that appended
|
||||
* twin when the fence resolves (`dropPlacedTwin`) — the fence is where the
|
||||
* model wanted the bytes. The scan is PER LOCATOR, so two honest ranges of
|
||||
* one file stay two panels.
|
||||
*
|
||||
* The house frontend rules hold throughout: every node is built with
|
||||
* `createElement` + `textContent` (no HTML string anywhere — the data here
|
||||
@@ -412,6 +420,50 @@ function matchVerbatimBlock(blocks, key) {
|
||||
return samePath.length === 1 ? samePath[0] : null;
|
||||
}
|
||||
|
||||
/* The key a PLACED panel carries for a block — `buildVerbatimPanel`'s own
|
||||
* convention (the server's canonical locator, the `source/path` identity
|
||||
* when the block carries none). The ONE place that key is derived, so the
|
||||
* append-side placed-scan and the hydration-side twin-scan can never drift
|
||||
* (phase 192: the two are the same dedupe seen from either end). */
|
||||
function verbatimPlacedKey(block) {
|
||||
return typeof block.locator === "string" && block.locator
|
||||
? block.locator
|
||||
: verbatimIdentity(block.source, block.path);
|
||||
}
|
||||
|
||||
/* dropPlacedTwin(host, block, keep) — the phase-192 (A3) dedupe,
|
||||
* the twin a MID-TURN frame left behind.
|
||||
*
|
||||
* `hydrateVerbatimNow` appends a block no fence named YET (the call moment —
|
||||
* the model may fence it in a later round). When that fence finally hydrates,
|
||||
* TWO panels would carry the same canonical locator. The IN-TEXT panel wins:
|
||||
* the fence is where the model wanted the bytes, the appended panel already
|
||||
* served its mid-turn purpose. So the placed twin is dropped as its
|
||||
* placeholder resolves.
|
||||
*
|
||||
* PER LOCATOR, never per identity: two different ranges of one file are two
|
||||
* honest panels (`shown_texts` dedupes per canonical locator too), and a
|
||||
* pending or unresolved node is NOT a placement (the same exclusion
|
||||
* `appendUnlocatedVerbatim` uses). `keep` is the placeholder about to
|
||||
* hydrate — never the node this scan removes.
|
||||
*
|
||||
* A fence the model repeated for the SAME locator collapses the same way: the
|
||||
* answer shows those bytes once, at the last fence, never twice. */
|
||||
function dropPlacedTwin(host, block, keep) {
|
||||
if (!host || typeof host.querySelectorAll !== "function") return 0;
|
||||
const placedKey = verbatimPlacedKey(block);
|
||||
let dropped = 0;
|
||||
host.querySelectorAll(".verbatim[data-bor-locator]").forEach((el) => {
|
||||
if (el === keep) return;
|
||||
if (el.classList.contains("verbatim-pending")) return;
|
||||
if (el.classList.contains("verbatim-unresolved")) return;
|
||||
if (el.getAttribute("data-bor-locator") !== placedKey) return;
|
||||
el.remove();
|
||||
dropped += 1;
|
||||
});
|
||||
return dropped;
|
||||
}
|
||||
|
||||
/* hydrateVerbatim(root, blocks, opts) — replace every PENDING placeholder
|
||||
* under `root` with its panel, or with the unresolved state.
|
||||
*
|
||||
@@ -419,14 +471,26 @@ function matchVerbatimBlock(blocks, key) {
|
||||
* this function built (or one restored from storage) is skipped on a re-run,
|
||||
* which keeps a `Copied` state and a scrolled body alive. Never throws —
|
||||
* a missing/malformed `blocks` settles every placeholder to the honest
|
||||
* state instead. Returns the number of placeholders settled. */
|
||||
* state instead. Returns the number of placeholders settled.
|
||||
*
|
||||
* A MATCHED placeholder also drops the panel the mid-turn pass appended for
|
||||
* the same canonical locator (dropPlacedTwin, phase 192 A3) — one locator,
|
||||
* one panel, and it sits where the model fenced it. */
|
||||
function hydrateVerbatim(root, blocks, opts) {
|
||||
if (!root || typeof root.querySelectorAll !== "function") return 0;
|
||||
const list = verbatimBlocks(blocks);
|
||||
// The message wrap is the scan's bound: the mid-turn panel lives in
|
||||
// `.msg-body` beside the bubble, so a bubble-local walk would miss it.
|
||||
// With no `.msg` ancestor (a bare bubble, a fragment not mounted yet) the
|
||||
// ROOT bounds the scan — a missing wrap must not switch the dedupe off.
|
||||
const host =
|
||||
(typeof root.closest === "function" ? root.closest(".msg") : null) || root;
|
||||
let settled = 0;
|
||||
// The NodeList is STATIC, so replacing a node never disturbs the walk.
|
||||
root.querySelectorAll(".verbatim-pending[data-bor-locator]").forEach((node) => {
|
||||
const key = node.getAttribute("data-bor-locator") || "";
|
||||
const block = matchVerbatimBlock(list, key);
|
||||
if (block) dropPlacedTwin(host, block, node);
|
||||
node.replaceWith(
|
||||
block ? buildVerbatimPanel(block, opts) : buildVerbatimUnresolved(key, opts)
|
||||
);
|
||||
@@ -435,6 +499,44 @@ function hydrateVerbatim(root, blocks, opts) {
|
||||
return settled;
|
||||
}
|
||||
|
||||
/* hydrateVerbatimNow(root, block, opts) — the MID-TURN pass (phase 192,
|
||||
* A2): settle the PENDING placeholders THIS ONE block matches, leave
|
||||
* every other placeholder pending.
|
||||
*
|
||||
* The matching is the done path's semantics (matchVerbatimBlock with a
|
||||
* ONE-block list): the exact canonical locator, or the identity
|
||||
* fallback when the block is the only one of that identity — a fence
|
||||
* for range B of a shown file A hydrates with A's server extent (the
|
||||
* caption shows the SERVER's range, the phase-182 contract), so the
|
||||
* mid-turn pass and the done pass can never disagree.
|
||||
*
|
||||
* ONLY matched nodes are replaced (buildVerbatimPanel); the unmatched
|
||||
* pending nodes STAY .verbatim-pending — settling them is the
|
||||
* done/failure path's job (settleVerbatimPlaceholders), and a mid-turn
|
||||
* pass must never burn a placeholder for a not-yet-shown fence.
|
||||
*
|
||||
* Returns the number of placeholders settled (the caller appends the
|
||||
* block — appendUnlocatedVerbatim — only when it is 0). Never throws:
|
||||
* a missing root → 0, and a frame that is not a block settles nothing
|
||||
* (the house never-throw contract — the guard is `isVerbatimBlock`, the
|
||||
* done path's own). */
|
||||
function hydrateVerbatimNow(root, block, opts) {
|
||||
if (!root || typeof root.querySelectorAll !== "function") return 0;
|
||||
// A malformed frame settles NOTHING (not even to the honest state — the
|
||||
// `done` pass and the terminal settle still own this turn's placeholders).
|
||||
if (!isVerbatimBlock(block)) return 0;
|
||||
const list = [block];
|
||||
let settled = 0;
|
||||
root.querySelectorAll(".verbatim-pending[data-bor-locator]").forEach((node) => {
|
||||
const key = node.getAttribute("data-bor-locator") || "";
|
||||
const matched = matchVerbatimBlock(list, key);
|
||||
if (!matched) return; // STAYS pending — the done frame may resolve it
|
||||
node.replaceWith(buildVerbatimPanel(matched, opts));
|
||||
settled += 1;
|
||||
});
|
||||
return settled;
|
||||
}
|
||||
|
||||
/* appendUnlocatedVerbatim(wrap, blocks, opts) — the blocks NO placeholder
|
||||
* matched (a `show_text` call whose answer never located the file), appended
|
||||
* in the frame's order at the `answer_image` position: below the answer
|
||||
@@ -458,10 +560,7 @@ function appendUnlocatedVerbatim(wrap, blocks, opts) {
|
||||
});
|
||||
let appended = 0;
|
||||
for (const block of list) {
|
||||
const key =
|
||||
typeof block.locator === "string" && block.locator
|
||||
? block.locator
|
||||
: verbatimIdentity(block.source, block.path);
|
||||
const key = verbatimPlacedKey(block);
|
||||
if (placed.has(key)) continue;
|
||||
host.appendChild(buildVerbatimPanel(block, opts));
|
||||
placed.add(key);
|
||||
|
||||
@@ -750,6 +750,22 @@ Implements just enough of the aipi surface:
|
||||
for, so the client must land on its honest unresolved state (no badge,
|
||||
no body, none of the model's bytes). Checked BEFORE the
|
||||
``DEFLECT_MODE`` branch, same convention.
|
||||
- user message containing ``walk through the stored text as you go``
|
||||
(``MIDTURN_TRIGGER``, phase 192 task 04 — the mid-turn panel suite
|
||||
``tests/e2e/test_verbatim_inline_midturn.py``) -> the MID-TURN turn the
|
||||
phase-182 flow cannot produce: a content delta (the intro, plus the
|
||||
closed fence in the fence-first arm) THEN the ``show_text`` call, and
|
||||
the final answer only after :data:`MIDTURN_GAP_S` of silence — the
|
||||
window in which the ``verbatim_block`` frame's panel is the only thing
|
||||
on screen. Arm markers, the two ranges of one file, the sentinel token
|
||||
and the byte-exact answer constants are documented at the definitions
|
||||
(``MIDTURN_*``). NOT clause-gated on purpose: the kill-switch arm boots
|
||||
a ``BOR_ANSWER_TEXT=0`` app with the SAME question, so the attempt
|
||||
reaches the server-side ``Unknown tool.`` refusal and the wire can be
|
||||
pinned byte-identical to pre-phase. Checked BEFORE the phase-182
|
||||
``SHOW_TEXT_TRIGGER`` flow (disjoint phrases — the ordering
|
||||
convention); no existing E2E question or fixture file carries the
|
||||
phrase.
|
||||
- user message containing ``which diagram is in your notes``
|
||||
(``DIAGRAM_TRIGGER``, phase 124 task 07 — the BUG-FIX-ONLY
|
||||
trigger, same suite) -> the deterministic ZERO-READ answer: the
|
||||
@@ -1619,6 +1635,157 @@ INLINE_ANSWER = (
|
||||
"And the inline markdown ends here — a plain closing line."
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 192 (task 04): the MID-TURN show_text flow (the phase's own E2E suite)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A user message containing this substring (case-insensitive) drives the
|
||||
#: deterministic phase-192 MID-TURN turn for
|
||||
#: ``tests/e2e/test_verbatim_inline_midturn.py`` — the shape the phase-182
|
||||
#: flow CANNOT produce, because that one streams the tool call with NO
|
||||
#: content in front of it and answers with NO gap behind it:
|
||||
#:
|
||||
#: round 1 ``delta``(the intro prose — and, in the fence-first arm, the
|
||||
#: closed ```bor fence itself) → a ``show_text`` tool_calls delta
|
||||
#: round 2 (the server has by now sent ``tool`` + the task-01
|
||||
#: ``verbatim_block`` frame) → a deliberate
|
||||
#: ``MIDTURN_GAP_S`` silence → the answer's ``delta`` frames,
|
||||
#: the LAST of them carrying ``MIDTURN_SENTINEL`` → ``done``
|
||||
#:
|
||||
#: The gap is the test's observation window: the panel the frame paints is
|
||||
#: on screen while the answer has NOT streamed a single token, so
|
||||
#: "the panel landed mid-turn" is pinned by the DOM itself, not by a poll.
|
||||
#: Arms, discriminated by markers in the question (all disjoint from every
|
||||
#: other trigger, ordering convention intact):
|
||||
#
|
||||
#: * default — no fence anywhere: the mid-turn block is APPENDED at the
|
||||
#: answer position (``appendUnlocatedVerbatim``).
|
||||
#: * ``MIDTURN_FENCE_BEFORE_MARKER`` — the fence rides round 1, so the
|
||||
#: placeholder EXISTS when the frame lands: the mid-turn
|
||||
#: ``hydrateVerbatimNow`` pass resolves it IN PLACE.
|
||||
#: * ``MIDTURN_FENCE_AFTER_MARKER`` — the fence rides the final answer
|
||||
#: (the realistic shape: the model quotes after the call), so the
|
||||
#: mid-turn panel is appended and the ``done`` pass re-homes it onto
|
||||
#: the fence and drops the twin (task 03's ``dropPlacedTwin``).
|
||||
#: * ``MIDTURN_TWO_RANGES_MARKER`` — TWO ``show_text`` calls (the two
|
||||
#: ranges the question names), no fences: two panels, in call order.
|
||||
#: Stateless per the house rule (the tool-result count is the state).
|
||||
#:
|
||||
#: Deliberately NOT gated on ``_SHOW_TEXT_CLAUSE_MARKER``: the kill-switch
|
||||
#: arm boots a ``BOR_ANSWER_TEXT=0`` app and asks the SAME question, where
|
||||
#: the attempt must reach the server-side ``UNKNOWN_TOOL`` refusal (the
|
||||
#: tool is not offered) so the wire can be pinned byte-identical to the
|
||||
#: pre-phase baseline. The trigger phrase appears in no other suite's
|
||||
#: question (verified by the suite's own grep) and no other trigger is a
|
||||
#: substring of it.
|
||||
MIDTURN_TRIGGER = "walk through the stored text as you go"
|
||||
#: The module KB's source + path (the locator the panel captions).
|
||||
MIDTURN_SOURCE = "midturnlab"
|
||||
MIDTURN_PATH = "runbooks/valve-panel.md"
|
||||
#: The ``show_text`` ``path`` argument: the COMBINED ``source/path`` identity
|
||||
#: (``app/rag/agent.py`` resolves it through ``_resolve_path`` — a bare
|
||||
#: ``path`` names a document that does not exist, the phase-72 refusal). The
|
||||
#: call identity stays DERIVED, never a label (phase 191 LOCKED D2).
|
||||
MIDTURN_COMBINED = f"{MIDTURN_SOURCE}/{MIDTURN_PATH}"
|
||||
#: The range the single-call arms request.
|
||||
MIDTURN_START, MIDTURN_END = 12, 20
|
||||
#: The second call's range in the two-range arm.
|
||||
MIDTURN_SECOND_START, MIDTURN_SECOND_END = 1, 10
|
||||
#: Arm markers (question text only — never rendered by the app).
|
||||
MIDTURN_FENCE_BEFORE_MARKER = "fence it before the call"
|
||||
MIDTURN_FENCE_AFTER_MARKER = "fence it in the answer"
|
||||
MIDTURN_TWO_RANGES_MARKER = "then the next block"
|
||||
#: The round-1 content delta — on screen BEFORE the panel, so the suite can
|
||||
#: assert the panel lands BELOW the text the model had already written.
|
||||
MIDTURN_INTRO = "Let me pull the stored block up here — MIDTURN_INTRO_4412"
|
||||
#: The final answer's witness token: it exists ONLY in the deltas that
|
||||
#: stream after the frame, so "no sentinel in the DOM" == "the turn is live".
|
||||
MIDTURN_SENTINEL = "MIDTURN_SENTINEL_7731"
|
||||
#: Bytes the model re-types into a fence body — they must NEVER paint.
|
||||
MIDTURN_FENCE_BODY_MARKER = "MIDTURN-FENCE-BODY-NEVER-ON-SCREEN"
|
||||
#: The silence between the ``verbatim_block`` frame and the answer's first
|
||||
#: delta (seconds). 1.5 s is comfortably longer than any Playwright query
|
||||
#: round trip and comfortably shorter than every turn guard.
|
||||
MIDTURN_GAP_S = 1.5
|
||||
#: The unfenced / fence-first answer (byte-exact: the suite compares the
|
||||
#: concatenated ``delta`` payloads against it on BOTH apps).
|
||||
MIDTURN_PLAIN_ANSWER = (
|
||||
"The panel above is the file itself, not my retyping of it.\n\n"
|
||||
f"{MIDTURN_SENTINEL} — the answer kept streaming after the panel landed.\n\n"
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
#: The fence-after-call answer (the dedupe arm).
|
||||
MIDTURN_FENCED_ANSWER = (
|
||||
"The block I pulled is quoted below.\n\n"
|
||||
f"```bor {MIDTURN_COMBINED}#{MIDTURN_START}-{MIDTURN_END}\n"
|
||||
f"{MIDTURN_FENCE_BODY_MARKER} — the model re-typed these bytes\n"
|
||||
"```\n\n"
|
||||
f"{MIDTURN_SENTINEL} — the answer kept streaming after the panel landed.\n\n"
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
|
||||
|
||||
def _midturn_fence() -> str:
|
||||
"""One CLOSED ```bor fence for the arm's primary range (the model's
|
||||
own bytes inside — the placeholder drops them), padded by blank lines:
|
||||
the client concatenates a mid-round ``delta`` and the next round's answer
|
||||
with NO separator, so a fence that ends without its own trailing newline
|
||||
has its closing ``` glued to the following word and stops being a fence
|
||||
(a REAL model writes the blank line — the mock has to be as well-formed
|
||||
as the thing it stands in for)."""
|
||||
return (
|
||||
f"```bor {MIDTURN_COMBINED}#{MIDTURN_START}-{MIDTURN_END}\n"
|
||||
f"{MIDTURN_FENCE_BODY_MARKER} — the model re-typed these bytes\n"
|
||||
"```\n\n"
|
||||
)
|
||||
|
||||
|
||||
def _midturn_ranges(text: str) -> list[tuple[int, int]]:
|
||||
"""Every ``N-M`` line range the question names, in order (the two-range
|
||||
arm carries two). Empty → the primary range."""
|
||||
found = [
|
||||
(int(start), int(end))
|
||||
for start, end in re.findall(r"\b(\d+)\s*-\s*(\d+)\b", text)
|
||||
]
|
||||
return found or [(MIDTURN_START, MIDTURN_END)]
|
||||
|
||||
|
||||
def _midturn_flow(body: dict) -> tuple[str, ...] | None:
|
||||
"""The deterministic phase-192 mid-turn step (see MIDTURN_TRIGGER).
|
||||
|
||||
``("call", path, start, end, mid_round_delta)`` — the round's content
|
||||
delta (possibly empty) followed by the scripted ``show_text`` call (the
|
||||
line numbers are STRINGS, the ``_show_text_request`` convention);
|
||||
``("answer", text)`` — the final answer, streamed after
|
||||
``MIDTURN_GAP_S`` of silence.
|
||||
"""
|
||||
text = _user(body)
|
||||
lower = text.lower()
|
||||
if MIDTURN_TRIGGER not in lower:
|
||||
return None
|
||||
ranges = _midturn_ranges(text)
|
||||
results = _tool_results(body)
|
||||
if not results:
|
||||
delta = MIDTURN_INTRO
|
||||
if MIDTURN_FENCE_BEFORE_MARKER in lower:
|
||||
delta = f"{MIDTURN_INTRO}\n\n{_midturn_fence()}"
|
||||
start, end = ranges[0]
|
||||
return ("call", MIDTURN_COMBINED, str(start), str(end), delta)
|
||||
if (
|
||||
MIDTURN_TWO_RANGES_MARKER in lower
|
||||
and len(results) == 1
|
||||
and len(ranges) > 1
|
||||
):
|
||||
start, end = ranges[1]
|
||||
return ("call", MIDTURN_COMBINED, str(start), str(end), "")
|
||||
return (
|
||||
"answer",
|
||||
MIDTURN_FENCED_ANSWER
|
||||
if MIDTURN_FENCE_AFTER_MARKER in lower
|
||||
else MIDTURN_PLAIN_ANSWER,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 67 (LLM retry, TODO.md L3): deterministic failure injection
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -5710,6 +5877,34 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 192 (task 04): the MID-TURN show_text turn — the round-1
|
||||
# content delta (+ optionally the fence itself) then the call, the
|
||||
# final answer only after MIDTURN_GAP_S of silence, so the panel
|
||||
# the verbatim_block frame paints is on screen BEFORE any answer
|
||||
# token. Checked BEFORE the phase-182 show_text flow (disjoint
|
||||
# trigger phrases — the ordering convention); that flow's own arms
|
||||
# stay byte-identical.
|
||||
midturn = _midturn_flow(body)
|
||||
if midturn is not None:
|
||||
if midturn[0] == "call":
|
||||
stream = _midround_tool_stream(
|
||||
"",
|
||||
midturn[4],
|
||||
"show_text",
|
||||
_show_text_call_arguments(midturn[1], midturn[2], midturn[3]),
|
||||
"call_midturn_show",
|
||||
)
|
||||
else: # "answer" — the final answer, after the observation gap
|
||||
stream = _sse_stream(
|
||||
_apply_max_tokens(midturn[1], body.get("max_tokens")),
|
||||
0.0,
|
||||
pre_content_delay=MIDTURN_GAP_S,
|
||||
)
|
||||
return StreamingResponse(
|
||||
stream,
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
turn_progress = _turn_progress_flow(body)
|
||||
if turn_progress is not None:
|
||||
stream = _turn_progress_stream(turn_progress)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -28,8 +28,9 @@ Postgres so the fixtures' exact text is the assertion baseline):
|
||||
badge (locked A5). Wire pins: exactly FIVE tools offered (the three base
|
||||
+ ``show_image`` + ``show_text``), exactly one executed call, and — from
|
||||
an in-page fetch tap over the real ``/api/chat`` stream — the turn's SSE
|
||||
``type`` set stays inside the pre-phase seven with ``answer_text``
|
||||
riding INSIDE ``done`` (no eighth event type).
|
||||
``type`` set stays inside the pre-phase seven PLUS the one additive type
|
||||
phase 192 introduced (``verbatim_block``, the mid-turn copy of a shown
|
||||
block), with ``answer_text`` still riding INSIDE ``done``.
|
||||
- *append path* — the same turn with ``describe it without a block`` (no
|
||||
fence in the answer): the shown block renders as a panel APPENDED to the
|
||||
bubble body, below the answer text and ABOVE the source clusters (the
|
||||
@@ -246,7 +247,7 @@ DOC_A_LINES = DOC_A_CONTENT.split("\n")
|
||||
DOC_B_LINES = DOC_B_CONTENT.split("\n")
|
||||
|
||||
#: The SSE ``type`` values the pre-phase stream already carried
|
||||
#: (``app/schemas.py``'s chat events): the phase added NO eighth type.
|
||||
#: (``app/schemas.py``'s chat events): phase 182 added NO eighth type.
|
||||
PRE_PHASE_EVENT_TYPES = {
|
||||
"delta",
|
||||
"thinking",
|
||||
@@ -257,6 +258,15 @@ PRE_PHASE_EVENT_TYPES = {
|
||||
"retry",
|
||||
}
|
||||
|
||||
#: The ONE type phase 192 (task 01, A1) adds: the mid-turn
|
||||
#: ``verbatim_block`` frame, sent by the turn whose ``show_text`` call just
|
||||
#: succeeded (one frame per shown block, before the next model round). App A
|
||||
#: calls ``show_text``, so its vocabulary is the seven PLUS this type and
|
||||
#: NOTHING else; a turn with no successful show never carries it, and the
|
||||
#: kill-switch stream's byte-identity is pinned on the wire in
|
||||
#: ``tests/integration/test_chat_api.py::test_kill_switch_stream_is_byte_identical``.
|
||||
PHASE_192_EVENT_TYPES = {"verbatim_block"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module apps: verbatim ON (App A) and the kill switch OFF (App B).
|
||||
@@ -847,13 +857,23 @@ def test_show_text_turn_paints_the_files_bytes(
|
||||
"Text shown to the user" in r for r in _tool_results(captured[1])
|
||||
), "the show_text confirming result never reached the model"
|
||||
|
||||
# --- the frame vocabulary (no eighth SSE type) ----------------------
|
||||
# --- the frame vocabulary (the seven + phase 192's ONE type) --------
|
||||
frames = _sse_frames(page)
|
||||
types: set[str] = {str(f.get("type")) for f in frames}
|
||||
assert types <= PRE_PHASE_EVENT_TYPES, (
|
||||
order = [str(f.get("type")) for f in frames]
|
||||
types: set[str] = set(order)
|
||||
allowed = PRE_PHASE_EVENT_TYPES | PHASE_192_EVENT_TYPES
|
||||
assert types <= allowed, (
|
||||
f"the stream invented an event type: {sorted(types)}"
|
||||
)
|
||||
assert len(types) <= 7
|
||||
assert len(types) <= 8
|
||||
# Phase 192 (task 01): the shown block also rides its own mid-turn frame,
|
||||
# ahead of the answer's deltas (the panel is on screen while the model is
|
||||
# still talking) — ADDITIVE, the done frame below still carries it.
|
||||
block_frames = [f for f in frames if f.get("type") == "verbatim_block"]
|
||||
assert [b["locator"] for b in block_frames] == [seeded["a_locator"]]
|
||||
i_block = order.index("verbatim_block")
|
||||
assert "delta" not in order[:i_block], order
|
||||
assert order.index("tool") < i_block, order
|
||||
assert "answer_text" not in types, "answer_text leaked out as its own event"
|
||||
done = [f for f in frames if f.get("type") == "done"]
|
||||
assert len(done) == 1
|
||||
|
||||
@@ -47,14 +47,14 @@ from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Document, FolderSummary, GitSource
|
||||
from app.rag import agent
|
||||
from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL, AgentHolder, run_agent
|
||||
from app.rag.llm import (
|
||||
LLMClient,
|
||||
RetryPiece,
|
||||
StreamPiece,
|
||||
ToolCallPiece,
|
||||
ToolResultPiece,
|
||||
from app.rag.agent import (
|
||||
AGENT_TOOLS,
|
||||
SHOW_TEXT_TOOL,
|
||||
AgentHolder,
|
||||
AgentPiece,
|
||||
run_agent,
|
||||
)
|
||||
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.rag.scaffolding import ScaffoldingFilter
|
||||
@@ -286,8 +286,8 @@ async def _consume(
|
||||
llm: LLMClient,
|
||||
db_factory: Callable[[], Session],
|
||||
holder: AgentHolder,
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
) -> list[AgentPiece]:
|
||||
out: list[AgentPiece] = []
|
||||
async for piece in run_agent(
|
||||
llm,
|
||||
db_factory, # SEC-14-04: session factory (short-lived sessions)
|
||||
|
||||
@@ -28,8 +28,8 @@ from sqlalchemy.orm import Session
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Document, GitSource
|
||||
from app.rag.agent import AgentHolder, _budget_note, run_agent
|
||||
from app.rag.llm import LLMClient, RetryPiece, StreamPiece, ToolCallPiece, ToolResultPiece
|
||||
from app.rag.agent import AgentHolder, AgentPiece, _budget_note, run_agent
|
||||
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.rag.scaffolding import ScaffoldingFilter
|
||||
@@ -128,9 +128,9 @@ def _run_call(
|
||||
|
||||
async def _consume(
|
||||
llm: LLMClient, db: Session, holder: AgentHolder
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> list[AgentPiece]:
|
||||
"""SEC-14-04: uses a short-lived session per tool call."""
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
out: list[AgentPiece] = []
|
||||
async for piece in run_agent(
|
||||
llm,
|
||||
lambda: SessionLocal(), # SEC-14-04: session factory (short-lived sessions)
|
||||
|
||||
@@ -41,7 +41,7 @@ from app.rag.importer import import_sources
|
||||
from app.rag.llm import EmbeddingError, LLMError, StreamPiece, ToolCallPiece, ToolResultPiece
|
||||
from app.rag.prompts import SHOW_TEXT_CLAUSE, build_high_prompt
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
from app.schemas import ChatDoneEvent, SourceRef
|
||||
from app.schemas import ChatDoneEvent, SourceRef, VerbatimBlockRef
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
#: The fixture documents' fixed creation date (phase 106, D5): the
|
||||
@@ -3276,6 +3276,9 @@ def test_grounded_turn_carries_show_text_clause_and_tool_per_the_flag(
|
||||
|
||||
#: A fixture doc with a stable, countable body (16 lines as imported).
|
||||
_K8S = "docs/homelab/kubernetes.md"
|
||||
#: The second countable fixture doc (phase 192: the two-call turn needs two
|
||||
#: different canonical locators — the ``shown_texts`` dedupe is per locator).
|
||||
_BACKUPS = "docs/homelab/backups.md"
|
||||
|
||||
|
||||
def _verbatim_settings(**overrides: Any) -> Settings:
|
||||
@@ -3299,6 +3302,20 @@ def _doc_lines(db, combined: str) -> list[str]:
|
||||
return content.splitlines()
|
||||
|
||||
|
||||
def _doc_title(db, combined: str) -> str:
|
||||
"""The indexed row's ``title`` — the same authority the resolver puts on
|
||||
the block, never a string retyped here (the phase-182 ``_doc_lines``
|
||||
convention)."""
|
||||
source, _, path = combined.partition("/")
|
||||
title = db.scalar(
|
||||
select(Document.title).where(
|
||||
Document.source == source, Document.path == path
|
||||
)
|
||||
)
|
||||
assert title, combined
|
||||
return str(title)
|
||||
|
||||
|
||||
def test_show_text_turn_done_frame_carries_the_server_block(
|
||||
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
@@ -3608,3 +3625,239 @@ def test_resolution_failure_degrades_to_no_blocks(
|
||||
assert any("verbatim block resolution failed" in m for m in lines)
|
||||
turn = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert turn and turn[-1].endswith("verbatim_blocks=0")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 192 (task 01, A1) — the additive mid-turn ``verbatim_block`` frame.
|
||||
#
|
||||
# A successful ``show_text`` used to reach the client ONLY on the ``done``
|
||||
# frame's ``answer_text`` list (phase 182 task 05), so every panel popped in
|
||||
# after the model had finished talking about the bytes. The agent loop now
|
||||
# also yields the block the moment the call succeeds, and the endpoint turns
|
||||
# it into one additive SSE frame — after the round's ``tool`` /
|
||||
# ``tool_result`` frames, before the next model round. The ``done`` frame,
|
||||
# the saved record and the ``verbatim_blocks=N`` log field keep the phase-182
|
||||
# contract byte for byte; a ``BOR_ANSWER_TEXT=0`` turn streams NOTHING new.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_verbatim_block_frame_lands_midturn_before_done(
|
||||
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""A scripted grounded turn whose model calls
|
||||
``show_text(docs/homelab/kubernetes.md, offset=3, limit=3)`` → the stream
|
||||
carries exactly ONE ``type="verbatim_block"`` frame with the EXACT
|
||||
:class:`VerbatimBlockRef` field set (plus ``type``), positioned after the
|
||||
round's ``tool`` frame and before the answer's ``delta`` frames and the
|
||||
``done`` frame. Its bytes are the DB's, its extent is the server's, and
|
||||
the ``done`` frame still carries the SAME block in ``answer_text`` (the
|
||||
restore/shared-page contract is untouched) — counted ONCE by the per-turn
|
||||
``verbatim_blocks=N`` line, which never counts mid-turn frames."""
|
||||
scripted = FakeRagLLM(
|
||||
tool_script=[
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="call_1",
|
||||
name="show_text",
|
||||
arguments={"path": _K8S, "offset": 3, "limit": 3},
|
||||
)
|
||||
]
|
||||
]
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
with caplog.at_level("INFO", logger="app.chat"):
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
types = [f["type"] for f in frames]
|
||||
assert types.count("verbatim_block") == 1
|
||||
i_tool, i_block, i_done = types.index("tool"), types.index("verbatim_block"), len(frames) - 1
|
||||
assert types[i_done] == "done"
|
||||
# The frame sits inside its own round: after the ``tool`` frame, before
|
||||
# the next round's answer deltas and long before ``done``.
|
||||
assert i_tool < i_block < types.index("delta") < i_done
|
||||
|
||||
lines = _doc_lines(db, _K8S)
|
||||
frame = frames[i_block]
|
||||
assert frame == {
|
||||
"type": "verbatim_block",
|
||||
"source": "docs",
|
||||
"path": "homelab/kubernetes.md",
|
||||
"title": "Kubernetes Homelab Cluster",
|
||||
"start_line": 3,
|
||||
"end_line": 5,
|
||||
"total_lines": len(lines),
|
||||
"text": "\n".join(lines[2:5]),
|
||||
"truncated": False,
|
||||
"locator": f"{_K8S}#3-5",
|
||||
}
|
||||
# The key set is the done-frame item's set plus ``type`` — the client's
|
||||
# ``isVerbatimBlock`` guard accepts the event as-is.
|
||||
assert set(frame) == set(VerbatimBlockRef.model_fields) | {"type"}
|
||||
|
||||
# The done frame is UNCHANGED from phase 182: the same block, in the same
|
||||
# shape, at the same place (the mid-turn frame is additive, never a
|
||||
# replacement — the client's idempotence keeps it from painting twice).
|
||||
done = frames[i_done]
|
||||
assert done["answer_text"] == [{k: v for k, v in frame.items() if k != "type"}]
|
||||
turn = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert turn and turn[-1].endswith("verbatim_blocks=1")
|
||||
|
||||
|
||||
def test_kill_switch_stream_is_byte_identical(
|
||||
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""The wire pin for the kill switch: with ``BOR_ANSWER_TEXT=0`` the tool
|
||||
is never offered, so a scripted ``show_text`` attempt lands on the
|
||||
phase-182 refusal path and the captured frame sequence is EXACTLY the
|
||||
pre-phase baseline — one ``tool`` frame, the answer's ``delta`` frames,
|
||||
the pre-phase ``done`` frame — with NO ``verbatim_block`` frame anywhere
|
||||
and no ``answer_text`` key on any frame."""
|
||||
answer = "The flag is off, so nothing is displayed here."
|
||||
monkeypatch.setattr(
|
||||
chat_api, "get_settings", lambda: _verbatim_settings(answer_text=False)
|
||||
)
|
||||
scripted = FakeRagLLM(
|
||||
answer=answer,
|
||||
tool_script=[
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="call_1",
|
||||
name="show_text",
|
||||
arguments={"path": _K8S, "offset": 3, "limit": 3},
|
||||
)
|
||||
]
|
||||
],
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
# The documented pre-phase baseline, spelled out: the refused call's
|
||||
# ``tool`` frame (the exact ChatToolEvent bytes — no range keys, they are
|
||||
# ``read``-only), then one ``delta`` frame per 12-char slice of the
|
||||
# canned answer, then ``done``. Nothing else, in that order.
|
||||
expected = [{"type": "tool", "name": "show_text", "argument": _K8S}] + [
|
||||
{"type": "delta", "text": answer[i : i + 12]} for i in range(0, len(answer), 12)
|
||||
]
|
||||
assert frames[:-1] == expected
|
||||
assert frames[-1]["type"] == "done"
|
||||
assert set(frames[-1]) == {"type", "deflected", "sources", "related", "suggestions"}
|
||||
assert "verbatim_block" not in json.dumps(frames)
|
||||
assert "answer_text" not in json.dumps(frames)
|
||||
# The refusal the model saw is the phase-182 one (the tool does not exist
|
||||
# while the flag is off), and no block was recorded anywhere.
|
||||
tool_message = scripted.seen_messages[1][3]
|
||||
assert tool_message["role"] == "tool"
|
||||
assert tool_message["content"].startswith("Unknown tool.")
|
||||
|
||||
|
||||
def test_done_frame_and_log_line_unchanged_by_midturn_frames(
|
||||
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
"""Phase 192 (task 03, A4) — the frame the mid-turn pass adds is a COPY of
|
||||
a done-frame item, never a participant in the done-frame contract.
|
||||
|
||||
Two agent rounds, two successful ``show_text`` calls, no fence in the
|
||||
answer: the stream carries TWO ``verbatim_block`` frames (each in its own
|
||||
round, before ``done``) and the ``done`` frame still carries exactly the
|
||||
phase-182 resolution — both blocks in CALL order with the server's
|
||||
extents, byte-identical to what the resolver reads out of the row. That
|
||||
list is the saved record's ``answer_text`` and what the reload / shared
|
||||
page hydrates from, so the dedupe the client performs lands on data the
|
||||
server never changed. The per-turn line therefore reads
|
||||
``verbatim_blocks=2`` — the DONE-frame count — and never 4: the mid-turn
|
||||
frames are counted NOWHERE."""
|
||||
scripted = FakeRagLLM(
|
||||
tool_script=[
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="call_1",
|
||||
name="show_text",
|
||||
arguments={"path": _K8S, "offset": 3, "limit": 3},
|
||||
)
|
||||
],
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="call_2",
|
||||
name="show_text",
|
||||
arguments={"path": _BACKUPS, "offset": 1, "limit": 2},
|
||||
)
|
||||
],
|
||||
]
|
||||
)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
with caplog.at_level("INFO", logger="app.chat"):
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
types = [f["type"] for f in frames]
|
||||
done = frames[-1]
|
||||
assert types[-1] == "done"
|
||||
mid = [f for f in frames if f["type"] == "verbatim_block"]
|
||||
assert len(mid) == 2 and types.count("verbatim_block") == 2
|
||||
# Each frame rides its own round: after that round's ``tool`` frame and
|
||||
# long before ``done`` (the phase-192 emission order, twice).
|
||||
tool_positions = [i for i, t in enumerate(types) if t == "tool"]
|
||||
assert len(tool_positions) == 2
|
||||
for round_index, frame in enumerate(mid):
|
||||
assert tool_positions[round_index] < frames.index(frame) < len(frames) - 1, (
|
||||
"the frame lands inside its OWN round: after its tool line, before done"
|
||||
)
|
||||
|
||||
# The done frame is the phase-182 resolution, spelled out from the row —
|
||||
# the same two blocks, in call order, with the server's extent/locator.
|
||||
k8s_lines = _doc_lines(db, _K8S)
|
||||
backup_lines = _doc_lines(db, _BACKUPS)
|
||||
assert done["answer_text"] == [
|
||||
{
|
||||
"source": "docs",
|
||||
"path": "homelab/kubernetes.md",
|
||||
"title": _doc_title(db, _K8S),
|
||||
"start_line": 3,
|
||||
"end_line": 5,
|
||||
"total_lines": len(k8s_lines),
|
||||
"text": "\n".join(k8s_lines[2:5]),
|
||||
"truncated": False,
|
||||
"locator": f"{_K8S}#3-5",
|
||||
},
|
||||
{
|
||||
"source": "docs",
|
||||
"path": "homelab/backups.md",
|
||||
"title": _doc_title(db, _BACKUPS),
|
||||
"start_line": 1,
|
||||
"end_line": 2,
|
||||
"total_lines": len(backup_lines),
|
||||
"text": "\n".join(backup_lines[0:2]),
|
||||
"truncated": False,
|
||||
"locator": f"{_BACKUPS}#1-2",
|
||||
},
|
||||
]
|
||||
# Each mid-turn frame is EXACTLY its done-frame twin minus ``type``: the
|
||||
# client hydrates both from one shape, and the mid-turn copy resolved at
|
||||
# the call moment cannot disagree with the done-time resolution.
|
||||
assert done["answer_text"] == [
|
||||
{k: v for k, v in frame.items() if k != "type"} for frame in mid
|
||||
]
|
||||
# The done frame's key set is the phase-182 set plus the additive
|
||||
# ``answer_text`` — no new key, none renamed, none dropped.
|
||||
assert set(done) == {
|
||||
"type",
|
||||
"deflected",
|
||||
"sources",
|
||||
"related",
|
||||
"suggestions",
|
||||
"answer_text",
|
||||
}
|
||||
# The ONE log line counts the blocks SENT on ``done`` (2), never the sum
|
||||
# of the mid-turn frames and the done items (4) — AGENTS.md rule 10.
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert len(lines) == 1, lines
|
||||
assert lines[-1].endswith("verbatim_blocks=2"), lines[-1]
|
||||
assert "verbatim_blocks=4" not in json.dumps(lines)
|
||||
|
||||
@@ -71,16 +71,11 @@ from app.rag.agent import (
|
||||
REREAD_PARTIAL,
|
||||
SHOW_TEXT_TOOL,
|
||||
AgentHolder,
|
||||
AgentPiece,
|
||||
run_agent,
|
||||
)
|
||||
from app.rag.importer import import_sources
|
||||
from app.rag.llm import (
|
||||
LLMClient,
|
||||
RetryPiece,
|
||||
StreamPiece,
|
||||
ToolCallPiece,
|
||||
ToolResultPiece,
|
||||
)
|
||||
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece, ToolResultPiece
|
||||
from app.rag.pdf import extract_pdf_text
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
from tests.fakes import FakeEmbedder
|
||||
@@ -278,11 +273,11 @@ async def _consume(
|
||||
llm: HandbookAgentLLM,
|
||||
holder: AgentHolder,
|
||||
settings: Settings,
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> list[AgentPiece]:
|
||||
"""Consume one scripted ``run_agent`` turn (the owner's question,
|
||||
no seed docs). SEC-14-04: a short-lived session per tool call —
|
||||
the same pattern as production and the phase-70 integration suite."""
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
out: list[AgentPiece] = []
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
lambda: SessionLocal(),
|
||||
@@ -296,7 +291,7 @@ async def _consume(
|
||||
return out
|
||||
|
||||
|
||||
def _answer(pieces: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]) -> str:
|
||||
def _answer(pieces: list[AgentPiece]) -> str:
|
||||
"""The turn's final answer text (the content pieces)."""
|
||||
return "".join(
|
||||
piece.text for piece in pieces if isinstance(piece, StreamPiece) and piece.kind == "content"
|
||||
|
||||
@@ -79,15 +79,10 @@ from app.rag import agent
|
||||
from app.rag.agent import (
|
||||
READ_TRUNCATION_NOTICE,
|
||||
AgentHolder,
|
||||
AgentPiece,
|
||||
run_agent,
|
||||
)
|
||||
from app.rag.llm import (
|
||||
LLMClient,
|
||||
RetryPiece,
|
||||
StreamPiece,
|
||||
ToolCallPiece,
|
||||
ToolResultPiece,
|
||||
)
|
||||
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece, ToolResultPiece
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -291,11 +286,11 @@ async def _consume(
|
||||
llm: TwoRangeAgentLLM,
|
||||
holder: AgentHolder,
|
||||
settings: Settings,
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> list[AgentPiece]:
|
||||
"""Consume one scripted ``run_agent`` turn. SEC-14-04: a
|
||||
short-lived session per tool call — the same pattern as
|
||||
production and the phase-70/131 integration suites."""
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
out: list[AgentPiece] = []
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
lambda: SessionLocal(),
|
||||
@@ -309,7 +304,7 @@ async def _consume(
|
||||
return out
|
||||
|
||||
|
||||
def _answer(pieces: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]) -> str:
|
||||
def _answer(pieces: list[AgentPiece]) -> str:
|
||||
"""The turn's final answer text (the content pieces)."""
|
||||
return "".join(
|
||||
piece.text for piece in pieces if isinstance(piece, StreamPiece) and piece.kind == "content"
|
||||
|
||||
+252
-11
@@ -51,11 +51,19 @@ retries (a dead-then-recovered round restarts before its first piece
|
||||
with a ``RetryPiece``; a mid-stream drop stays terminal — locked A2;
|
||||
the forced final no-tools call retries too; ``llm_retries=0`` is one
|
||||
plain attempt; retries are invisible to the round cap; consumer abandon
|
||||
mid-retry-sleep leaks nothing).
|
||||
mid-retry-sleep leaks nothing), and the phase-192 (task 01) mid-turn
|
||||
``VerbatimBlockPiece``: exactly ONE piece per SUCCESSFUL ``show_text`` —
|
||||
after the round's ``tool`` / ``tool_result`` pieces and before the next
|
||||
model round, the piece's ``block`` being the holder's own
|
||||
``shown_texts`` entry — and ZERO pieces on every refusal path (missing
|
||||
``path``, the ``answer_text=False`` posture, no document, an ``is_image``
|
||||
row, the spent ``show_text_max_blocks`` budget, the already-shown dedupe)
|
||||
or on the ``BOR_ANSWER_TEXT=0`` turn where the tool is not even offered.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
@@ -82,7 +90,9 @@ from app.rag.agent import (
|
||||
SHOWN_TEXT_TRUNCATION_NOTICE,
|
||||
UNKNOWN_TOOL,
|
||||
AgentHolder,
|
||||
AgentPiece,
|
||||
MalformedReplyError,
|
||||
VerbatimBlockPiece,
|
||||
agent_tools,
|
||||
run_agent,
|
||||
)
|
||||
@@ -96,6 +106,7 @@ from app.rag.llm import (
|
||||
)
|
||||
from app.rag.prompts import TOOLS_SECTION, _base, build_deflect_prompt, build_high_prompt
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
from app.rag.verbatim import VerbatimBlock
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.rag.scaffolding import ScaffoldingFilter
|
||||
@@ -201,12 +212,14 @@ async def _run(
|
||||
seed_docs: list[Document] | None = None,
|
||||
history: Sequence[dict[str, Any]] = (),
|
||||
db_factory: Callable[[], Session] | None = None,
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> list[AgentPiece]:
|
||||
"""Consume one ``run_agent`` turn; *history* (phase 74) is the
|
||||
client's prior turns spliced between system and user (default
|
||||
``()`` — the pre-phase-74 two-message request). Phase 95: the loop
|
||||
may also yield a ``ToolResultPiece`` (a truncated ``read``)."""
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
may also yield a ``ToolResultPiece`` (a truncated ``read``), and
|
||||
phase 192 a ``VerbatimBlockPiece`` (a successful ``show_text``) —
|
||||
both members of the one :data:`app.rag.agent.AgentPiece` union."""
|
||||
out: list[AgentPiece] = []
|
||||
factory = db_factory or (lambda: _mock_session())
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
@@ -5143,8 +5156,8 @@ def test_round_failure_after_first_piece_is_terminal(monkeypatch: pytest.MonkeyP
|
||||
)
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
|
||||
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
async def drain() -> list[AgentPiece]:
|
||||
out: list[AgentPiece] = []
|
||||
with pytest.raises(LLMError, match="mid-stream drop"):
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
@@ -5205,8 +5218,8 @@ def test_zero_retries_is_one_plain_attempt(monkeypatch: pytest.MonkeyPatch) -> N
|
||||
llm = FailingLLM([([], LLMError("connection refused"))])
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
|
||||
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
async def drain() -> list[AgentPiece]:
|
||||
out: list[AgentPiece] = []
|
||||
with pytest.raises(LLMError, match="connection refused"):
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
@@ -5255,9 +5268,7 @@ def test_abandon_mid_retry_sleep_leaks_nothing(monkeypatch: pytest.MonkeyPatch)
|
||||
holder=holder,
|
||||
)
|
||||
|
||||
async def consumer() -> list[
|
||||
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
|
||||
]:
|
||||
async def consumer() -> list[AgentPiece]:
|
||||
return [p async for p in gen]
|
||||
|
||||
task = asyncio.ensure_future(consumer())
|
||||
@@ -6149,3 +6160,233 @@ def test_show_text_scripted_turn_returns_the_slice_to_the_model(
|
||||
assert len(holder.shown_texts) == 1
|
||||
assert holder.shown_texts[0].locator == f"{_SHOW_IDENTITY}#3-5"
|
||||
assert holder.tool_calls == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 192 (task 01, A1) — the mid-turn ``VerbatimBlockPiece``
|
||||
#
|
||||
# The block a successful ``show_text`` appended to ``holder.shown_texts``
|
||||
# used to reach the client only on the ``done`` frame (phase 182 task 05),
|
||||
# so every verbatim panel popped in AFTER the model finished talking about
|
||||
# the bytes. The loop now also yields ONE ``VerbatimBlockPiece`` the moment
|
||||
# the block exists — after the round's ``tool`` / ``tool_result`` pieces and
|
||||
# before the next model round — and the API layer turns it into the additive
|
||||
# ``verbatim_block`` SSE frame (pinned in ``tests/integration/test_chat_api.py``
|
||||
# and on the client in task 02). Success-only: every refusal path returns
|
||||
# before the ``shown_texts.append``, so it yields NOTHING.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _patch_two_docs(
|
||||
monkeypatch: pytest.MonkeyPatch, *docs: Document
|
||||
) -> None:
|
||||
"""Resolve several fixture rows by ``(source, path)`` (the multi-document
|
||||
variant of :func:`_patch_show_doc` — a truncated ``read`` doc AND a
|
||||
showable doc in one turn)."""
|
||||
|
||||
def _find(_db: Session, source: str, path: str) -> Document | None:
|
||||
for doc in docs:
|
||||
if (source, path) == (doc.source, doc.path):
|
||||
return doc
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(agent, "find_document", _find)
|
||||
|
||||
|
||||
_DEFAULT_SHOW_DOC = object() # "the resolvable fixture row" sentinel
|
||||
|
||||
|
||||
def _show_turn(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
arguments: dict[str, Any],
|
||||
settings: Settings | None = None,
|
||||
doc: Document | None | object = _DEFAULT_SHOW_DOC,
|
||||
) -> tuple[AgentHolder, ScriptedLLM, list[AgentPiece]]:
|
||||
"""One scripted turn whose model calls ``show_text`` with *arguments*
|
||||
(the default resolves the fixture row; pass ``None`` for the
|
||||
no-document refusal, or an ``is_image`` row for the A6 refusal)."""
|
||||
_patch_show_doc(
|
||||
monkeypatch, _show_doc() if doc is _DEFAULT_SHOW_DOC else cast("Document | None", doc)
|
||||
)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="show_text", arguments=arguments)],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
return holder, llm, asyncio.run(_run(llm, holder, settings or _settings()))
|
||||
|
||||
|
||||
def test_verbatim_block_piece_is_one_frozen_block() -> None:
|
||||
"""The piece's shape (the :class:`app.rag.llm.ToolResultPiece` pattern):
|
||||
a FROZEN dataclass with exactly one field, ``block``, holding the
|
||||
server's :class:`app.rag.verbatim.VerbatimBlock` — an immutable wire
|
||||
value, never a re-derivation."""
|
||||
block = VerbatimBlock(
|
||||
source=_SHOW_SRC,
|
||||
path=_SHOW_PATH,
|
||||
title="Gufo quadlet",
|
||||
start_line=2,
|
||||
end_line=3,
|
||||
total_lines=5,
|
||||
text="L2-two\nL3-three",
|
||||
)
|
||||
piece = VerbatimBlockPiece(block=block)
|
||||
assert [f.name for f in dataclasses.fields(piece)] == ["block"]
|
||||
assert piece.block is block
|
||||
with pytest.raises(dataclasses.FrozenInstanceError):
|
||||
piece.block = None # pyright: ignore[reportAttributeAccessIssue]
|
||||
|
||||
|
||||
def test_show_text_success_yields_one_verbatim_block_piece(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The success pin: a scripted ``show_text`` yields EXACTLY ONE
|
||||
``VerbatimBlockPiece``, AFTER the round's ``ToolCallPiece`` (the ``tool``
|
||||
frame) and BEFORE the next model round's first piece (the answer's
|
||||
``StreamPiece``). Its ``block`` IS the ``holder.shown_texts`` entry — the
|
||||
same object, so the mid-turn frame and the done frame can never
|
||||
disagree."""
|
||||
_patch_show_doc(monkeypatch, _show_doc())
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "here"), StreamPiece("content", " it is")],
|
||||
)
|
||||
out = asyncio.run(_run(llm, holder, _settings()))
|
||||
|
||||
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
|
||||
assert len(pieces) == 1
|
||||
piece = pieces[0]
|
||||
# Order: the tool call, the block, THEN the next round's answer pieces.
|
||||
assert isinstance(out[0], ToolCallPiece) and out[0].name == "show_text"
|
||||
assert out[1] is piece
|
||||
assert all(isinstance(p, StreamPiece) for p in out[2:])
|
||||
# The block is the holder's own entry (identity, not a copy).
|
||||
assert piece.block is holder.shown_texts[0]
|
||||
assert piece.block.locator == f"{_SHOW_IDENTITY}#2-5"
|
||||
assert piece.block.text == "L2-two\nL3-three\nL4-four\nL5-five"
|
||||
# The done-frame bookkeeping is untouched by the piece (task 05's path).
|
||||
assert holder.tool_calls == 1
|
||||
assert len(holder.shown_texts) == 1
|
||||
assert len(llm.requests) == 2 # the tool round + the answer round
|
||||
|
||||
|
||||
def test_verbatim_block_piece_lands_after_a_truncation_piece(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The emission-order contract with BOTH derived pieces in one turn: a
|
||||
truncated ``read`` (round 1 → ``ToolResultPiece``) then a
|
||||
``show_text`` (round 2 → ``VerbatimBlockPiece``) stream
|
||||
``tool, tool_result, tool, verbatim_block`` and only THEN the answer —
|
||||
every derived piece stays inside its own round."""
|
||||
big = _doc("S", "big.md", "Big", "\n".join(f"line {i}" for i in range(1, 30)))
|
||||
_patch_two_docs(monkeypatch, big, _show_doc())
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/big.md"})],
|
||||
[ToolCallPiece(id="c2", name="show_text", arguments={"path": _SHOW_IDENTITY})],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
out = asyncio.run(_run(llm, holder, _settings(read_max_chars=20)))
|
||||
|
||||
kinds = [
|
||||
(
|
||||
"tool:" + p.name
|
||||
if isinstance(p, ToolCallPiece)
|
||||
else "trunc"
|
||||
if isinstance(p, ToolResultPiece)
|
||||
else "block"
|
||||
if isinstance(p, VerbatimBlockPiece)
|
||||
else "retry"
|
||||
if isinstance(p, RetryPiece)
|
||||
else p.kind
|
||||
)
|
||||
for p in out
|
||||
]
|
||||
assert kinds == [
|
||||
"tool:read",
|
||||
"trunc",
|
||||
"tool:show_text",
|
||||
"block",
|
||||
"content",
|
||||
]
|
||||
assert len([p for p in out if isinstance(p, VerbatimBlockPiece)]) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"case",
|
||||
["missing_args", "flag_off", "no_document", "image_doc", "block_cap"],
|
||||
)
|
||||
def test_show_text_refusals_yield_no_verbatim_block_piece(
|
||||
monkeypatch: pytest.MonkeyPatch, case: str
|
||||
) -> None:
|
||||
"""Every refusal path yields ZERO pieces — the single success gate is
|
||||
the ``shown_texts.append`` (the mid-turn frame is success-only by
|
||||
construction, and the refusal itself is byte-identical to phase 182):
|
||||
the missing-``path`` refusal, the ``answer_text=False`` UNKNOWN_TOOL
|
||||
posture, the no-document teaching, the ``is_image`` refusal (LOCKED A6)
|
||||
and the spent ``show_text_max_blocks`` budget."""
|
||||
if case == "missing_args":
|
||||
arguments: dict[str, Any] = {}
|
||||
settings, doc = None, _DEFAULT_SHOW_DOC
|
||||
elif case == "flag_off":
|
||||
arguments = {"path": _SHOW_IDENTITY}
|
||||
settings, doc = _settings(answer_text=False), _DEFAULT_SHOW_DOC
|
||||
elif case == "no_document":
|
||||
arguments = {"path": "nosuch/file.md"}
|
||||
settings, doc = None, None
|
||||
elif case == "image_doc":
|
||||
arguments = {"path": _SHOW_IDENTITY}
|
||||
settings, doc = None, _show_doc(is_image=True)
|
||||
else:
|
||||
arguments = {"path": _SHOW_IDENTITY}
|
||||
settings, doc = _settings(show_text_max_blocks=0), _DEFAULT_SHOW_DOC
|
||||
|
||||
holder, _llm, out = _show_turn(monkeypatch, arguments, settings, doc)
|
||||
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
|
||||
_assert_refusal(holder) # counts in nothing, displays nothing
|
||||
|
||||
|
||||
def test_show_text_already_shown_dedupe_yields_no_second_piece(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The canonical-locator dedupe is a REFUSAL (D7), so the second,
|
||||
identical show adds no holder entry and yields no second piece —
|
||||
exactly ONE ``VerbatimBlockPiece`` across both rounds (the client can
|
||||
therefore never be told to paint the same block twice)."""
|
||||
_patch_show_doc(monkeypatch, _show_doc())
|
||||
holder = AgentHolder()
|
||||
call = ToolCallPiece(
|
||||
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2, "limit": 2}
|
||||
)
|
||||
llm = ScriptedLLM(
|
||||
[call],
|
||||
[ToolCallPiece(id="c2", name="show_text", arguments=call.arguments)],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
out = asyncio.run(_run(llm, holder, _settings()))
|
||||
|
||||
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
|
||||
assert len(pieces) == 1
|
||||
assert pieces[0].block.locator == f"{_SHOW_IDENTITY}#2-3"
|
||||
assert len(holder.shown_texts) == 1 # the dedupe refused the second show
|
||||
|
||||
|
||||
def test_kill_switch_turn_yields_no_verbatim_block_piece(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""``BOR_ANSWER_TEXT=0`` (the phase-182 D8 posture): the tool is NOT
|
||||
offered — so no call can succeed — and a scripted call anyway yields
|
||||
zero pieces. The turn's piece sequence is byte-identical to pre-phase
|
||||
(the wire pin lives in ``tests/integration/test_chat_api.py``)."""
|
||||
holder, llm, out = _show_turn(
|
||||
monkeypatch, {"path": _SHOW_IDENTITY}, _settings(answer_text=False)
|
||||
)
|
||||
offered = [t["function"]["name"] for t in (llm.requests[0][1] or [])]
|
||||
assert "show_text" not in offered
|
||||
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
|
||||
_assert_refusal(holder)
|
||||
|
||||
@@ -19,14 +19,8 @@ from sqlalchemy.orm import Session
|
||||
from app.config import Settings
|
||||
from app.models import Document
|
||||
from app.rag import agent
|
||||
from app.rag.agent import AgentHolder, run_agent
|
||||
from app.rag.llm import (
|
||||
LLMClient,
|
||||
RetryPiece,
|
||||
StreamPiece,
|
||||
ToolCallPiece,
|
||||
ToolResultPiece,
|
||||
)
|
||||
from app.rag.agent import AgentHolder, AgentPiece, run_agent
|
||||
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
|
||||
|
||||
#: Fixture document creation date (phase 106, D5).
|
||||
_FIXTURE_CREATED_AT = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
|
||||
@@ -131,9 +125,9 @@ async def _consume(
|
||||
holder: AgentHolder,
|
||||
settings: Settings,
|
||||
seed_docs: list[Document] | None = None,
|
||||
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
|
||||
) -> list[AgentPiece]:
|
||||
"""Consume one ``run_agent`` turn."""
|
||||
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
|
||||
out: list[AgentPiece] = []
|
||||
async for piece in run_agent(
|
||||
cast("LLMClient", llm),
|
||||
db_factory,
|
||||
|
||||
@@ -1471,6 +1471,30 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (ev.type === "verbatim_block") {
|
||||
// Phase 192 (task 02, A2): the show_text panel lands at the moment
|
||||
// the call succeeds — the fence it names hydrates IN PLACE; an
|
||||
// unfenced block lands at the answer position now, not at done.
|
||||
// The done pass re-runs the same functions and can never double this
|
||||
// panel: hydrateVerbatim is idempotent (only .verbatim-pending is
|
||||
// touched), it DROPS this appended twin when the answer fences the
|
||||
// same locator later (dropPlacedTwin, phase 192 A3), and
|
||||
// appendUnlocatedVerbatim skips an already-placed locator. NO
|
||||
// client-side answer_text gate: the server's gate is the only one
|
||||
// (a BOR_ANSWER_TEXT=0 turn sends no frame, so this branch never
|
||||
// fires — a second, client-side gate would only drift).
|
||||
// `wrap` always exists here (the round's `tool` frame created the
|
||||
// live bubble) but the optional chain keeps a frame that arrives
|
||||
// without one a silent no-op, like the `tool_result` branch.
|
||||
const n = hydrateVerbatimNow(
|
||||
wrap?.querySelector(".bubble"),
|
||||
ev,
|
||||
VERBATIM_LINK_OPTS
|
||||
);
|
||||
// The append is the ZERO-match path only: a fence already on screen
|
||||
// keeps its panel IN PLACE (the model's order), and only a block no
|
||||
// placeholder names lands at the answer position.
|
||||
if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);
|
||||
} else if (ev.type === "error") {
|
||||
// Phase 182 (task 07): an `error` frame is terminal — the `done`
|
||||
// hydrate will never run, so a placeholder that streamed in stays at
|
||||
@@ -1701,14 +1725,22 @@ def test_run_turn_is_pinned_at_phase_159_state() -> None:
|
||||
Copy buttons BEFORE ``appendSources``; the ``answer_text`` meta key rides
|
||||
``rememberBrainTurn`` with ``|| undefined``; and the terminal paths — the
|
||||
``error`` frame, the 300 s guard, its own abort surface, and the Stop
|
||||
branch — settle any leftover placeholder to the unresolved state):
|
||||
branch — settle any leftover placeholder to the unresolved state)
|
||||
and phase 192 (task 02)'s sanctioned addition (the ONE new SSE branch:
|
||||
``ev.type === "verbatim_block"`` settles the pending fence the shown
|
||||
block names IN PLACE through ``hydrateVerbatimNow`` and appends the block
|
||||
through ``appendUnlocatedVerbatim`` ONLY when that pass matched nothing —
|
||||
the mid-turn landing, so a panel appears the moment the ``show_text``
|
||||
call succeeds instead of at ``done``; no client-side gate, the server's
|
||||
``BOR_ANSWER_TEXT`` gate is the only one):
|
||||
the full source must match the pin — the ``reask`` path, the turn-local
|
||||
resets, the
|
||||
SSE contract, the save points, and the sanctioned additions (the
|
||||
task-01 ``lastUserWrap`` assignment + the idle-settle mark
|
||||
pass, the task-03 aborted-settle mark pair, the phase-160
|
||||
done-settle pair, the phase-161 live-container additions, the
|
||||
D5 refinement, and the phase-182 verbatim settle/hydrate pair) included."""
|
||||
D5 refinement, the phase-182 verbatim settle/hydrate pair and the
|
||||
phase-192 mid-turn branch) included."""
|
||||
assert _fn_source(_js(), "runTurn") == PINNED_RUN_TURN
|
||||
|
||||
|
||||
|
||||
@@ -27,6 +27,17 @@ Why these particular pins (the phase's drift risks, in order):
|
||||
call the SAME globals, in the SAME order (hydrate → append the unlocated →
|
||||
settle the code blocks, all inside the ``answer_image`` band and BEFORE the
|
||||
source clusters), so one answer cannot render three different ways.
|
||||
* **The mid-turn pass settles ONLY what it matches** (phase 192, task 02):
|
||||
the additive ``verbatim_block`` frame carries ONE block, and
|
||||
``hydrateVerbatimNow`` may replace only the pending placeholders that block
|
||||
matches — every other placeholder STAYS ``.verbatim-pending`` for the
|
||||
``done`` / failure path, because burning a fence for a block that has not
|
||||
arrived yet is how a resolved panel turns into "I could not verify this".
|
||||
* **One canonical locator, one panel** (phase 192, A3): a mid-turn block that
|
||||
no fence named yet is appended at the answer position, and when the answer
|
||||
fences it later the hydration drops that appended twin (``dropPlacedTwin``,
|
||||
bounded by the ``.msg`` wrap) — the scan is PER LOCATOR, so two honest
|
||||
ranges of one file stay two panels.
|
||||
* **No panel is ever left lying.** Every terminal path — the ``error`` frame,
|
||||
the 300 s guard, the Stop action, the failed-turn funnel — settles the
|
||||
leftover placeholders; nothing reads "Retrieving the exact text…" after the
|
||||
@@ -43,6 +54,7 @@ import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -80,6 +92,7 @@ COPY_ARIA = "Copy ${shown} lines from ${identity}"
|
||||
PUBLIC_GLOBALS = (
|
||||
"buildVerbatimPanel",
|
||||
"hydrateVerbatim",
|
||||
"hydrateVerbatimNow", # phase 192 (task 02): the MID-TURN pass
|
||||
"appendUnlocatedVerbatim",
|
||||
"settleVerbatimPlaceholders",
|
||||
"enhanceAnswerCodeBlocks",
|
||||
@@ -112,6 +125,28 @@ def _body(js: str, name: str) -> str:
|
||||
return _fn(js, name)
|
||||
|
||||
|
||||
def _doc(js: str, name: str) -> str:
|
||||
"""The ``/* … */`` doc comment that sits directly above
|
||||
``function <name>`` — the house docstring-pinning convention for the
|
||||
clients: the prose IS the contract record, so a refactor that "simplifies"
|
||||
the rule away has to delete the sentence in the same change."""
|
||||
start = js.find(f"\nfunction {name}(")
|
||||
assert start != -1, f"function {name}() must exist"
|
||||
open_ = js.rfind("/*", 0, start)
|
||||
close = js.find("*/", open_)
|
||||
assert open_ != -1 and close != -1 and close < start, (
|
||||
f"{name}() must carry the doc comment above it"
|
||||
)
|
||||
return js[open_ : close]
|
||||
|
||||
|
||||
def _flow(text: str) -> str:
|
||||
"""Comment prose as one line — the pins quote the SENTENCE, not its
|
||||
wrapping (the `` * `` line leaders come out with the wrap)."""
|
||||
lines = [re.sub(r"^\s*\*+\s?", " ", line) for line in text.splitlines()]
|
||||
return " ".join(" ".join(lines).split())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# the client ships, and ships in the right shape
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -346,6 +381,134 @@ def test_hydration_is_idempotent_and_never_throws_on_garbage() -> None:
|
||||
assert "Array.isArray(blocks)" in guard, "a missing frame key is not an error"
|
||||
|
||||
|
||||
def test_midturn_hydration_settles_only_the_placeholders_one_block_matches() -> None:
|
||||
"""Phase 192 (task 02, A2) — the heart of the mid-turn pass. The additive
|
||||
``verbatim_block`` frame carries ONE block, and ``hydrateVerbatimNow``:
|
||||
|
||||
* walks the SAME pending set as the done pass
|
||||
(``.verbatim-pending[data-bor-locator]``, the attribute ``markdown.js``
|
||||
normalized — no second grammar);
|
||||
* matches through the ONE rule (``matchVerbatimBlock`` with a one-block
|
||||
list) so the mid-turn pass and the ``done`` pass can never disagree
|
||||
about which fence this block owns;
|
||||
* replaces ONLY the matched nodes (``buildVerbatimPanel``) — the unmatched
|
||||
stay ``.verbatim-pending``, NEVER the honest unresolved state, which is
|
||||
the done/failure path's job (``settleVerbatimPlaceholders``);
|
||||
* returns the number settled (``0`` is the caller's signal to append);
|
||||
* is total without a ``try`` — a missing root, a root that is not an
|
||||
element, or a frame that is not a block all return ``0``."""
|
||||
js = _text(VERBATIM_JS)
|
||||
fn = _fn(js, "hydrateVerbatimNow")
|
||||
assert '.verbatim-pending[data-bor-locator]"' in fn, (
|
||||
"the mid-turn walk is the done path's pending set"
|
||||
)
|
||||
assert 'node.getAttribute("data-bor-locator")' in fn, (
|
||||
"the key is the attribute markdown.js wrote — never re-derived"
|
||||
)
|
||||
assert "matchVerbatimBlock(" in fn, "the ONE match rule, not a mid-turn variant"
|
||||
assert "const list = [block]" in fn, "the one-block frame rides the done list shape"
|
||||
assert "replaceWith(buildVerbatimPanel(" in fn, "only a MATCH builds a panel"
|
||||
assert "buildVerbatimUnresolved" not in fn, (
|
||||
"an unmatched fence stays pending mid-turn — settling is the terminal path's job"
|
||||
)
|
||||
assert "if (!matched) return" in fn, "the unmatched node is left in place"
|
||||
assert "settled += 1" in fn and "return settled" in fn, (
|
||||
"the count is the caller's append/no-append signal"
|
||||
)
|
||||
assert 'typeof root.querySelectorAll !== "function"' in fn, "a missing root is 0"
|
||||
assert "isVerbatimBlock(block)" in fn, "a malformed frame settles NOTHING"
|
||||
assert "try {" not in fn, "the guards make it total — no exception path"
|
||||
assert "innerHTML" not in fn
|
||||
|
||||
|
||||
def test_hydration_drops_the_midturn_twin_and_never_an_unresolved_one() -> None:
|
||||
"""Phase 192 (A3): a block that arrived mid-turn before its fence was
|
||||
appended at the answer position; when its fence hydrates, that twin is
|
||||
removed so ONE locator keeps ONE panel and the panel sits where the model
|
||||
fenced it. The shape of the scan is the contract:
|
||||
|
||||
* bounded by the message wrap (the appended panel lives in ``.msg-body``,
|
||||
beside the bubble — a bubble-local walk would miss it);
|
||||
* ``.verbatim-pending`` and ``.verbatim-unresolved`` are NOT placements
|
||||
(the ``appendUnlocatedVerbatim`` exclusion, shared);
|
||||
* the key is the ONE ``verbatimPlacedKey`` derivation (the same one the
|
||||
append-side scan reads — two derivations is how the dedupe drifts);
|
||||
* it compares the canonical LOCATOR (never an identity — two ranges of one
|
||||
file are two panels) and never removes the placeholder being hydrated."""
|
||||
js = _text(VERBATIM_JS)
|
||||
twin = _fn(js, "dropPlacedTwin")
|
||||
assert 'host.querySelectorAll(".verbatim[data-bor-locator]")' in twin
|
||||
assert 'el.classList.contains("verbatim-pending")' in twin
|
||||
assert 'el.classList.contains("verbatim-unresolved")' in twin
|
||||
assert 'el.getAttribute("data-bor-locator") !== placedKey' in twin, (
|
||||
"PER LOCATOR — an identity scan would collapse two honest ranges"
|
||||
)
|
||||
assert "verbatimSplitLocator" not in twin and "verbatimIdentity(" not in twin, (
|
||||
"no identity fallback in the twin scan"
|
||||
)
|
||||
assert "if (el === keep) return" in twin, "the hydrating placeholder is never the twin"
|
||||
assert "el.remove()" in twin
|
||||
hydrate = _fn(js, "hydrateVerbatim")
|
||||
assert 'root.closest(".msg")' in hydrate, "the scan is bounded by the message wrap"
|
||||
assert "if (block) dropPlacedTwin(host, block, node);" in hydrate, (
|
||||
"only a MATCHED placeholder drops a twin"
|
||||
)
|
||||
# The ONE key derivation, read by both dedupe sides.
|
||||
assert js.count("function verbatimPlacedKey(") == 1
|
||||
assert "verbatimPlacedKey(block)" in _fn(js, "appendUnlocatedVerbatim")
|
||||
|
||||
|
||||
#: The sentences the phase-192 (A3) prose must keep carrying. They are the
|
||||
#: only place a reader of ``verbatim.js`` learns the dedupe's two rules, and
|
||||
#: the fastest way to lose a rule is a refactor that drops its documentation
|
||||
#: first and its code second.
|
||||
TWIN_DOC_SENTENCES = (
|
||||
# PER LOCATOR, never per identity — the two-ranges-stay clause.
|
||||
"PER LOCATOR, never per identity: two different ranges of one file are"
|
||||
" two honest panels",
|
||||
# The in-text panel wins over the mid-turn appended twin.
|
||||
"The IN-TEXT panel wins",
|
||||
)
|
||||
|
||||
|
||||
def test_the_twin_dedupe_docstring_keeps_the_per_locator_clause() -> None:
|
||||
"""Phase 192 (task 03, A3): the prose is part of the contract. The
|
||||
``dropPlacedTwin`` docstring names the per-LOCATOR rule (and the
|
||||
two-honest-ranges clause that follows from it) and why the in-text panel
|
||||
wins; ``hydrateVerbatim``'s docstring says the matched placeholder drops
|
||||
that twin ("one locator, one panel, and it sits where the model fenced
|
||||
it"); and the module header states the same rule for whoever reads the
|
||||
file top-down."""
|
||||
js = _text(VERBATIM_JS)
|
||||
twin = _flow(_doc(js, "dropPlacedTwin"))
|
||||
for sentence in TWIN_DOC_SENTENCES:
|
||||
assert sentence in twin, f"the docstring lost: {sentence}"
|
||||
assert "appendUnlocatedVerbatim" in twin, (
|
||||
"the placement exclusion is documented as SHARED with the append side"
|
||||
)
|
||||
hydrate = _flow(_doc(js, "hydrateVerbatim"))
|
||||
assert "dropPlacedTwin" in hydrate
|
||||
assert "one locator, one panel, and it sits where the model fenced it" in hydrate
|
||||
header = _flow(js[: js.find("*/")])
|
||||
assert "ONE CANONICAL LOCATOR, ONE PANEL (phase 192)" in header
|
||||
assert "two honest ranges of one file stay two panels" in header
|
||||
|
||||
|
||||
def test_the_twin_scan_bounds_to_the_msg_wrap_or_the_root() -> None:
|
||||
"""The scan's host is the message wrap (the appended twin lives in
|
||||
``.msg-body``, beside the bubble), and when a root has no ``.msg``
|
||||
ancestor the ROOT bounds it — a missing wrap narrows the scan, it never
|
||||
switches the dedupe off."""
|
||||
hydrate = _fn(_text(VERBATIM_JS), "hydrateVerbatim")
|
||||
assert 'root.closest(".msg")' in hydrate
|
||||
assert 'root.closest(".msg") : null) || root' in hydrate, (
|
||||
"no .msg ancestor → the root bounds the scan"
|
||||
)
|
||||
assert "document.querySelectorAll" not in hydrate, (
|
||||
"never a page-wide scan — one message's twin, one message's panels"
|
||||
)
|
||||
|
||||
|
||||
def test_unlocated_blocks_land_in_the_answer_image_band() -> None:
|
||||
"""The blocks no placeholder matched (a ``show_text`` call the prose never
|
||||
named) append to ``.msg-body`` — the same band as the answer image, below
|
||||
@@ -458,6 +621,50 @@ def test_the_done_handler_hydrates_before_the_source_clusters() -> None:
|
||||
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatim(") :][:200]
|
||||
|
||||
|
||||
def _midturn_branch(js: str) -> str:
|
||||
start = js.index('ev.type === "verbatim_block"')
|
||||
end = js.index('ev.type === "error"', start)
|
||||
return js[start:end]
|
||||
|
||||
|
||||
def test_the_midturn_frame_handler_hydrates_in_place_and_appends_only_on_zero() -> None:
|
||||
"""Phase 192 (task 02, A2): the live turn's ``verbatim_block`` branch.
|
||||
|
||||
The shape IS the behaviour: hydrate the pending fence FIRST, and reach
|
||||
for ``appendUnlocatedVerbatim`` ONLY on the zero-match path — a fence that
|
||||
is already on screen keeps its panel where the model put it, and only a
|
||||
block no placeholder names lands at the answer position (the
|
||||
``answer_image`` band). The branch sits in the same ``readSSE`` chain, after
|
||||
``tool_result`` (the server's emission order) and before ``error``.
|
||||
|
||||
And there is NO client-side kill switch: the server's
|
||||
``BOR_ANSWER_TEXT`` gate is the only gate, so a disabled turn simply sends
|
||||
no frame (a second, client-side settings check is a drift risk, exactly
|
||||
the phase's reasoning for keeping the gate server-side)."""
|
||||
js = _text(APP_JS)
|
||||
branch = _midturn_branch(js)
|
||||
assert "hydrateVerbatimNow(" in branch, "the mid-turn pass settles the fence"
|
||||
assert "appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS)" in branch, (
|
||||
"the unmatched block lands at the answer position, ONE opts object"
|
||||
)
|
||||
assert re.search(r"if \(\w+ === 0\)\s*appendUnlocatedVerbatim\(", branch), (
|
||||
"the append is the ZERO-match path only — never alongside an in-place panel"
|
||||
)
|
||||
order = [branch.index("hydrateVerbatimNow("), branch.index("appendUnlocatedVerbatim(")]
|
||||
assert order == sorted(order), "hydrate in place FIRST, append only if nothing matched"
|
||||
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatimNow(") :][:200], (
|
||||
"the mid-turn panel navigates exactly like the done panel (chip parity)"
|
||||
)
|
||||
# No second gate: nothing in the branch reads settings / a flag / the
|
||||
# done frame's key — the arrival of the frame is the whole condition.
|
||||
code = _strip_comments(branch)
|
||||
for gate in ("answer_text", "answerText", "settings", "await ", "fetch("):
|
||||
assert gate not in code, f"{gate} looks like a client-side gate (server-only)"
|
||||
# The frame type is the ONE new thing the live turn knows (the server
|
||||
# sends it between `tool_result` and the next round).
|
||||
assert js.index('ev.type === "tool_result"') < js.index('ev.type === "verbatim_block"')
|
||||
|
||||
|
||||
def test_the_answer_renderers_opt_in_and_nothing_else_does() -> None:
|
||||
"""The per-delta write and the ``done``-time re-render both pass
|
||||
``{ verbatim: true }`` (the third site is the restore renderer, pinned in
|
||||
@@ -865,3 +1072,642 @@ def test_malformed_blocks_are_dropped_and_never_throw(tmp_dir: Path) -> None:
|
||||
assert out["identity"] == {"ok": True, "value": "homelab/runbooks/a.md"}
|
||||
assert out["unresolved_copy"] == {"ok": True, "value": UNRESOLVED_COPY}
|
||||
assert out["badge_title"] == {"ok": True, "value": BADGE_TITLE}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# node: the MID-TURN pass (phase 192, task 02), run against the real file on
|
||||
# a minimal DOM — the marker pins above prove the SHAPE, these prove the
|
||||
# behaviour the E2E (task 04) will show in a browser: matched fences settle in
|
||||
# place, everything else stays pending, a bad frame settles nothing.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The minimal DOM + the fixtures (``pendingBubble`` / ``shownBlock`` /
|
||||
#: ``snapshot``). ``querySelectorAll`` understands the ONE compound selector
|
||||
#: the client walks (``.a.b[attr]``) and throws on anything else, so a
|
||||
#: selector drift in the client fails loudly instead of matching nothing.
|
||||
_DOM_LIB = """
|
||||
class El {
|
||||
constructor(tag) {
|
||||
this.tag = tag;
|
||||
this.children = [];
|
||||
this.parent = null;
|
||||
this.attrs = {};
|
||||
this.dataset = {};
|
||||
this.classes = new Set();
|
||||
this._own = "";
|
||||
}
|
||||
get className() { return [...this.classes].join(" "); }
|
||||
set className(v) { this.classes = new Set(String(v).split(/\\s+/).filter(Boolean)); }
|
||||
get classList() {
|
||||
return { contains: (c) => this.classes.has(c), add: (c) => this.classes.add(c) };
|
||||
}
|
||||
setAttribute(k, v) { this.attrs[k] = String(v); }
|
||||
addEventListener(ev, fn) { (this.listeners = this.listeners || {})[ev] = fn; }
|
||||
getAttribute(k) {
|
||||
if (Object.prototype.hasOwnProperty.call(this.attrs, k)) return this.attrs[k];
|
||||
if (k.startsWith("data-")) {
|
||||
const camel = k.slice(5).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
|
||||
if (Object.prototype.hasOwnProperty.call(this.dataset, camel)) {
|
||||
return String(this.dataset[camel]);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
appendChild(c) { c.parent = this; this.children.push(c); return c; }
|
||||
append(...cs) { for (const c of cs) this.appendChild(c); }
|
||||
insertBefore(box, ref) {
|
||||
const i = this.children.indexOf(ref);
|
||||
this.children.splice(i < 0 ? this.children.length : i, 0, box);
|
||||
box.parent = this;
|
||||
return box;
|
||||
}
|
||||
replaceWith(node) {
|
||||
if (!this.parent) throw new Error("replaceWith on a detached node");
|
||||
const i = this.parent.children.indexOf(this);
|
||||
this.parent.children[i] = node;
|
||||
node.parent = this.parent;
|
||||
this.parent = null;
|
||||
}
|
||||
remove() {
|
||||
if (!this.parent) return;
|
||||
const i = this.parent.children.indexOf(this);
|
||||
if (i >= 0) this.parent.children.splice(i, 1);
|
||||
this.parent = null;
|
||||
}
|
||||
closest(sel) {
|
||||
let cur = this;
|
||||
while (cur) {
|
||||
if (matches(cur, sel)) return cur;
|
||||
cur = cur.parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
querySelectorAll(sel) { return descendants(this).filter((el) => matches(el, sel)); }
|
||||
querySelector(sel) { return descendants(this).find((el) => matches(el, sel)) || null; }
|
||||
get textContent() {
|
||||
if (!this.children.length) return this._own;
|
||||
return this.children.map((c) => c.textContent).join("");
|
||||
}
|
||||
set textContent(v) { this.children = []; this._own = String(v); }
|
||||
}
|
||||
|
||||
function descendants(el, out) {
|
||||
const list = out || [];
|
||||
for (const c of el.children) { list.push(c); descendants(c, list); }
|
||||
return list;
|
||||
}
|
||||
|
||||
function matches(el, sel) {
|
||||
const parts = sel.match(/\\.[A-Za-z0-9_-]+|\\[[^[\\]]+\\]/g) || [];
|
||||
if (!parts.length) throw new Error("unsupported selector: " + sel);
|
||||
return parts.every((p) =>
|
||||
p.startsWith(".")
|
||||
? el.classes.has(p.slice(1))
|
||||
: el.getAttribute(p.slice(1, -1)) !== null
|
||||
);
|
||||
}
|
||||
|
||||
globalThis.document = {
|
||||
createElement: (t) => new El(t),
|
||||
createTextNode: (t) => {
|
||||
const node = new El("#text");
|
||||
node._own = String(t);
|
||||
return node;
|
||||
},
|
||||
querySelectorAll: () => [],
|
||||
body: new El("body"),
|
||||
};
|
||||
|
||||
function pendingBubble(locators) {
|
||||
const bubble = new El("div");
|
||||
bubble.classes.add("bubble");
|
||||
for (const loc of locators) {
|
||||
const node = new El("div");
|
||||
node.className = "verbatim verbatim-pending";
|
||||
node.dataset.borLocator = loc;
|
||||
bubble.appendChild(node);
|
||||
}
|
||||
return bubble;
|
||||
}
|
||||
|
||||
function shownBlock(over) {
|
||||
return Object.assign({
|
||||
source: "homelab",
|
||||
path: "runbooks/a.md",
|
||||
title: "Runbook A",
|
||||
start_line: 12,
|
||||
end_line: 40,
|
||||
total_lines: 210,
|
||||
text: "line twelve\\nline thirteen",
|
||||
truncated: false,
|
||||
locator: "homelab/runbooks/a.md#12-40",
|
||||
}, over || {});
|
||||
}
|
||||
|
||||
function pendingWrap(locators) {
|
||||
const msg = new El("div");
|
||||
msg.className = "msg brain";
|
||||
const body = new El("div");
|
||||
body.className = "msg-body";
|
||||
const bubble = new El("div");
|
||||
bubble.className = "bubble";
|
||||
msg.appendChild(body);
|
||||
body.appendChild(bubble);
|
||||
for (const loc of locators) {
|
||||
const node = new El("div");
|
||||
node.className = "verbatim verbatim-pending";
|
||||
node.dataset.borLocator = loc;
|
||||
bubble.appendChild(node);
|
||||
}
|
||||
return msg;
|
||||
}
|
||||
|
||||
function findClass(root, cls) {
|
||||
return descendants(root).find((el) => el.classes.has(cls)) || null;
|
||||
}
|
||||
|
||||
function inBubble(el) {
|
||||
let cur = el.parent;
|
||||
while (cur) {
|
||||
if (cur.classes.has("bubble")) return true;
|
||||
cur = cur.parent;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/* The RESOLVED panels in document order — a pending or unresolved node is
|
||||
* NOT a placement (appendUnlocatedVerbatim's own convention), which is what
|
||||
* makes this the "one locator, one panel" reading. */
|
||||
function placed(msg) {
|
||||
return descendants(msg)
|
||||
.filter((el) => el.classes.has("verbatim"))
|
||||
.filter((el) => !el.classes.has("verbatim-pending"))
|
||||
.filter((el) => !el.classes.has("verbatim-unresolved"))
|
||||
.map((el) => ({
|
||||
locator: el.getAttribute("data-bor-locator"),
|
||||
where: inBubble(el) ? "bubble" : "body",
|
||||
}));
|
||||
}
|
||||
|
||||
/* The fence the model writes in a LATER round (the mid-turn frame arrived
|
||||
* before it): a fresh pending node inside the bubble. */
|
||||
function addFence(msg, locator) {
|
||||
const node = new El("div");
|
||||
node.className = "verbatim verbatim-pending";
|
||||
node.dataset.borLocator = locator;
|
||||
msg.querySelector(".bubble").appendChild(node);
|
||||
return node;
|
||||
}
|
||||
|
||||
/* The same later-round fence inside a BARE bubble (no `.msg` wrap) — the arm
|
||||
* that proves the ROOT bounds the twin scan when there is no message wrap. */
|
||||
function bareFence(bubble, locator) {
|
||||
const node = new El("div");
|
||||
node.className = "verbatim verbatim-pending";
|
||||
node.dataset.borLocator = locator;
|
||||
bubble.appendChild(node);
|
||||
return node;
|
||||
}
|
||||
|
||||
function snapshot(root) {
|
||||
return descendants(root)
|
||||
.filter((el) => el.classes.has("verbatim"))
|
||||
.map((el) => {
|
||||
const caption = findClass(el, "verbatim-caption");
|
||||
const pre = findClass(el, "verbatim-text");
|
||||
const link = findClass(el, "verbatim-open");
|
||||
return {
|
||||
pending: el.classes.has("verbatim-pending"),
|
||||
unresolved: el.classes.has("verbatim-unresolved"),
|
||||
locator: el.getAttribute("data-bor-locator"),
|
||||
caption: caption ? caption.textContent : null,
|
||||
text: pre ? pre.textContent : null,
|
||||
href: link ? link.href : null,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
El, pendingBubble, pendingWrap, shownBlock, snapshot, placed, addFence,
|
||||
bareFence, findClass, descendants,
|
||||
};
|
||||
"""
|
||||
|
||||
_DOM_HARNESS = """
|
||||
const fs = require("fs");
|
||||
const src = fs.readFileSync(process.argv[2], "utf8");
|
||||
const dom = require(process.argv[3]); // installs globalThis.document
|
||||
const cases = JSON.parse(fs.readFileSync(process.argv[4], "utf8"));
|
||||
const panel = new Function(
|
||||
src +
|
||||
"\\nreturn { hydrateVerbatimNow, hydrateVerbatim, appendUnlocatedVerbatim," +
|
||||
" buildVerbatimPanel, isVerbatimBlock };"
|
||||
)();
|
||||
const scope = Object.assign({}, panel, dom);
|
||||
const out = {};
|
||||
for (const [name, body] of Object.entries(cases)) {
|
||||
try {
|
||||
const fn = new Function("api", "with (api) { return (" + body + "); }");
|
||||
out[name] = { ok: true, value: fn(scope) };
|
||||
} catch (err) {
|
||||
out[name] = { ok: false, error: String(err && err.message ? err.message : err) };
|
||||
}
|
||||
}
|
||||
process.stdout.write(JSON.stringify(out));
|
||||
"""
|
||||
|
||||
#: The block a mid-turn ``verbatim_block`` frame carries in these cases.
|
||||
SHOWN_BLOCK_LOCATOR = "homelab/runbooks/a.md#12-40"
|
||||
#: The OTHER fence on screen mid-answer — the one this frame must NOT touch.
|
||||
OTHER_PENDING_LOCATOR = "homelab/runbooks/b.md#1-5"
|
||||
#: The caption the panel must carry in every arm: the SERVER's extent (the
|
||||
#: phase-182 contract), never the range the model typed into its fence.
|
||||
CAPTION_SERVER_EXTENT = "homelab/runbooks/a.md \u00b7 lines 12\u201340 of 210"
|
||||
|
||||
|
||||
def _node_dom(cases: dict[str, str], tmp_dir: Path) -> dict[str, Any]:
|
||||
"""Run *cases* (JS expressions with the panel API + the DOM fixtures in
|
||||
scope) under node against the real ``verbatim.js`` on the minimal DOM.
|
||||
|
||||
``Any`` on purpose: the DOM cases return nested objects/arrays, and the
|
||||
assertions below walk them (``value["nodes"][0]["caption"]``)."""
|
||||
lib = tmp_dir / "bor_dom.cjs"
|
||||
lib.write_text(_DOM_LIB, encoding="utf-8")
|
||||
script = tmp_dir / "bor_dom_harness.cjs"
|
||||
script.write_text(_DOM_HARNESS, encoding="utf-8")
|
||||
case_file = tmp_dir / "dom_cases.json"
|
||||
case_file.write_text(json.dumps(dict(cases)), encoding="utf-8")
|
||||
proc = subprocess.run(
|
||||
["node", str(script), str(VERBATIM_JS), str(lib), str(case_file)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
check=False,
|
||||
)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
return json.loads(proc.stdout)
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_midturn_pass_settles_the_matched_fence_and_leaves_the_rest(tmp_dir: Path) -> None:
|
||||
"""The behaviour of ``hydrateVerbatimNow`` against the real file: the fence
|
||||
whose canonical locator the frame carries settles IN PLACE with the
|
||||
server's caption and bytes; the next fence in the same answer is NOT
|
||||
touched (no panel, no unresolved sentence — it stays a promise the ``done``
|
||||
frame still has to keep)."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"exact": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
|
||||
return {{ n, nodes: snapshot(bubble) }};
|
||||
}})()""",
|
||||
"wrong_range_uses_the_servers_extent": """(() => {
|
||||
const bubble = pendingBubble(["homelab/runbooks/a.md#99-120"]);
|
||||
const n = hydrateVerbatimNow(bubble, shownBlock(), {});
|
||||
return { n, nodes: snapshot(bubble) };
|
||||
})()""",
|
||||
"two_fences_one_locator": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
|
||||
return {{ n, pending: snapshot(bubble).filter((x) => x.pending).length }};
|
||||
}})()""",
|
||||
"opts_thread_through": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, shownBlock(), {{
|
||||
documentHref: (s, p) => "HOOK/" + s + "/" + p,
|
||||
}});
|
||||
return {{ n, nodes: snapshot(bubble) }};
|
||||
}})()""",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
exact = out["exact"]
|
||||
assert exact["ok"] is True, exact
|
||||
assert exact["value"]["n"] == 1, "exactly the matching placeholder settles"
|
||||
settled, untouched = exact["value"]["nodes"]
|
||||
assert settled == {
|
||||
"pending": False,
|
||||
"unresolved": False,
|
||||
"locator": SHOWN_BLOCK_LOCATOR,
|
||||
"caption": CAPTION_SERVER_EXTENT,
|
||||
"text": "line twelve\nline thirteen",
|
||||
"href": "/document.html?source=homelab&path=runbooks%2Fa.md",
|
||||
}, "the panel is the done path's panel, built from the SERVER's bytes"
|
||||
assert untouched == {
|
||||
"pending": True,
|
||||
"unresolved": False,
|
||||
"locator": OTHER_PENDING_LOCATOR,
|
||||
"caption": None,
|
||||
"text": None,
|
||||
"href": None,
|
||||
}, "a fence this block does not name STAYS pending"
|
||||
# The model's range was wrong: the identity fallback still applies, and
|
||||
# the caption / data attribute carry the SERVER's extent (mid-turn and
|
||||
# done can never disagree — the phase-182 contract).
|
||||
fallback = out["wrong_range_uses_the_servers_extent"]
|
||||
assert fallback["ok"] is True, fallback
|
||||
assert fallback["value"]["n"] == 1
|
||||
assert fallback["value"]["nodes"][0]["locator"] == SHOWN_BLOCK_LOCATOR
|
||||
assert fallback["value"]["nodes"][0]["caption"] == CAPTION_SERVER_EXTENT
|
||||
dupes = out["two_fences_one_locator"]
|
||||
assert dupes["ok"] is True, dupes
|
||||
assert dupes["value"]["n"] == 2 and dupes["value"]["pending"] == 0
|
||||
hooked = out["opts_thread_through"]
|
||||
assert hooked["ok"] is True, hooked
|
||||
assert hooked["value"]["n"] == 1
|
||||
assert hooked["value"]["nodes"][0]["href"] == "HOOK/homelab/runbooks/a.md", (
|
||||
"opts (the chat's modal hook) reach the mid-turn panel"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_midturn_pass_is_total_on_garbage_and_no_match(tmp_dir: Path) -> None:
|
||||
"""Nothing a mid-turn frame can carry settles a fence the block does not
|
||||
name \u2014 and nothing throws: no match, a malformed frame (the ``isVerbatimBlock``
|
||||
guard), a null root, a root that is not an element, no root at all. The
|
||||
pending node stays ``.verbatim-pending`` in every arm (NEVER the honest
|
||||
unresolved sentence \u2014 that is the terminal path's verdict, not this one)."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"other_file": f"""(() => {{
|
||||
const bubble = pendingBubble(["{OTHER_PENDING_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
|
||||
return {{ n, nodes: snapshot(bubble) }};
|
||||
}})()""",
|
||||
"malformed_no_source": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, {{ path: "runbooks/a.md", text: "x" }}, {{}});
|
||||
return {{ n, nodes: snapshot(bubble) }};
|
||||
}})()""",
|
||||
"malformed_not_an_object": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
|
||||
const n = hydrateVerbatimNow(bubble, "verbatim", {{}});
|
||||
return {{ n, nodes: snapshot(bubble) }};
|
||||
}})()""",
|
||||
"null_root": "hydrateVerbatimNow(null, shownBlock(), {})",
|
||||
"plain_object_root": "hydrateVerbatimNow({}, shownBlock(), {})",
|
||||
"undefined_root": "hydrateVerbatimNow(undefined, shownBlock(), undefined)",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
for name in ("other_file", "malformed_no_source", "malformed_not_an_object"):
|
||||
case = out[name]
|
||||
assert case["ok"] is True, (name, case)
|
||||
assert case["value"]["n"] == 0, name
|
||||
node = case["value"]["nodes"][0]
|
||||
assert node["pending"] is True and node["unresolved"] is False, (
|
||||
f"{name}: the fence stays a promise, never a burned placeholder"
|
||||
)
|
||||
assert node["text"] is None, f"{name}: no panel, so no bytes on screen"
|
||||
for name in ("null_root", "plain_object_root", "undefined_root"):
|
||||
assert out[name] == {"ok": True, "value": 0}, name
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_midturn_panel_survives_the_done_pass(tmp_dir: Path) -> None:
|
||||
"""The parity that keeps the phase additive: the mid-turn pass runs, then
|
||||
the ``done`` frame arrives with the SAME block. ``hydrateVerbatim`` only
|
||||
touches ``.verbatim-pending``, so the panel the mid-turn pass built is the
|
||||
SAME node afterwards (its ``Copied`` state and scroll position survive),
|
||||
and the unrelated fence settles to the honest sentence."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"then_done": f"""(() => {{
|
||||
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
|
||||
const n1 = hydrateVerbatimNow(bubble, shownBlock(), {{}});
|
||||
const midturnPanel = findClass(bubble, "verbatim");
|
||||
const n2 = hydrateVerbatim(bubble, [shownBlock()], {{}});
|
||||
const nodes = snapshot(bubble);
|
||||
return {{
|
||||
n1,
|
||||
n2,
|
||||
sameNode: findClass(bubble, "verbatim") === midturnPanel,
|
||||
nodes,
|
||||
}};
|
||||
}})()""",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
case = out["then_done"]
|
||||
assert case["ok"] is True, case
|
||||
value = case["value"]
|
||||
assert value["n1"] == 1 and value["n2"] == 1
|
||||
assert value["sameNode"] is True, (
|
||||
"the done pass re-runs over the mid-turn panel without rebuilding it"
|
||||
)
|
||||
settled, unresolved = value["nodes"]
|
||||
assert settled["pending"] is False and settled["locator"] == SHOWN_BLOCK_LOCATOR
|
||||
assert unresolved["pending"] is False and unresolved["unresolved"] is True, (
|
||||
"the fence no frame ever resolved lands on the honest state at done"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_one_locator_one_panel_across_the_midturn_and_done_passes(tmp_dir: Path) -> None:
|
||||
"""The duplicate the mid-turn landing would otherwise leave behind (the
|
||||
phase's A3): a ``show_text`` block arrives BEFORE the answer fences it, so
|
||||
the mid-turn pass appends it at the answer position; the fence lands in a
|
||||
later round and hydrates at ``done``. Exactly ONE panel remains and it sits
|
||||
IN THE BUBBLE, where the model put the fence.
|
||||
|
||||
The two arms that must NOT collapse: two different ranges of one file are
|
||||
two honest panels (the scan is per canonical locator), and a block the
|
||||
answer never fences keeps its single appended panel."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"fence_arrives_later": """(() => {
|
||||
const msg = pendingWrap([]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
const mid = hydrateVerbatimNow(bubble, shownBlock(), {});
|
||||
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
|
||||
addFence(msg, "homelab/runbooks/a.md#99-120");
|
||||
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
|
||||
return { mid, appended, settled, panels: placed(msg) };
|
||||
})()""",
|
||||
"two_ranges_one_file": """(() => {
|
||||
const b1 = shownBlock();
|
||||
const b2 = shownBlock({ start_line: 10, end_line: 14,
|
||||
locator: "homelab/runbooks/a.md#10-14" });
|
||||
const msg = pendingWrap([]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
appendUnlocatedVerbatim(msg, [b1], {});
|
||||
addFence(msg, "homelab/runbooks/a.md#10-14");
|
||||
addFence(msg, "homelab/runbooks/a.md#12-40");
|
||||
hydrateVerbatim(bubble, [b1, b2], {});
|
||||
return { panels: placed(msg) };
|
||||
})()""",
|
||||
"never_fenced_keeps_its_panel": """(() => {
|
||||
const msg = pendingWrap([]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
|
||||
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
|
||||
const again = appendUnlocatedVerbatim(msg, [shownBlock()], {});
|
||||
return { appended, settled, again, panels: placed(msg) };
|
||||
})()""",
|
||||
"repeated_fence_shows_the_bytes_once": """(() => {
|
||||
const msg = pendingWrap(["homelab/runbooks/a.md#12-40",
|
||||
"homelab/runbooks/a.md#12-40"]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
|
||||
return { settled, panels: placed(msg) };
|
||||
})()""",
|
||||
"unresolved_twin_is_not_a_placement": """(() => {
|
||||
const msg = pendingWrap(["homelab/runbooks/a.md#1-2"]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
hydrateVerbatim(bubble, [], {}); // the honest state, same locator
|
||||
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
|
||||
return { appended, panels: placed(msg) };
|
||||
})()""",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
fenced = out["fence_arrives_later"]
|
||||
assert fenced["ok"] is True, fenced
|
||||
assert fenced["value"] == {
|
||||
"mid": 0,
|
||||
"appended": 1,
|
||||
"settled": 1,
|
||||
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
|
||||
}, "the fence wins: one panel, in the answer, the mid-turn twin gone"
|
||||
|
||||
ranges = out["two_ranges_one_file"]
|
||||
assert ranges["ok"] is True, ranges
|
||||
assert ranges["value"]["panels"] == [
|
||||
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
|
||||
], "two ranges of one file are TWO panels — the scan is per locator"
|
||||
|
||||
unfenced = out["never_fenced_keeps_its_panel"]
|
||||
assert unfenced["ok"] is True, unfenced
|
||||
assert unfenced["value"] == {
|
||||
"appended": 1,
|
||||
"settled": 0,
|
||||
"again": 0,
|
||||
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}],
|
||||
}, "a block no fence names keeps exactly its one answer-position panel"
|
||||
|
||||
repeated = out["repeated_fence_shows_the_bytes_once"]
|
||||
assert repeated["ok"] is True, repeated
|
||||
assert repeated["value"]["settled"] == 2
|
||||
assert repeated["value"]["panels"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
|
||||
], "the same fence twice shows those bytes once"
|
||||
|
||||
unresolved = out["unresolved_twin_is_not_a_placement"]
|
||||
assert unresolved["ok"] is True, unresolved
|
||||
assert unresolved["value"]["appended"] == 1, (
|
||||
"an unresolved node is not a placement, so the block still lands"
|
||||
)
|
||||
assert unresolved["value"]["panels"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_the_twin_scan_bounds_to_the_message_it_hydrates(tmp_dir: Path) -> None:
|
||||
"""The dedupe's BOUND, run against the real file (task 03): the scan reads
|
||||
the message the hydration belongs to.
|
||||
|
||||
* a bare bubble with no ``.msg`` ancestor still dedupes — the ``|| root``
|
||||
fallback means "no wrap" narrows the scan to the root, it never turns
|
||||
the dedupe off (two panels for one locator is the bug either way);
|
||||
* a second range of the same file keeps its own panel under that fallback
|
||||
too (per locator, never per identity — the fallback is not a wider scan);
|
||||
* a twin that belongs to ANOTHER message survives: hydrating one answer
|
||||
never reaches into the next one's panels."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"bare_bubble_dedupes_too": """(() => {
|
||||
const bubble = pendingBubble([]);
|
||||
const appended = appendUnlocatedVerbatim(bubble, [shownBlock()], {});
|
||||
bareFence(bubble, "homelab/runbooks/a.md#12-40");
|
||||
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
|
||||
return { appended, settled, panels: placed(bubble) };
|
||||
})()""",
|
||||
"bare_bubble_two_ranges_stay_two": """(() => {
|
||||
const other = shownBlock({ start_line: 10, end_line: 14,
|
||||
locator: "homelab/runbooks/a.md#10-14" });
|
||||
const bubble = pendingBubble([]);
|
||||
appendUnlocatedVerbatim(bubble, [shownBlock()], {});
|
||||
bareFence(bubble, "homelab/runbooks/a.md#10-14");
|
||||
const settled = hydrateVerbatim(bubble, [shownBlock(), other], {});
|
||||
return { settled, panels: placed(bubble) };
|
||||
})()""",
|
||||
"another_message_is_out_of_bounds": """(() => {
|
||||
const here = pendingWrap(["homelab/runbooks/a.md#12-40"]);
|
||||
const neighbour = pendingWrap([]);
|
||||
appendUnlocatedVerbatim(neighbour, [shownBlock()], {});
|
||||
const settled = hydrateVerbatim(
|
||||
here.querySelector(".bubble"), [shownBlock()], {});
|
||||
return { settled, here: placed(here), neighbour: placed(neighbour) };
|
||||
})()""",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
bare = out["bare_bubble_dedupes_too"]
|
||||
assert bare["ok"] is True, bare
|
||||
assert bare["value"] == {
|
||||
"appended": 1,
|
||||
"settled": 1,
|
||||
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
|
||||
}, "no .msg wrap still means ONE panel for one locator"
|
||||
|
||||
ranges = out["bare_bubble_two_ranges_stay_two"]
|
||||
assert ranges["ok"] is True, ranges
|
||||
assert ranges["value"]["settled"] == 1
|
||||
assert ranges["value"]["panels"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
|
||||
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
|
||||
], "the fallback bounds the scan — it does not widen it past the locator"
|
||||
|
||||
neighbour = out["another_message_is_out_of_bounds"]
|
||||
assert neighbour["ok"] is True, neighbour
|
||||
assert neighbour["value"]["settled"] == 1
|
||||
assert neighbour["value"]["here"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
|
||||
], "this answer hydrates to one panel"
|
||||
assert neighbour["value"]["neighbour"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
|
||||
], "the next answer's panel is not this scan's to remove"
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_a_stored_record_restores_one_panel_per_locator(tmp_dir: Path) -> None:
|
||||
"""The restore/shared parity of the dedupe (task 03): both call sites run
|
||||
the SAME two passes in the SAME order — hydrate the record's fences, then
|
||||
append the blocks the answer never located. So a stored answer that repeats
|
||||
one locator twice and additionally carries an unlocated block comes back as
|
||||
exactly TWO panels: the locator once, IN the text, and the unlocated block
|
||||
once, in the answer band. One reload, one render — the duplicate the
|
||||
mid-turn frame could leave behind never reaches the record's reader."""
|
||||
out = _node_dom(
|
||||
{
|
||||
"restore": f"""(() => {{
|
||||
const msg = pendingWrap(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
|
||||
const bubble = msg.querySelector(".bubble");
|
||||
const record = [
|
||||
shownBlock(),
|
||||
shownBlock({{
|
||||
source: "homelab", path: "runbooks/c.md", title: "Runbook C",
|
||||
start_line: 1, end_line: 3, total_lines: 9,
|
||||
text: "c one\\nc two", locator: "homelab/runbooks/c.md#1-3",
|
||||
}}),
|
||||
];
|
||||
const settled = hydrateVerbatim(bubble, record, {{}});
|
||||
const appended = appendUnlocatedVerbatim(msg, record, {{}});
|
||||
return {{ settled, appended, panels: placed(msg) }};
|
||||
}})()""",
|
||||
},
|
||||
tmp_dir,
|
||||
)
|
||||
case = out["restore"]
|
||||
assert case["ok"] is True, case
|
||||
assert case["value"]["settled"] == 2, "every fence settles — nothing is left pending"
|
||||
assert case["value"]["appended"] == 1, (
|
||||
"only the block the answer never located lands in the answer band"
|
||||
)
|
||||
assert case["value"]["panels"] == [
|
||||
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
|
||||
{"locator": "homelab/runbooks/c.md#1-3", "where": "body"},
|
||||
], "one locator, one panel — the same answer on every surface"
|
||||
|
||||
@@ -1401,3 +1401,117 @@ def test_saved_chat_total_budget_counts_verbatim_block_text() -> None:
|
||||
assert "8,000,000-character budget" in exc.value.errors()[0]["msg"]
|
||||
# 14 × 560 000 = 7 840 000 fits (plus the small text, still inside).
|
||||
assert len(SavedChatCreate.model_validate({"messages": heavy[:-1]}).messages) == 14
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 192 (task 01, A1): ChatVerbatimBlockEvent — the mid-turn twin of the
|
||||
# done frame's ``answer_text`` item.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_chat_verbatim_block_event_shape() -> None:
|
||||
"""The mid-turn frame's wire shape: ``type`` FIRST (the sibling events'
|
||||
convention) then the exact :class:`VerbatimBlockRef` field set, the
|
||||
``verbatim_block`` default, ``truncated`` defaulting False (the block was
|
||||
not capped), and no ``model_config`` — the phase-95
|
||||
:class:`ChatToolResultEvent` convention mirrored: an SSE frame is
|
||||
server-built, so unknown keys are ignored rather than forbidden."""
|
||||
from app.schemas import ChatVerbatimBlockEvent
|
||||
|
||||
ev = ChatVerbatimBlockEvent(**_verbatim_block())
|
||||
dumped = ev.model_dump()
|
||||
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
|
||||
assert dumped == {"type": "verbatim_block", **_verbatim_block()}
|
||||
minimal = ChatVerbatimBlockEvent(
|
||||
source="Homelab",
|
||||
path="a.md",
|
||||
title="a.md",
|
||||
start_line=1,
|
||||
end_line=1,
|
||||
total_lines=1,
|
||||
text="one line",
|
||||
locator="Homelab/a.md",
|
||||
).model_dump()
|
||||
assert minimal["truncated"] is False
|
||||
assert minimal["type"] == "verbatim_block"
|
||||
# The sibling convention mirrored EXACTLY: an SSE event model declares no
|
||||
# ``extra="forbid"`` (the frame is server-built, so an unknown key is
|
||||
# ignored rather than rejected — like ``ChatToolResultEvent``).
|
||||
from app.schemas import ChatToolResultEvent
|
||||
|
||||
assert ChatVerbatimBlockEvent.model_config.get("extra") is None
|
||||
assert ChatToolResultEvent.model_config.get("extra") is None
|
||||
|
||||
|
||||
def test_chat_verbatim_block_event_field_set_matches_the_done_item() -> None:
|
||||
"""The drift pin: the mid-turn frame carries EXACTLY the done frame's
|
||||
item fields (plus ``type``) with the SAME caps — the one client
|
||||
(``isVerbatimBlock`` + ``buildVerbatimPanel``) hydrates both from one
|
||||
shape, so a field added to :class:`VerbatimBlockRef` without its twin
|
||||
here fails this test."""
|
||||
from app.schemas import ChatVerbatimBlockEvent
|
||||
|
||||
assert set(ChatVerbatimBlockEvent.model_fields) == set(
|
||||
VerbatimBlockRef.model_fields
|
||||
) | {"type"}
|
||||
for name, ref_field in VerbatimBlockRef.model_fields.items():
|
||||
mine = ChatVerbatimBlockEvent.model_fields[name]
|
||||
assert mine.annotation == ref_field.annotation, name
|
||||
assert mine.is_required() == ref_field.is_required(), name
|
||||
assert mine.metadata == ref_field.metadata, name # the caps, verbatim
|
||||
|
||||
# The caps fire (the same limits, enforced on this model too).
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(text="x" * 600_001))
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(source="s" * 121))
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(path="p" * 1001))
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(title="t" * 501))
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(locator="l" * 2101))
|
||||
for field in ("start_line", "end_line", "total_lines"):
|
||||
with pytest.raises(ValidationError):
|
||||
ChatVerbatimBlockEvent(**_verbatim_block(**{field: -1}))
|
||||
|
||||
# ``isVerbatimBlock`` (frontend/assets/verbatim.js) needs exactly these
|
||||
# three: a non-empty ``source`` string, a non-empty ``path`` string and a
|
||||
# ``text`` string — so the mid-turn event is accepted as-is.
|
||||
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
|
||||
assert isinstance(dumped["source"], str) and dumped["source"]
|
||||
assert isinstance(dumped["path"], str) and dumped["path"]
|
||||
assert isinstance(dumped["text"], str)
|
||||
|
||||
|
||||
def test_chat_verbatim_block_event_dump_keys_are_the_client_contract() -> None:
|
||||
"""Phase 192 (task 03, A4): the key-set pin the ONE panel client leans on.
|
||||
|
||||
``frontend/assets/verbatim.js`` has no schema of its own: the
|
||||
``isVerbatimBlock`` guard is a shape check and ``buildVerbatimPanel``
|
||||
reads the fields by name — so the mid-turn frame is only renderable while its
|
||||
``model_dump()`` key set is EXACTLY :class:`VerbatimBlockRef`'s key set
|
||||
plus the ``type`` discriminator. A field added to one model and not the
|
||||
other, a rename, or an ``exclude=`` on the dump all fail here (the client
|
||||
would silently lose the caption, the extent, or the whole block).
|
||||
"""
|
||||
from app.schemas import ChatVerbatimBlockEvent
|
||||
|
||||
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
|
||||
assert set(dumped) == set(VerbatimBlockRef.model_fields) | {"type"}
|
||||
# ``type`` FIRST and the ref's fields in declaration order — the sibling
|
||||
# SSE events' convention, unchanged by the dump.
|
||||
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
|
||||
# The three keys ``isVerbatimBlock`` cannot live without, plus the keys
|
||||
# ``buildVerbatimPanel`` prints (the caption's extent and its open link).
|
||||
for key in ("source", "path", "text", "title", "start_line", "end_line",
|
||||
"total_lines", "truncated", "locator"):
|
||||
assert key in dumped, f"the client reads {key} off both frames"
|
||||
# The done item round-trips the SAME keys: the mid-turn frame is a copy of
|
||||
# a done item, so the restore/shared hydration needs no second reader.
|
||||
assert (
|
||||
VerbatimBlockRef.model_validate(
|
||||
{k: v for k, v in dumped.items() if k != "type"}
|
||||
).model_dump()
|
||||
== {k: v for k, v in dumped.items() if k != "type"}
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user