diff --git a/.agents/phases/todo/189_chat_save_long_turns/00_phase.md b/.agents/phases/complete/189_chat_save_long_turns/00_phase.md similarity index 100% rename from .agents/phases/todo/189_chat_save_long_turns/00_phase.md rename to .agents/phases/complete/189_chat_save_long_turns/00_phase.md diff --git a/.agents/phases/todo/189_chat_save_long_turns/01_schema_cap_realign.md b/.agents/phases/complete/189_chat_save_long_turns/01_schema_cap_realign.md similarity index 100% rename from .agents/phases/todo/189_chat_save_long_turns/01_schema_cap_realign.md rename to .agents/phases/complete/189_chat_save_long_turns/01_schema_cap_realign.md diff --git a/.agents/phases/todo/189_chat_save_long_turns/02_share_saves_first.md b/.agents/phases/complete/189_chat_save_long_turns/02_share_saves_first.md similarity index 100% rename from .agents/phases/todo/189_chat_save_long_turns/02_share_saves_first.md rename to .agents/phases/complete/189_chat_save_long_turns/02_share_saves_first.md diff --git a/.agents/phases/todo/189_chat_save_long_turns/03_persist_rerun_sticky.md b/.agents/phases/complete/189_chat_save_long_turns/03_persist_rerun_sticky.md similarity index 100% rename from .agents/phases/todo/189_chat_save_long_turns/03_persist_rerun_sticky.md rename to .agents/phases/complete/189_chat_save_long_turns/03_persist_rerun_sticky.md diff --git a/.agents/phases/todo/189_chat_save_long_turns/04_e2e_and_gates.md b/.agents/phases/complete/189_chat_save_long_turns/04_e2e_and_gates.md similarity index 100% rename from .agents/phases/todo/189_chat_save_long_turns/04_e2e_and_gates.md rename to .agents/phases/complete/189_chat_save_long_turns/04_e2e_and_gates.md diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.err b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md new file mode 100644 index 0000000..a3ae4c1 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md @@ -0,0 +1,11 @@ +**Phase 189 final verification — all green** + +- **Verified (all 4 tasks already implemented):** A1 caps in `app/schemas.py` — `sources`/`related`/`tools` 200, `suggestions` 100, `segments` 450; `text`/`thinking` 600 000, `error`/`image` 500, `answer_text` 50, `messages` 500, 8 M budget all UNCHANGED; rationale docstrings rewritten to `merge_cited_docs`/`agent_max_rounds`. A2 `shareCurrentChat` linked branch upserts (PUT, 404→create+re-link) before `POST /{id}/share`, aborts on failure (banner, no token); unlinked branch byte-identical. A3 `savePending` + `finally` re-run. A4 `saveFailedSticky` + one 5 s `setTimeout` retry on network/5xx only, 4xx never retries. +- `uv run pytest --cov=app --cov-report=term-missing` → **4609 passed, 1 warning, EXIT=0**, `TOTAL 99%`, `app/schemas.py 100%` (>90%). +- `uv run pytest tests/e2e/test_chat_save_long_turns.py -v --no-cov` → **4 passed, EXIT=0** (isolation). +- Regression green & **unedited** (`git status` clean): `test_chat_save_payload_limits.py` 2, `test_chat_history.py` 5, `test_share_chat.py` 4. Extra blast-radius sweep (share/save/persist/segments surface, all EXIT=0): `test_save_share_ux`, `test_stale_saved_chats`, `test_history_pagination`, `test_history_copy`, `test_bottom_chat_actions`, `test_reasoning_bubble`, `test_tool_budget_telemetry`, `test_verbatim_show_text`, `test_smoke`. No leaked `saved_chats` rows (`save-long-%` = 0). +- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**. +- **Completion criteria:** caps ✓ · incident payload 22/33/51 → 201/200 anonymously + 201 tools/451 segments/101 suggestions → 422 naming the offender, row unchanged ✓ · share-saves-first + abort + unlinked byte-identical ✓ · overlap re-run + sticky note + one-shot retry ✓ · phase E2E in isolation + 3 regression suites unedited ✓ · full suite / coverage / lint / types ✓ · `git diff --stat` = `app/schemas.py`, `frontend/assets/app.js`, 3 test files, `README.md` + new E2E & phase files (no `app/api/chats.py`, no migration, no new dependency) ✓ · commit + `todo/`→`complete/` move left to the harness (rules forbid git index writes). +- **Docs:** full README review (all 2 258 lines) + machine cross-check — all 85 settings documented in the Configuration reference, all 34 `.env.example` vars documented, documented defaults match `app/config.py`. Fixed 2 inconsistencies: (1) the 117-char prose line task 04 left when reflowing the Saved-chats pager paragraph; (2) the phase-189 sentence implying the 8 M total bounds the widened arrays — now states precisely that the total counts `text`/`thinking`/verbatim-block text only, and that the arrays are bounded by fixed item COUNT (matches `_require_within_total_budget`). AGENTS.md verified consistent — it enumerates no save contract or list cap (rule 5's phase-166 in-flight-turn clause untouched), so **no AGENTS.md change** per the phase mandate. +- **Notable/deviation:** the wider caps raise the *theoretical* anonymous-POST worst case (arrays are outside the 8 M char budget); left as designed (A1 fixes every other bound), documented honestly in README — realistic payloads stay ~10 MB, so the existing `client_max_body_size ≥ 16 MB` guidance still holds. No live-infra changes were made (data-repair of the two stale shared rows stays owner action). +- **Next pending phase:** none — `189_chat_save_long_turns` is the last entry in `.agents/phases/todo/`. diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md.status b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.validate b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.validate new file mode 100644 index 0000000..7163747 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__00_phase.a1.validate @@ -0,0 +1,146 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 4%] +........................................................................ [ 6%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 18%] +........................................................................ [ 20%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 32%] +........................................................................ [ 34%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 57%] +........................................................................ [ 59%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +........................................................................ [ 99%] +. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 114 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 292 1 99% +app/api/chat_images.py 52 0 100% +app/api/chats.py 111 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 239 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 231 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 325 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 43 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/memory.py 32 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_migrate.py 32 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 40 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 105 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 494 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 865 3 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 162 0 100% +app/rag/retriever.py 259 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/rag/svg_text.py 111 0 100% +app/rag/verbatim.py 160 0 100% +app/schemas.py 549 0 100% +-------------------------------------------------- +TOTAL 6946 16 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.err b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md new file mode 100644 index 0000000..5a3e186 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md @@ -0,0 +1,18 @@ +**Task 01 complete — all gates green.** + +**Implemented** +- `app/schemas.py` — `ChatMessage` list caps moved to the real agent configuration (A1): `sources` 20→**200**, `related` 20→**200**, `tools` 50→**200**, `suggestions` 50→**100**, `segments` 50→**450**; every other bound untouched (`text`/`thinking` 600 000, `error`/`image` 500, `answer_text` 50, `SourceRef`/`ToolCall` field caps, `messages` 500, `SAVED_CHAT_TOTAL_CHARS_BUDGET` 8 M). Class docstring + field comments rewritten to the `merge_cited_docs` / `agent_max_rounds` rationale (phase-83/164 history kept) with the item-count-vs-8 M-byte-bound sentence; stale "cap matches the tools/segments number" notes on `answer_text` corrected. No route/API/SSE change. +- `tests/unit/test_schemas.py` — constants moved to the new values (`SOURCES_CAP`/`TOOLS_CAP` 200, `SUGGESTIONS_CAP` 100, new `RELATED_CAP` 200 / `SEGMENTS_CAP` 450); new at-cap/one-over pairs for `related` and `segments`; segments list pin retargeted off `TOOLS_CAP`; new `test_incident_long_turn_payload_round_trips` (22 sources / 33 tools / 40 segments, byte-identical `model_dump()`, `path` still absent). +- `tests/integration/test_chats_api.py` — `…_sources_over_20…` → `…_sources_over_200…` (201 refs → 422, no row); `test_segments_over_50_entries_rejected` → `…_over_450…`; NEW `test_post_accepts_the_incident_payload_shape` (22/33/51 → **201**, verified in the 201 body, raw JSONB and the admin GET), `test_put_accepts_the_incident_payload_shape` (**200**, `message_count` bumps), `test_put_rejects_over_cap_lists_and_leaves_row_unchanged` (201 tools / 451 segments / 101 suggestions → 422, row byte-identical). +- `README.md` — "Conversation payload limits": 4 new cap rows + the "list caps follow the agent's real round cap" ops note (incident explained in owner terms); stale "the tools/segments number" row reworded. + +**Gates** (`uv run …`) +- `pytest tests/unit/test_schemas.py tests/integration/test_chats_api.py -v` → green; `pytest` → **4602 passed** (re-run 5×, all green); `pytest --cov=app --cov-report=term-missing` → **TOTAL 99%**, `app/schemas.py` **100%**; `ruff check .` → All checks passed; `pyright` → 0 errors. +- Regression E2E green unedited: `test_chat_save_payload_limits.py` (2), `test_share_chat.py` (4), `test_chat_history.py` (5). + +**Notes / deviations** +- `AGENTS.md`: verified it enumerates no save points or saved-chat caps → **no change** (per the phase's "verify, do not add"). +- One-off in the *first* full run of this session: 3 phase-188 `test_chunks_hnsw_recall.py` planner pins failed once and passed in every run since (isolated, paired, and 4 full runs); not reproducible, phase-188 pins left untouched (LOCKED A3/A6) — DB left at head 0029, `ix_chunks_embedding_hnsw` VALID, `chunks` empty. +- Diff limited to `app/schemas.py`, `README.md`, the two test files (no `app/api/chats.py`, no migration, no dependency). + +**Next pending task:** `189_chat_save_long_turns/02_share_saves_first.md` (A2 — `shareCurrentChat` upserts before minting the token). diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md.status b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.validate b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.validate new file mode 100644 index 0000000..b13b7ba --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__01_schema_cap_realign.a1.validate @@ -0,0 +1,145 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 4%] +........................................................................ [ 6%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 18%] +........................................................................ [ 20%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 25%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 32%] +........................................................................ [ 34%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 50%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 57%] +........................................................................ [ 59%] +........................................................................ [ 61%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 75%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 98%] +.................................................................. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 114 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 292 1 99% +app/api/chat_images.py 52 0 100% +app/api/chats.py 111 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 239 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 231 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 325 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 43 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/memory.py 32 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_migrate.py 32 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 40 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 105 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 494 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 865 3 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 162 0 100% +app/rag/retriever.py 259 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/rag/svg_text.py 111 0 100% +app/rag/verbatim.py 160 0 100% +app/schemas.py 549 0 100% +-------------------------------------------------- +TOTAL 6946 16 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.err b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md new file mode 100644 index 0000000..428828c --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md @@ -0,0 +1 @@ +(no final assistant message — see the .err log) diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md.status b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md.status new file mode 100644 index 0000000..44a0abf --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__02_share_saves_first.a1.md.status @@ -0,0 +1 @@ +error: assistant error: litellm.APIConnectionError: APIConnectionError: OpenAIException - Response payload is not completed: 90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.err b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md new file mode 100644 index 0000000..2dffd2e --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md @@ -0,0 +1,16 @@ +**Implemented (task 03 — A3/A4 in `frontend/assets/app.js`)** +- `persistConversation`: added module-scope `savePending` / `saveFailedSticky` / `retryScheduled`; the overlap guard is now `if (persisting) { savePending = true; return; }`, a fresh run consumes the flag before building the payload, and the `finally` (after `persisting = false`) re-runs `void persistConversation()` once — serialized, never concurrent, latest `conversation`. +- Failure arms (non-ok + `catch`) keep the exact phase-55 note, set `saveFailedSticky`, and schedule ONE 5 s retry (`res.status >= 500` arm / the network `catch` arm) behind the one-shot `retryScheduled` guard set before the `setTimeout`, reset inside it; a 4xx never retries. Success clears the flag and resets `#send-status`; still no banner, still silent on success. +- `persistConversation` + the file-header docstrings updated to the A3/A4 reality; `regenerateStaleChat` and `app/api/chats.py` untouched. +- `tests/unit/test_save_chat_ui.py`: 4 new marker pins (overlap re-run ordering, sticky-note lifetime, one-shot 5 s transient retry, 4xx-never-retries) + the module docstring extended. + +**Gates** +- `uv run pytest` → **4609 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **99% TOTAL, `app/schemas.py` 100%** (>90%). +- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. +- E2E regressions (isolation runs, `--no-cov`): `test_chat_save_payload_limits.py`, `test_share_chat.py`, `test_chat_persistence.py`, `test_chat_history.py`, `test_hidden_tab_stream.py`, `test_nav_switch_keeps_stream.py`, `test_edit_and_resend.py` → all green unedited. + +**Docs**: `README.md` — Saved-chats section gains the phase-189 auto-save note (queued save re-runs; failure note persists until the next success; 5 s retry on network/5xx only). `AGENTS.md` — verified it never enumerates the save contract, so no change (per phase doc). + +**Deviation**: one pre-existing literal pin (`"if (persisting) return;"`) is superseded by the phase's own design (`if (persisting) { savePending = true; return; }`); re-pinned to assert the arm still returns with no second request and exactly one `persisting = true;`. + +**Next pending task**: `.agents/phases/todo/189_chat_save_long_turns/04_e2e_and_gates.md`. diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md.status b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.validate b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.validate new file mode 100644 index 0000000..7163747 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__03_persist_rerun_sticky.a1.validate @@ -0,0 +1,146 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 4%] +........................................................................ [ 6%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 18%] +........................................................................ [ 20%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 32%] +........................................................................ [ 34%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 57%] +........................................................................ [ 59%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +........................................................................ [ 99%] +. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 114 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 292 1 99% +app/api/chat_images.py 52 0 100% +app/api/chats.py 111 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 239 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 231 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 325 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 43 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/memory.py 32 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_migrate.py 32 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 40 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 105 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 494 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 865 3 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 162 0 100% +app/rag/retriever.py 259 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/rag/svg_text.py 111 0 100% +app/rag/verbatim.py 160 0 100% +app/schemas.py 549 0 100% +-------------------------------------------------- +TOTAL 6946 16 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.err b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md new file mode 100644 index 0000000..f66e2a6 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md @@ -0,0 +1,21 @@ +All green. Summary: + +**Implemented (task 04)** +- `tests/e2e/test_chat_save_long_turns.py` — the phase's dedicated Playwright suite (module docstring carries the incident evidence): (1) incident payload (22 sources / 33 tools / 51 segments) `POST /api/chats` **201** + `PUT` **200** anonymously + admin `GET` round-trip of all three lists; (2) a real mock turn's done-time auto-save lands (row 2 msgs, saved brain text = the mock answer); (3) send-time save parked 6 s → dropped done-time save **re-runs** (log `create`(1 msg) → `update`(2 msgs) + a page-side `MutationObserver` clock that proves the overlap, so the test can't pass vacuously); (4) auto-saves rejected → Share **upserts before minting** (request-log ordering) and the shared page renders turn 2's brain answer, plus the A4 sticky note visible with no error banner. +- README consistency pass: reflowed the merged Saved-chats paragraph; corrected the phase-182 sentence "the owner-locked share strings **and behavior** are unchanged" (now: strings unchanged, phase 189 adds the save-before-token upsert). **AGENTS.md: verified, no change** (no save-contract/cap sentence exists there; rule 5's in-flight-turn clause untouched). + +**Gates (exact)** +- `uv run pytest --cov=app --cov-report=term-missing` → **4609 passed**, `TOTAL 99%`, `app/schemas.py 100%` (>90%). +- `uv run pytest tests/e2e/test_chat_save_long_turns.py -v --no-cov` → **4 passed in isolation**. +- Regression, green unedited: `test_chat_save_payload_limits.py` 2 passed, `test_chat_history.py` 5 passed, `test_share_chat.py` 4 passed. +- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**. +- **Regression proof:** with HEAD's `app/schemas.py` + `app.js` the new suite is 3 failed / 1 passed (baseline); files restored (md5 verified). + +**Notable decisions / deviations** +- Scenario 4 makes the row stale with an injected **422** (the incident's own rejection) instead of the task's route delay: a route-handler `time.sleep` blocks the Playwright sync dispatcher (measured), so a "click while parked" test races the parked stale write against the share's write; a 422 also exercises A4's no-retry rule, leaving the share's upsert as the only writer → deterministic. Documented in the module docstring. +- Scenario 3 holds the first **write** (the `POST` create — a fresh conversation's send-time save point is a create, not the PUT the task text assumed). +- Found and documented a house-testing trap: the sync dispatcher only pumps route interception while inside a Playwright call, so the row-poll uses `page.wait_for_timeout` (an `httpx`/`time.sleep` poll starves the re-run upsert and times out on a healthy browser). +- Tests clean up their rows via title lookup, so a **failing** test leaks nothing; purged 8 rows my debug runs left in the shared e2e DB (0 remaining) and removed the temporary probe files. +- `git diff --stat` scope is exactly the allowed files (no `app/api/chats.py`, no migration, no dependency). **Steps 5–6 (the atomic commit and the `todo/`→`complete/` move) were left to the harness** per the binding "no git index/history commands" rule; suggested message: `fix(chats): save long agent turns — realign schema caps, share saves first, dropped saves re-run` (+ `LLM-Generated: true`, `--no-gpg-sign`; house convention = one commit folding in the phase dir, per `f20d40e`). + +**Next pending task:** none — 04 is the last task of phase 189. diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md.status b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.validate b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.validate new file mode 100644 index 0000000..7163747 --- /dev/null +++ b/.agents/reports/189_chat_save_long_turns/189_chat_save_long_turns__04_e2e_and_gates.a1.validate @@ -0,0 +1,146 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 4%] +........................................................................ [ 6%] +........................................................................ [ 7%] +........................................................................ [ 9%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 18%] +........................................................................ [ 20%] +........................................................................ [ 21%] +........................................................................ [ 23%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 32%] +........................................................................ [ 34%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 39%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 43%] +........................................................................ [ 45%] +........................................................................ [ 46%] +........................................................................ [ 48%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 57%] +........................................................................ [ 59%] +........................................................................ [ 60%] +........................................................................ [ 62%] +........................................................................ [ 64%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 71%] +........................................................................ [ 73%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 78%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 85%] +........................................................................ [ 87%] +........................................................................ [ 89%] +........................................................................ [ 90%] +........................................................................ [ 92%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +........................................................................ [ 99%] +. [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 114 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 292 1 99% +app/api/chat_images.py 52 0 100% +app/api/chats.py 111 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 239 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 231 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 325 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 43 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/memory.py 32 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_migrate.py 32 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 40 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 105 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 494 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 865 3 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 162 0 100% +app/rag/retriever.py 259 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/rag/svg_text.py 111 0 100% +app/rag/verbatim.py 160 0 100% +app/schemas.py 549 0 100% +-------------------------------------------------- +TOTAL 6946 16 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/README.md b/README.md index a236f77..aa95920 100644 --- a/README.md +++ b/README.md @@ -285,9 +285,21 @@ List, edit, or remove them here — no conversation required. ### Saved chats (`/history.html`) Every conversation is saved automatically. Click a title to return to that -chat. Admin-only view. The list is paginated server-side (25 rows per -page) — with more than 25, a Prev / “Page N of M” / Next pager appears -under the table to walk it, and **Refresh** re-fetches the current page. +chat. Admin-only view. + +**The auto-save never drops or silently loses a turn** (phase 189). Save points +overlap on a long turn (the send, the answer landing, a page unload), and only +one upsert runs at a time — a save point that arrives mid-upsert is now queued +and re-runs once the in-flight request settles, so the last word always reaches +the row instead of waiting for a next save point that may never come. When a +save does fail, the one-line note under the composer *stays* until the next +successful save (it used to be overwritten and forgotten), and a transient +failure — a network drop or a 5xx — retries by itself after 5 s; a rejected +payload (4xx) never retries, because retrying it would reject it again. + +The list is paginated server-side (25 rows per page) — with more than 25, a +Prev / “Page N of M” / Next pager appears under the table to walk it, and +**Refresh** re-fetches the current page. `GET /api/chats` carries the same contract: `?limit=` (default 25, clamped 1–100) / `?offset=` and a full `total` on every response. @@ -325,6 +337,18 @@ file has to let the reader copy it and check it. Both come from the same owner's), and a guest who follows the viewer link lands on the sign-in page — the documents API is admin-gated. +**Sharing saves first** (phase 189). **Share** on a conversation that is +already saved writes the CURRENT conversation to its row — the same upsert the +auto-save uses, deleted-row included (a 404 recreates the row and re-links) — +and only then mints the token, so the shared page is the conversation you are +looking at and never a snapshot it has outgrown. The owner incident: a long +research turn whose done-time auto-save was rejected left the row holding only +the question, and the Share froze that loss behind a link. If that save fails +the share **aborts** — the neutral *"Couldn't share the conversation — try +again."* banner, no link, no toast — because a link to content that is not the +current conversation is worse than no link. A conversation that was never +saved still saves **and** shares in one action. + ### Conversation payload limits A conversation has one shape, and it carries the **same static bounds** @@ -339,10 +363,29 @@ saving a conversation can never be stricter than continuing it (phase | Turns sent as `history` | 500 | the whole browser conversation (the client sends everything it holds, minus the question being asked) | | Messages per saved chat | 500 | the same conversation, saved | | One verbatim block (`answer_text[].text`) | 600 000 chars | the same answer-text cap (phase 182) — comfortably above `BOR_SHOW_TEXT_MAX_CHARS`, so raising that budget never hits the wire limit | -| Verbatim blocks per saved message | 50 | the `tools` / `segments` number (phase 182) — far past the `BOR_SHOW_TEXT_MAX_BLOCKS` default of 5 | +| Verbatim blocks per saved message | 50 | 10× the `BOR_SHOW_TEXT_MAX_BLOCKS` default of 5 (phase 182) — this one follows the display budget, not the round count | +| Cited / related docs per message (`sources`, `related`) | 200 each | **4× the round cap** (production `BOR_AGENT_MAX_ROUNDS=50`) — the citation chips are the documents the agent actually READ, not just the top-N retrieved, so one long research turn cites roughly one document per round and more (phase 189) | +| Tool calls per message (`tools`) | 200 | the same 4× the round cap: a round is one model request, and one-or-more tool calls per round are legal (phase 189) | +| Timeline entries per message (`segments`) | 450 | the thinking/tool timeline of that same turn — ≤ 200 tool entries + one thinking entry per contiguous thinking run (≤ ~50) + margin (phase 189) | +| Suggestion chips per message | 100 | 20× the `BOR_SUGGESTED_DOCS` default of 5; each chip stays ≤ 200 chars (phase 189) | | **Total chars per saved chat** | **8 000 000** | ~15 full-length answers; the one bound on the anonymous save surface, which stays bounded on purpose — and it counts the verbatim blocks' bytes too (phase 182), so the display blocks cannot become a second, unbounded byte channel | | One question (`message`) | 4 000 chars | unchanged; the composer counts it live | +**The list caps follow the agent's real round cap** (phase 189). They were +sized for the shipped `BOR_AGENT_MAX_ROUNDS` default of 10, when the citation +row was assumed to be "the top-N docs plus a handful of reads"; since phase +119 the cited tier *is* the agent's read documents, so a long research turn — +like the owner's, on a container running `BOR_AGENT_MAX_ROUNDS=50` — cites two +dozen documents. Such a turn used to be **rejected by its own auto-save** (a +silent 422 at the old 20-source cap, swallowed by the quiet-failure contract), +which is why a shared chat could hold only the question. An operator raising +`BOR_AGENT_MAX_ROUNDS` far past 50 still hits a fixed boundary — the caps +scale with the configuration, not with a guess. The byte bound is unchanged +too: the 8 000 000-character total above counts each message's `text`, +`thinking` and verbatim-block text (never the citation/tool arrays), which is +exactly why those arrays keep a fixed item COUNT instead of a byte count — +neither channel can grow without a limit. + These are sanity ceilings, not the prompt's budget: the real gate is `BOR_HISTORY_MAX_TURNS` / `BOR_HISTORY_MAX_CHARS`, and a turn that does not fit the budget is dropped **whole** from later prompts (never cut @@ -1303,7 +1346,8 @@ The model never retypes quoted text, so a displayed quote cannot drift. model's, never a silently-substituted hybrid. Buttons appear when the turn settles, never mid-stream. The share link's copy path delegates to the same `copyTextWithFallback`, so the http-origin inline select-to-copy degradation - exists once (the owner-locked share strings and behavior are unchanged). + exists once (the owner-locked share strings are unchanged; phase 189 added + the save-before-the-token upsert in front of the same success paths). - **Parity, and what stays untouched.** The live turn, a reload and an anonymous shared page call the same three globals in the same order, so one answer renders one way. Only the two answer renderers (plus the shared diff --git a/app/schemas.py b/app/schemas.py index 1fa29fb..61bfb7d 100644 --- a/app/schemas.py +++ b/app/schemas.py @@ -1410,13 +1410,35 @@ class ChatMessage(BaseModel): path accepts", and it survives the raise only if BOTH sides move: raise one and every done-time auto-save of a full-length answer 422s into the phase-113 quiet-failure), and the nested lists get length - caps (``max_length``) sized to the realistic ``bor.chat.v1`` record - the UI produces (``sources`` ≤ 20 — top-N docs - + agent reads; ``related`` ≤ 20 — the related-doc tier, same ref - shape; ``suggestions`` ≤ 50 chips of ≤ 200 chars; ``tools`` - ≤ 50 — one entry per tool call, the round cap is 10). Only value - bounds were added — the accepted/rejected KEYS are unchanged. The - per-conversation TOTAL is bounded on that same anonymous surface by + caps (``max_length``). + + Phase 189 (A1, owner incident 2026-10-16) re-sizes those LIST caps + to the real agent configuration. The cited tier has been the agent's + READ DOCS since phase 119 (``merge_cited_docs``, + ``app/api/chat.py`` — retrieval top-N ∪ every document the agent + read), so its size is bounded by ``agent_max_rounds`` (production + ``BOR_AGENT_MAX_ROUNDS=50``; the validator is unbounded above), NOT + by ``top_n_docs``. The phase-83 sizing ("sources ≤ 20 — top-N docs + + agent reads; tools ≤ 50 — the round cap is 10") was therefore wrong + for a long research turn: a 22-source done frame 422'd its own + done-time auto-save into the same quiet-failure, and the shared row + froze holding only the question. The caps are now + ``sources`` / ``related`` / ``tools`` **200** (4× the production + round cap — a round is ONE model request and one-or-more tool calls + per round are legal; ``related`` shares the ``SourceRef``-family cap + because ``related_max_docs`` is owner-tunable with no upper bound), + ``suggestions`` **100** (20× the ``suggested_docs`` default of 5, also + tunable unbounded above; the ≤ 200-char chip cap is unchanged) and + ``segments`` **450** (see the field comment). Only value bounds + moved — the accepted/rejected KEYS are unchanged, the 422 stays the + framework's standard response and no route changed (the phase-83 A2 + style). The fixed boundary is what bounds item COUNT (an operator + raising ``BOR_AGENT_MAX_ROUNDS`` far past 50 still hits a boundary, + never an unbounded list); :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET` is + the BYTE bound — the worst-case list bytes (≈ 360 KB of + ``SourceRef``s, ≈ 440 KB of ``ToolCall``s, ≈ 1 MB of segments) stay + orders of magnitude inside the 8 000 000-character total. That + per-conversation TOTAL is enforced on the same anonymous surface by :class:`SavedChatCreate` / :class:`SavedChatUpdate`'s character budget (phase 164): the caps above bind per message, so the list-level validator is what keeps the worst case at the audited @@ -1454,9 +1476,11 @@ class ChatMessage(BaseModel): round-trips byte-identically to pre-phase. The field EXISTS so the ``extra="forbid"`` boundary does NOT 422 the done-time auto-save carrying it (the phase-113 lesson, applied up front; - the phase-124 ``answer_image`` precedent). ``max_length=50`` (the - phase-83 voice): far past any realistic turn — the round cap is - 10 → ≤ 20 entries; 50 matches the ``tools`` cap. Segments never + the phase-124 ``answer_image`` precedent). ``max_length=450`` (the + phase-189 sizing, replaces the phase-83 "round cap is 10 → ≤ 20 + entries"): ≤ 200 ``tool`` entries (the ``tools`` cap — 4× the + production round cap of 50) + ≤ ~50 ``thinking`` runs (one per + round, the cap-50 worst case) + margin. Segments never travel the history wire (A10 unchanged — the concatenated ``thinking`` travels as ``reasoning_content`` exactly as today; :class:`HistoryTurn` and @@ -1467,12 +1491,22 @@ class ChatMessage(BaseModel): who: Literal["user", "brain"] text: str = Field(min_length=1, max_length=600_000) - sources: list[SourceRef] | None = Field(default=None, max_length=20) - related: list[SourceRef] | None = Field(default=None, max_length=20) + # Phase 189 (A1): 20 → 200 — the cited tier is the agent's READ DOCS + # (``merge_cited_docs``), so its size follows ``agent_max_rounds`` + # (production 50), not ``top_n_docs``. See the class docstring. + sources: list[SourceRef] | None = Field(default=None, max_length=200) + # Phase 189 (A1): 20 → 200 — same SourceRef family, same cap + # (``related_max_docs`` is owner-tunable with no upper bound). + related: list[SourceRef] | None = Field(default=None, max_length=200) deflected: bool | None = None - suggestions: list[_Chip] | None = Field(default=None, max_length=50) + # Phase 189 (A1): 50 → 100 — 20× the ``suggested_docs`` default of 5 + # (tunable, unbounded above); the ≤ 200-char chip cap is unchanged. + suggestions: list[_Chip] | None = Field(default=None, max_length=100) thinking: str | None = Field(default=None, max_length=600_000) - tools: list[ToolCall] | None = Field(default=None, max_length=50) + # Phase 189 (A1): 50 → 200 — a round is ONE model request and + # one-or-more tool calls per round are legal, so the cap follows + # 4× the production round cap, not the round cap itself. + tools: list[ToolCall] | None = Field(default=None, max_length=200) # Phase 158 (task 01, D4/D6 — owner direction 2026-10-04): the # turn's CHRONOLOGICAL run order — one ``thinking`` entry per # contiguous thinking run (that run's accumulated text), one @@ -1487,10 +1521,10 @@ class ChatMessage(BaseModel): # byte-identically to pre-phase. The field EXISTS so the # extra="forbid" boundary does NOT 422 the done-time auto-save # carrying it (the phase-113 lesson, applied up front; the - # phase-124 answer_image precedent). max_length=50 (the phase-83 - # voice, the tools cap): far past any realistic turn — the round - # cap is 10 → ≤ 20 entries; 50 matches the tools cap. - segments: list[ChatSegment] | None = Field(default=None, max_length=50) + # phase-124 answer_image precedent). max_length=450 (the phase-189 + # sizing): ≤ 200 tool entries (the tools cap, 4× the production + # round cap of 50) + ≤ ~50 thinking runs (one per round) + margin. + segments: list[ChatSegment] | None = Field(default=None, max_length=450) stopped: bool | None = None # Phase 120 (task 01, locked A1): the failed-turn marker + the # persisted error detail (the phase-48 ``stopped`` precedent). @@ -1562,8 +1596,9 @@ class ChatMessage(BaseModel): # :meth:`_drop_image_when_absent` serializer below), so a turn with # no blocks — and every pre-phase record — round-trips # byte-identically (A9). The per-block ``text`` cap is - # :class:`VerbatimBlockRef`'s own; the list cap matches the - # ``tools``/``segments`` cap (50), far past the + # :class:`VerbatimBlockRef`'s own; the list cap stays 50 (the phase-83 + # number, UNCHANGED by phase 189 — it follows + # ``show_text_max_blocks``, not the round count), far past the # ``show_text_max_blocks`` default of 5 — and the block bytes count # toward :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET`, so the display blocks # never become a second, unbounded channel on the anonymous save diff --git a/frontend/assets/app.js b/frontend/assets/app.js index 68da46e..e69b1e7 100644 --- a/frontend/assets/app.js +++ b/frontend/assets/app.js @@ -174,12 +174,17 @@ * our back) unlinks and retries as a create, so a stale link can never * wedge the conversation. The module-level `persisting` flag is the * double-fire guard: the save points can overlap (pagehide during a - * stream), so a call while an upsert is in flight is a no-op — the - * next save point retries. The A2 quiet contract: a FAILED auto-save + * stream), so a call while an upsert is in flight does not start a + * second one — phase 189 (A3) marks it PENDING (`savePending`) and the + * in-flight upsert's finally re-runs it once, so a save point is never + * silently dropped. The A2 quiet contract: a FAILED auto-save * never blocks the conversation — a one-line #send-status note ("Couldn't * save automatically — will try on the next message."), NO error banner; - * a SUCCESSFUL auto-save is silent (the History page is the visible - * proof — the toast is reserved for share). The row link survives + * phase 189 (A4) keeps that note until the NEXT SUCCESSFUL save and + * retries ONCE after 5 s on a transient failure (network or 5xx — never + * a 4xx, which is deterministic); a SUCCESSFUL auto-save is silent (the + * History page is the visible proof — the toast is reserved for share). + * The row link survives * reloads: the bor.chat.v1 record carries `chatId` (null when unlinked; * a pre-55 record without the field reads as null — never throws), so a * refresh restores the conversation AND its link. "New chat" unlinks @@ -3206,19 +3211,41 @@ async function restoreSavedChatFromUrl() { * * The `persisting` flag is the double-fire guard: the save points can * overlap (pagehide during a stream), so a call while an upsert is in - * flight is a no-op — the next save point retries. The A2 quiet contract + * flight never starts a second upsert. Phase 189 (A3, the owner incident + * 2026-10-16) retired the DROP: the overlapping save point sets + * `savePending` and the in-flight upsert's finally re-runs + * persistConversation() once it has fully settled — a fresh, serialized + * call that reads `conversation` at ITS call time, so a done-time save + * dropped by a still-running send-time save lands with the brain record + * instead of vanishing until a save point that may never come (the + * incident row kept only the question). The A2 quiet contract * on failure (non-ok HTTP or network): a one-line #send-status note, - * NO error banner, the turn never blocks. Success is SILENT (the History + * NO error banner, the turn never blocks. Phase 189 (A4) extends that + * contract's LIFETIME without changing its shape: the note stays until + * the NEXT SUCCESSFUL save (`saveFailedSticky`), and a TRANSIENT failure + * (a network throw or a 5xx) schedules ONE retry after 5 s + * (`retryScheduled` — overlapping failures never stack retries); a 4xx + * never retries, because a 422 is deterministic and task 01's cap realign + * is the remedy. Success is SILENT (the History * page is the visible proof — the toast is reserved for share), apart * from clearing the phase-53 stale banner (a successful re-save * re-stamps the row to the current generation — the row is no longer - * stale). */ + * stale) and clearing a stuck failure note. */ let persisting = false; // phase 55: one upsert at a time (double-fire guard) +let savePending = false; // phase 189 (A3): a save point that arrived mid-upsert re-runs +let saveFailedSticky = false; // phase 189 (A4): the failure note stays until a success +let retryScheduled = false; // phase 189 (A4): one transient retry in flight at a time async function persistConversation() { if (!conversation.length) return; // nothing to save - if (persisting) return; // an upsert is already in flight (double-fire guard) + // A3 (phase 189): an upsert is already in flight — mark this save point + // PENDING (never drop it) and let the finally below re-run it once. + if (persisting) { + savePending = true; + return; + } persisting = true; + savePending = false; // this run consumes the pending flag (a re-run is this run) const body = JSON.stringify({ messages: conversation }); const headers = { "Content-Type": "application/json" }; try { @@ -3240,6 +3267,18 @@ async function persistConversation() { // next save point retries. sendStatus.textContent = "Couldn't save automatically — will try on the next message."; + // A4 (phase 189): the note is STICKY — it stays until a save + // succeeds — and a 5xx is transient, so it gets ONE retry after 5 s. + // A 4xx never retries: a 422 is deterministic (task 01's cap realign + // is the remedy), and the retry would 422 again. + saveFailedSticky = true; + if (res.status >= 500 && !retryScheduled) { + retryScheduled = true; // one retry at a time — overlapping failures never stack + setTimeout(() => { + retryScheduled = false; + void persistConversation(); // the LATEST conversation, the same guards + }, 5000); + } return; } if (res.status === 201) { @@ -3250,12 +3289,39 @@ async function persistConversation() { // current generation (phase 53, task 03): the row is no longer // stale, so the banner is done. if (staleBanner) staleBanner.hidden = true; + // A4 (phase 189): a success clears a stuck failure note — the note's + // lifetime is "until the next successful save" (other flows may + // overwrite it meanwhile; the flag + the next save point are the + // guarantee, never a silently-failed save with no trace). + if (saveFailedSticky) { + saveFailedSticky = false; + sendStatus.textContent = ""; + } } catch { // A2 quiet contract: a network failure is the same one-line note. sendStatus.textContent = "Couldn't save automatically — will try on the next message."; + // A4 (phase 189): the catch arm IS the network-failure arm — sticky + // note plus ONE retry after 5 s (the one-shot guard sits BEFORE the + // setTimeout so overlapping failures cannot stack retries). + saveFailedSticky = true; + if (!retryScheduled) { + retryScheduled = true; + setTimeout(() => { + retryScheduled = false; + void persistConversation(); // the LATEST conversation, the same guards + }, 5000); + } } finally { persisting = false; // released on EVERY outcome + // A3 (phase 189): a save point that arrived while THIS upsert ran is + // re-run once, after the in-flight request fully settled — never + // concurrent with it, and the flag is cleared first so the re-run + // does not schedule a second one. + if (savePending) { + savePending = false; + void persistConversation(); + } } } @@ -3341,7 +3407,20 @@ function showToast(message) { * banner is the failure UI). 403/5xx → the actionable banner * (neutral — the write surface is public, phase 55 task 01); a network * failure → the reachable? banner. The double-click guard releases in - * the finally — never stale (PLAN §7.4). */ + * the finally — never stale (PLAN §7.4). + * + * Phase 189 (task 02, A2 — the owner incident 2026-10-16): the LINKED + * branch UPSERTS the current conversation BEFORE minting the token, on + * the same upsert semantics as persistConversation (PUT; a 404 unlinks + * and retries as a create, a 201 re-links). A Share that minted a token + * on a row the conversation had outgrown published a truncated snapshot + * — the incident row held ONLY the question because the done-time + * auto-save had 422'd into the phase-55 quiet failure, and the Share + * then froze that loss behind a link. An upsert FAILURE ABORTS the share + * (the neutral banner, no token, no toast): the owner must never be + * handed a link to content that is not the current conversation. The + * UNLINKED branch is unchanged — its POST { messages, share: true } + * already saves AND shares the CURRENT conversation in one commit. */ async function shareCurrentChat() { if (!conversation.length) { sendStatus.textContent = "Nothing to share yet."; @@ -3352,14 +3431,38 @@ async function shareCurrentChat() { try { let shareUrl; if (currentChatId) { - // Linked (already saved): the idempotent share — an existing - // token comes back unchanged, a new one is minted. - const res = await fetch(`/api/chats/${currentChatId}/share`, { method: "POST" }); + // Phase 189 (A2): SAVE FIRST. persistConversation's own upsert — + // PUT the current conversation, a 404 (the row was deleted on the + // History page) unlinks and retries as a create, a 201 re-links. + // Sharing without this would mint the token on whatever the row + // last held, freezing a truncated snapshot behind a link. + const body = JSON.stringify({ messages: conversation }); + const headers = { "Content-Type": "application/json" }; + let res = await fetch(`/api/chats/${currentChatId}`, { method: "PUT", headers, body }); + if (res.status === 404) { + // Stale link: the row is gone — unlink and retry as a create, + // so a Share never dies on (or publishes) a deleted row. + currentChatId = null; + res = await fetch("/api/chats", { method: "POST", headers, body }); + } if (!res.ok) { + // The upsert failed: ABORT the share — no token, no toast, no + // link. The neutral banner is the existing failure UI and the + // finally below still releases the button (never stale). showErrorBanner("Couldn't share the conversation — try again."); return; } - shareUrl = (await res.json()).share_url; + if (res.status === 201) { + currentChatId = String((await res.json()).id); // re-linked to the new row + } + // Saved and current: the idempotent share — an existing + // token comes back unchanged, a new one is minted. + const minted = await fetch(`/api/chats/${currentChatId}/share`, { method: "POST" }); + if (!minted.ok) { + showErrorBanner("Couldn't share the conversation — try again."); + return; + } + shareUrl = (await minted.json()).share_url; } else { // Unsaved: save AND share in ONE action (owner-locked) — the // server sets the 128-bit uuid4 token in the same commit. diff --git a/tests/e2e/test_chat_save_long_turns.py b/tests/e2e/test_chat_save_long_turns.py new file mode 100644 index 0000000..e152fe7 --- /dev/null +++ b/tests/e2e/test_chat_save_long_turns.py @@ -0,0 +1,791 @@ +"""Phase 189 E2E (Playwright): a LONG agent turn saves — and Share publishes it. + +Run in isolation (DB must be up: ``podman compose up -d db``): + + uv run pytest tests/e2e/test_chat_save_long_turns.py -v --no-cov + +WHY THESE TESTS EXIST (owner incident 2026-10-16, read this before editing) +-------------------------------------------------------------------------- +Shared chat ``0101d2ca-b24c-4d13-9f65-c0837e993c50`` (row +``8b345ab5-aab3-4412-9642-0f869b3cec4d``) shows ONLY the user's question +although the LLM answered it completely. The app log for that row carries the +whole sequence: + +* the per-turn line ``thinking=xhigh thinking_chars=23937 tool_calls=33 + rounds=34 cap_hit=0 total_ms=400173`` + ``POST /api/chat 200 stream=1`` — + the client consumed the FULL 400 s stream, so the ``done`` frame landed; +* the done frame carried **22 sources** (the cited tier is the agent's READ + docs since phase 119 — ``merge_cited_docs`` — not the retrieval top-N, and + production runs ``BOR_AGENT_MAX_ROUNDS=50``), 33 tool calls and one + thinking entry per contiguous thinking run; +* immediately after it: ``PUT /api/chats/8b345ab5… 422 Unprocessable + Entity`` — the done-time auto-save was REJECTED at the phase-83 list caps + sized for the round-cap-10 era (``sources`` ≤ 20, ``tools`` ≤ 50, + ``segments`` ≤ 50); +* the phase-55 A2 quiet contract swallowed the failure (a one-line + ``#send-status`` note, no banner) and NOTHING retried, so the row kept its + pre-turn state — ``created_at == updated_at``, ``n_msgs = 1``, a + single-write user-only row; +* ``POST …/share`` then minted its token on that stale row (84 other + single-write user-only rows exist, 2 of them shared). + +The four tests below pin the owner-visible contract of the phase's four +fixes, through the real UI and the real wire: + +* **A1 (task 01) — the caps follow the real configuration.** The incident + payload shape (22 ``sources`` / 33 ``tools`` / 51 ``segments``) is + **201** on ``POST /api/chats`` and **200** on ``PUT /api/chats/``, + anonymously, and the row round-trips all three lists (pre-phase: both + **422** — the incident). ``tests/integration/test_chats_api.py`` pins the + over-cap variants at the new boundaries (201/201/451/101 → 422). +* **A3 (task 03) — a save point dropped by the in-flight guard re-runs.** + A real mock turn's done-time auto-save lands even when the send-time + auto-save is still parked at the server: the row ends with BOTH messages + (pre-phase: the dropped save never came back and the row held 1). +* **A4 (task 03) — a failed save stays visible.** The one-line + "Couldn't save automatically — will try on the next message." note is on + screen after the failed auto-saves (no error banner — the A2 shape is + unchanged) and the next successful save clears it. +* **A2 (task 02) — Share saves first.** With the auto-saves failing, the + Share click UPSERTS the current conversation before minting the token + (pinned in the request log: an in-cap ``PUT`` immediately precedes the + ``POST …/share``), and the shared page renders the BRAIN answer — not just + the question (pre-phase: exactly that, the frozen truncated snapshot). + +Wire-shape notes for whoever edits the interception helpers: a Playwright +route handler that ``time.sleep``s blocks the sync dispatcher thread, so the +test thread cannot act during the hold — the delay-based "click while the +auto-save is parked" shape therefore races the parked (stale) write against +the share's own write for the row. Scenario 4 makes the row stale the +deterministic way instead (an injected **422** on the auto-saves — exactly +the incident's rejection, and one phase 189 A4 deliberately never retries), +which leaves no competing write in flight; scenario 3 keeps the hold and pins +the overlap PROPERLY by timestamping the answer's arrival inside the page (a +``MutationObserver``) and comparing it with the release time of the parked +request — so it cannot quietly degrade into a non-overlapping turn that passes +for the wrong reason. Scenario 3 holds the first WRITE (the ``POST`` create — +a fresh conversation's send-time save point creates the row; a ``PUT`` only +exists once a row is linked) and scenario 4's stale-row turn is a second turn +on a linked row, where the save points are ``PUT``s. + +DB isolation: the shared e2e Postgres keeps ``saved_chats`` rows across +suites, so every test here uses a DISTINCTIVE question text (its auto-title +is therefore unique), never asserts on absolute row counts, and deletes the +rows it creates in a ``finally`` (admin cookie). The KB tables are truncated ++ re-seeded the house way (deterministic mock embeddings) ONCE per module — +these tests only need a GROUNDED mock answer, never a corpus change. +""" +from __future__ import annotations + +import asyncio +import json +import re +import time +from collections.abc import Callable +from pathlib import Path +from threading import Thread +from typing import Any + +import httpx +import pytest +from playwright.sync_api import Browser, BrowserContext, Page, Route, expect +from sqlalchemy import text + +from app.config import Settings +from app.db import SessionLocal +from app.rag.importer import ImportSummary, import_sources +from app.rag.llm import LLMClient +from e2e.auth_helpers import login + +REPO = Path(__file__).resolve().parents[2] +FIXTURES = REPO / "tests" / "fixtures" / "docs" +MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E" +#: The phase-55 A2 quiet note (unchanged by phase 189 — only its LIFETIME +#: is longer: it now stays until the next successful save). +SAVE_NOTE = "Couldn't save automatically — will try on the next message." + +#: The saved-chat WRITE surface as a URL pattern: the collection (the +#: ``POST /api/chats`` create) and one row (the ``PUT /api/chats/`` +#: update). The share mint (``POST /api/chats//share``) and the chat +#: SSE stream (``/api/chat``) deliberately do NOT match. +CHAT_WRITE_URL = re.compile(r"https?://[^/]+/api/chats(/[0-9a-f-]+)?$") +#: The share mint, observed (never held) so the request LOG proves the +#: upsert-before-token ordering of phase 189 A2. +CHAT_SHARE_URL = re.compile(r"https?://[^/]+/api/chats/[0-9a-f-]+/share$") + +#: Long enough that the mock turn (≈ 1–2 s on this corpus) settles while the +#: first auto-save is still parked; short enough to keep the suite quick. +HOLD_S = 6.0 + +#: The page-side watch that stamps WHEN the answer text first reached the +#: DOM. Needed because the route handler's hold blocks the sync dispatcher +#: (see the module docstring): without a page-side clock the overlap +#: scenario could not prove the done-time save point arrived mid-upsert. +_WATCH_ANSWER_JS = """ +(marker) => { + window.__borAnswerAt = null; + const check = () => { + const bubbles = document.querySelectorAll('.msg.brain .bubble'); + if (!bubbles.length) return; + const last = bubbles[bubbles.length - 1]; + if ((last.textContent || '').includes(marker) + && window.__borAnswerAt === null) { + window.__borAnswerAt = Date.now(); + } + }; + new MutationObserver(check).observe(document.body, { + childList: true, subtree: true, characterData: true, + }); + check(); +} +""" + + +# --------------------------------------------------------------------------- +# DB / app plumbing (the house pattern from test_chat_history.py) +# --------------------------------------------------------------------------- + + +async def _import_fixtures(mock_port: int) -> ImportSummary: + kwargs: dict[str, Any] = { + "_env_file": None, + "llm_base_url": f"http://127.0.0.1:{mock_port}/v1", + } + settings = Settings(**kwargs) # pyright: ignore[reportCallIssue] + return await import_sources([FIXTURES], LLMClient(settings)) + + +def _run_in_thread(coro: Any) -> Any: + """Run a coroutine on a worker thread — Playwright's sync API keeps an + asyncio loop on the test thread, so ``asyncio.run`` cannot be called + directly from a test body.""" + box: dict[str, Any] = {} + + def runner() -> None: + try: + box["value"] = asyncio.run(coro) + except BaseException as e: # noqa: BLE001 — re-raised on the test thread + box["error"] = e + + t = Thread(target=runner) + t.start() + t.join() + if "error" in box: + raise box["error"] + return box["value"] + + +@pytest.fixture(scope="module") +def kb(mock_llm: int) -> None: + """Truncate the KB (and the query log — deterministic mock answers) and + re-import the fixtures ONCE for the module: every scenario here only + needs a grounded (non-deflected) mock answer. ``saved_chats`` is + deliberately NOT touched — rows persist across suites and each test + cleans up after itself.""" + with SessionLocal() as db: + db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes")) + db.commit() + _run_in_thread(_import_fixtures(mock_llm)) + + +# --------------------------------------------------------------------------- +# Admin-side helpers (the API is the proof — auto-saves are SILENT, A2) +# --------------------------------------------------------------------------- + + +def _admin_cookies(page: Page) -> dict[str, str]: + """The signed session cookies the browser holds after a form login.""" + return { + c["name"]: c["value"] + for c in page.context.cookies() + if "name" in c and "value" in c + } + + +def _auto_title(question: str) -> str: + """The phase-50 auto-title convention: the first question, + whitespace-collapsed, capped at 120 chars.""" + return " ".join(question.split())[:120] + + +def _find_row( + app_url: str, cookies: dict[str, str], title: str +) -> dict[str, Any] | None: + r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies) + assert r.status_code == 200 + return next((c for c in r.json()["chats"] if c["title"] == title), None) + + +def _row(app_url: str, cookies: dict[str, str], chat_id: str) -> dict[str, Any]: + r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies) + assert r.status_code == 200, r.text + return r.json() + + +def _wait_row_messages( + app_url: str, + cookies: dict[str, str], + title: str, + messages: int, + page: Page, + timeout_s: float = 20.0, +) -> dict[str, Any]: + """Poll the admin list until the row with this conversation's auto-title + holds AT LEAST ``messages`` messages (the upsert is fire-and-forget from + the UI's point of view, and the parked one lands when the hold ends). + + The poll pumps the browser (``page.wait_for_timeout``) rather than + ``time.sleep``: the sync Playwright dispatcher only runs while the test + thread is inside a Playwright call, so a sleeping poll starves a pending + route interception — the re-run upsert would sit unanswered and the wait + would time out on a browser that is actually fine.""" + deadline = time.monotonic() + timeout_s + last: dict[str, Any] | None = None + while time.monotonic() < deadline: + last = _find_row(app_url, cookies, title) + if last is not None and last["message_count"] >= messages: + return last + page.wait_for_timeout(200) + raise AssertionError(f"no auto-saved row for {title!r} with {messages} msgs (last: {last!r})") + + +def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None: + """Best-effort row cleanup (a 404 — already deleted — is fine).""" + httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies) + + +def _cleanup( + app_url: str, cookies: dict[str, str], chat_id: str | None, title: str +) -> None: + """Delete the row this test created — by id when the test got that far, + by its unique auto-title otherwise (a FAILING test must not leak a row + into the shared e2e database).""" + if chat_id is None: + leaked = _find_row(app_url, cookies, title) + if leaked is None: + return + chat_id = str(leaked["id"]) + _delete_chat(app_url, cookies, chat_id) + + +def _ask(page: Page, question: str) -> float: + """Send one turn and wait until the answer has fully landed (the + ``done`` event restored the Send button). Returns the seconds the turn + took — the calibration the ``HOLD_S`` comment cites.""" + started = time.monotonic() + page.fill("#message-input", question) + page.click("#send-btn") + expect(page.locator(".msg.user .bubble").last).to_contain_text(question) + expect(page.locator(".msg.brain .bubble").last).to_contain_text( + MOCK_ANSWER_MARKER, timeout=30_000 + ) + expect(page.locator("#send-btn")).to_be_enabled() + expect(page.locator("#send-label")).to_have_text("Send") + return time.monotonic() - started + + +def _watch_answer(page: Page) -> None: + """Install the page-side clock (see ``_WATCH_ANSWER_JS``) BEFORE the send + whose turn it stamps.""" + page.evaluate(_WATCH_ANSWER_JS, MOCK_ANSWER_MARKER) + + +def _answer_at(page: Page) -> float | None: + """The wall-clock ms at which the answer text first reached the DOM + (``None`` when the watch never fired).""" + value = page.evaluate("() => window.__borAnswerAt") + return None if value is None else float(value) + + +def _msg_count(post_data: str | None) -> int: + """How many messages a save body carries — the request-level view of + what the auto-save was trying to write.""" + try: + messages = json.loads(post_data or "{}").get("messages") + except json.JSONDecodeError: + return -1 + return len(messages) if isinstance(messages, list) else -1 + + +def _hold_first_chat_write(page: Page, hold_s: float = HOLD_S) -> list[dict[str, Any]]: + """Hold the FIRST write to the saved-chat surface for ``hold_s`` seconds + before letting it reach the server (the archive-upload suite's pattern), + and log every write that passes through. + + Each entry is ``{kind, messages, started_ms, released_ms}`` with ``kind`` + in ``create``/``update``/``mint``; ``started_ms``/``released_ms`` are wall + clock ms (comparable with the page's ``Date.now()``), which is what lets + scenario 3 prove the done-time save point landed while the create was + still parked.""" + log: list[dict[str, Any]] = [] + state = {"held": False, "seq": 0} + + def handle(route: Route) -> None: + req = route.request + if req.method not in ("POST", "PUT"): + route.continue_() + return + started = time.time() * 1000 + entry = { + "seq": state["seq"], + "kind": "create" if req.method == "POST" else "update", + "messages": _msg_count(req.post_data), + "outcome": "held", + "started_ms": started, + "released_ms": started, + } + state["seq"] += 1 + if not state["held"]: + state["held"] = True + time.sleep(hold_s) # the parked auto-save — nothing reaches the server yet + entry["released_ms"] = time.time() * 1000 + log.append(entry) + route.continue_() + + def handle_mint(route: Route) -> None: + log.append( + { + "seq": state["seq"], + "kind": "mint", + "messages": -1, + "outcome": "pass", + "started_ms": time.time() * 1000, + "released_ms": time.time() * 1000, + } + ) + state["seq"] += 1 + route.continue_() + + page.route(CHAT_WRITE_URL, handle) + page.route(CHAT_SHARE_URL, handle_mint) + return log + + +def _reject_chat_writes( + page: Page, +) -> tuple[list[dict[str, Any]], Callable[[bool], None]]: + """Answer every write to the saved-chat surface with an injected **422** + (the incident's own rejection) and log every write that passes through. + + Returns the log and the arming switch: ``set_armed(False)`` lets the + writes through (the Share's own upsert must reach the server) while the + log keeps recording, so the ordering assertion covers the share too. + + Two deliberate choices: + + * a REJECTION, not a delay — a held request keeps a STALE body in flight + that then races the share's own write for the row (see the module + docstring), while a rejected save leaves the row exactly as the + incident left it (behind the conversation) with no competing write; + * a **422**, not a 5xx — phase 189 A4 retries a TRANSIENT failure only, + so nothing re-saves behind the test's back and the ONLY thing that can + bring the row up to date is the Share's own upsert (A2). The shared + page is therefore a clean witness for A2, and the request log is the + ordering proof.""" + log: list[dict[str, Any]] = [] + state = {"seq": 0, "armed": True} + + def handle(route: Route) -> None: + req = route.request + if req.method not in ("POST", "PUT"): + route.continue_() + return + entry = { + "seq": state["seq"], + "kind": "create" if req.method == "POST" else "update", + "messages": _msg_count(req.post_data), + "outcome": "rejected" if state["armed"] else "pass", + } + state["seq"] += 1 + log.append(entry) + if state["armed"]: + route.fulfill( + status=422, + content_type="application/json", + body='{"detail":[{"loc":["body"],"msg":"injected by phase 189 e2e"}]}', + ) + return + route.continue_() + + def handle_mint(route: Route) -> None: + log.append( + { + "seq": state["seq"], + "kind": "mint", + "messages": -1, + "outcome": "pass", + } + ) + state["seq"] += 1 + route.continue_() + + def set_armed(value: bool) -> None: + state["armed"] = value + + page.route(CHAT_WRITE_URL, handle) + page.route(CHAT_SHARE_URL, handle_mint) + return log, set_armed + + +def _grant_clipboard(page: Page, app_url: str) -> None: + """Grant the async-clipboard permissions (``http://127.0.0.1`` is a + secure context, but headless Chromium still needs the grant before + ``writeText`` resolves — otherwise the inline-link fallback fires).""" + page.context.grant_permissions( + ["clipboard-read", "clipboard-write"], origin=app_url + ) + + +def _click_share(page: Page) -> None: + """Press the chat page's Share pill and wait for its outcome on the live + region — the clipboard line or the inline-field fallback, whichever the + context supports (the phase-51 contract).""" + page.locator("#share-chat-btn").click() + if page.evaluate("() => !!navigator.clipboard"): + expect(page.locator("#send-status")).to_have_text( + "Share link copied.", timeout=20_000 + ) + else: + expect(page.locator("#send-status")).to_have_text( + "Share link ready — copy it from the field.", timeout=20_000 + ) + + +# --------------------------------------------------------------------------- +# The incident payload: the done frame of a 34-round / 33-tool turn +# --------------------------------------------------------------------------- + + +def _incident_source(i: int) -> dict[str, str]: + """A real ``SourceRef`` at realistic column values (``source`` ≤ 120, + ``path`` ≤ 1000, ``title`` ≤ 500 — the phase-83 mirrors of the + ``documents`` columns).""" + return { + "source": "Homelab", + "path": f"notes/deploy/kubernetes/component-{i:02d}.md", + "title": f"Component {i:02d} — cluster setup, upgrades and rollback notes", + } + + +def _incident_tool(i: int) -> dict[str, Any]: + """A real ``ToolCall`` as the UI's ``toolAcc`` keeps it (the read the + agent ran in round ``i``).""" + return { + "name": "read", + "argument": f"Homelab/notes/deploy/kubernetes/component-{i:02d}.md", + "truncated": False, + "chars_shown": 4_200 + i, + "chars_total": 4_200 + i, + "offset": None, + "limit": None, + } + + +def _incident_brain() -> dict[str, Any]: + """The incident turn's brain record: 22 ``sources`` (the READ docs — + ``merge_cited_docs``), 33 ``tools``, 51 ``segments`` (the 33 tool entries + + 18 thinking entries, one per contiguous thinking run of the 34-round + ``xhigh`` turn).""" + tools = [_incident_tool(i) for i in range(33)] + segments: list[dict[str, Any]] = [] + for i, tool in enumerate(tools): + if i < 18: # a thinking run opens the first 18 rounds + segments.append({"kind": "thinking", "text": f"round {i} scratchpad"}) + segments.append({"kind": "tool", **tool}) + return { + "who": "brain", + "text": "Everything I found, across all 22 documents I read.", + "sources": [_incident_source(i) for i in range(22)], + "related": [_incident_source(i) for i in range(2)], + "deflected": False, + "suggestions": [f"What about component {i}?" for i in range(5)], + "thinking": " ".join(f"round {i}" for i in range(18)), + "tools": tools, + "segments": segments, + } + + +def _assert_incident_lists(messages: list[dict[str, Any]]) -> None: + """All three lists survived the round trip intact — nothing silently + truncated on the way in, and the segment chronology is preserved.""" + brain = messages[1] + assert len(brain["sources"]) == 22 + assert len(brain["related"]) == 2 + assert len(brain["tools"]) == 33 + assert len(brain["segments"]) == 51 + assert [s["path"] for s in brain["sources"]] == [ + _incident_source(i)["path"] for i in range(22) + ] + assert [t["argument"] for t in brain["tools"]] == [ + _incident_tool(i)["argument"] for i in range(33) + ] + kinds = [s["kind"] for s in brain["segments"]] + assert kinds[:3] == ["thinking", "tool", "thinking"] + assert kinds[-1] == "tool" + assert kinds.count("thinking") == 18 and kinds.count("tool") == 33 + + +# --------------------------------------------------------------------------- +# 1. A1 at the wire: the incident payload saves (201 / 200), anonymously +# --------------------------------------------------------------------------- + + +def test_incident_payload_shape_saves_at_the_wire( + page: Page, app_url: str, db_ready: None +) -> None: + """THE incident regression at the network layer: the done-time auto-save + of a long turn — 22 cited READ docs, 33 tool calls, 51 timeline segments + — is accepted anonymously (``POST`` **201**, the re-save ``PUT`` + **200**) and the row round-trips all three lists. Pre-phase-189 both + calls were **422** at the round-cap-10 list caps, the quiet contract + swallowed the failure and the row kept its pre-turn (question-only) + state.""" + page.set_default_timeout(30_000) + # Settled ANONYMOUS state — the write surface is public, and the + # rejection happened on an anonymous browser save (the audit vector). + page.goto(app_url + "/") + expect(page.locator("#sign-in-link")).to_be_visible(timeout=15_000) + + question = "Audit my whole homelab and tell me what needs updating (phase 189 wire)" + body = { + "messages": [{"who": "user", "text": question}, _incident_brain()], + } + + created = page.request.post(app_url + "/api/chats", data=body) + assert created.status == 201, ( + f"the incident shape must save, not 422: {created.status} {created.text()}" + ) + payload = created.json() + chat_id = str(payload["id"]) + assert payload["message_count"] == 2 + _assert_incident_lists(payload["messages"]) + + try: + # The done-time auto-save's actual path: the re-save upsert of the + # linked row (the incident's 422 rode exactly this request). + updated = page.request.put(app_url + f"/api/chats/{chat_id}", data=body) + assert updated.status == 200, ( + f"the incident shape must re-save, not 422: {updated.status} {updated.text()}" + ) + _assert_incident_lists(updated.json()["messages"]) + + # The row itself, through the admin read (the browser's own origin). + login(page, app_url, next="/") + cookies = _admin_cookies(page) + got = _row(app_url, cookies, chat_id) + assert got["message_count"] == 2 + _assert_incident_lists(got["messages"]) + finally: + login(page, app_url, next="/") + _delete_chat(app_url, _admin_cookies(page), chat_id) + + +# --------------------------------------------------------------------------- +# 2. A real mock turn's done-time auto-save lands (row: 2 messages) +# --------------------------------------------------------------------------- + + +def test_done_time_auto_save_lands_for_a_mock_turn( + page: Page, app_url: str, kb: None, db_ready: None +) -> None: + """The happy path through the real UI: one grounded mock turn, no + interception at all. The conversation auto-saves at the save points and + the row ends with BOTH messages, the brain record holding the answer the + bubble showed (the baseline every other scenario here deviates from).""" + page.set_default_timeout(30_000) + login(page, app_url, next="/") + expect(page).to_have_url(app_url + "/", timeout=30_000) + + q = "How is my Kubernetes cluster set up? (save-long-baseline)" + turn_s = _ask(page, q) + + cookies = _admin_cookies(page) + created: str | None = None + try: + row = _wait_row_messages(app_url, cookies, _auto_title(q), 2, page) + created = str(row["id"]) + assert row["message_count"] == 2, "the send + done save points both landed" + messages = _row(app_url, cookies, created)["messages"] + assert [m["who"] for m in messages] == ["user", "brain"] + assert messages[0]["text"] == q + assert MOCK_ANSWER_MARKER in messages[1]["text"], ( + "the saved brain record holds the answer the bubble showed" + ) + assert q[:40] in messages[1]["text"], ( + "the mock answer echoes the question — the saved text is THAT turn" + ) + # The calibration HOLD_S cites: the mock turn settles long before the + # hold ends, which is what makes scenario 3's overlap deterministic. + assert turn_s < HOLD_S - 2.0, ( + f"the mock turn took {turn_s:.2f}s — HOLD_S ({HOLD_S}s) no longer " + "clears it, so scenario 3's overlap would not be deterministic" + ) + finally: + _cleanup(app_url, cookies, created, _auto_title(q)) + + +# --------------------------------------------------------------------------- +# 3. A3: the done-time save dropped by the in-flight guard RE-RUNS +# --------------------------------------------------------------------------- + + +def test_dropped_done_time_save_reruns_after_the_inflight_upsert_settles( + page: Page, app_url: str, kb: None, db_ready: None +) -> None: + """The owner-visible half of evidence 6: the send-time auto-save is + parked at the server (held in the route), the mock turn settles inside + that window, and the done-time save point it produces is refused by the + one-upsert-at-a-time guard. Pre-phase-189 that save point VANISHED + ("the next save point retries" — which never came) and the row stayed + question-only; phase 189 A3 marks it PENDING and the parked upsert's + ``finally`` re-runs it once it settles, so the row ends with BOTH + messages.""" + page.set_default_timeout(30_000) + login(page, app_url, next="/") + expect(page).to_have_url(app_url + "/", timeout=30_000) + + _watch_answer(page) + log = _hold_first_chat_write(page) + + q = "How is my Kubernetes cluster set up? (save-long-overlap)" + _ask(page, q) # the answer lands while the create is still parked + + cookies = _admin_cookies(page) + created: str | None = None + try: + row = _wait_row_messages(app_url, cookies, _auto_title(q), 2, page) + created = str(row["id"]) + # The owner-visible outcome: the conversation saved COMPLETE. + assert row["message_count"] == 2, ( + "the dropped done-time save must re-run once the parked upsert settles" + ) + messages = _row(app_url, cookies, created)["messages"] + assert [m["who"] for m in messages] == ["user", "brain"] + assert MOCK_ANSWER_MARKER in messages[1]["text"] + + # The mechanism, from the request log: ONE create carrying only the + # question (the send-time save point), then exactly ONE update + # carrying both messages (the pending re-run). Pre-phase-189 the + # update never happened at all. + assert [e["kind"] for e in log] == ["create", "update"], ( + f"expected the parked create then the single re-run, got {log}" + ) + assert log[0]["messages"] == 1, "the send-time save held only the question" + assert log[1]["messages"] == 2, "the re-run carried the brain record" + + # And the overlap is REAL, not a turn that happened to be slow: the + # answer reached the DOM while the create was still parked, so the + # done-time save point ran into the in-flight guard (a page-side + # clock — the test thread cannot observe anything during the hold). + answer_at = _answer_at(page) + parked = log[0] + assert answer_at is not None, "the page-side answer watch never fired" + assert parked["started_ms"] < answer_at < parked["released_ms"], ( + f"the answer reached the DOM at {answer_at}, outside the window the " + f"send-time save was parked " + f"[{parked['started_ms']}, {parked['released_ms']}] — the save points " + "did not overlap, so this test would not pin the re-run" + ) + finally: + _cleanup(app_url, cookies, created, _auto_title(q)) + + +# --------------------------------------------------------------------------- +# 4. A2 (+A4 visible): Share saves the CURRENT conversation before the token +# --------------------------------------------------------------------------- + + +def test_share_saves_first_publishes_the_brain_message( + page: Page, + browser: Browser, + app_url: str, + kb: None, + db_ready: None, +) -> None: + """The incident's second half, through the real UI: the auto-saves do not + land (an injected 422 — EXACTLY the incident's rejection), so the row + stays behind the conversation, and the owner clicks Share. Pre-phase-189 + the Share minted its token on whatever the row last held, so the shared + page showed ONLY the question; phase 189 A2 upserts the current + conversation first (an in-cap ``PUT`` in the request log immediately + before the ``POST …/share``) and the shared page carries the brain + answer. A4 rides along visibly: the failed auto-saves leave the one-line + note on screen (still no error banner, the phase-55 A2 shape).""" + page.set_default_timeout(30_000) + login(page, app_url, next="/") + expect(page).to_have_url(app_url + "/", timeout=30_000) + + # Turn 1 saves cleanly, so the conversation is LINKED to a row (the + # branch the incident rode: a linked Share, token-only pre-phase). + q1 = "How is my Kubernetes cluster set up? (save-long-share-1)" + _ask(page, q1) + cookies = _admin_cookies(page) + row = _wait_row_messages(app_url, cookies, _auto_title(q1), 2, page) + chat_id = row["id"] + + anon_ctx: BrowserContext | None = None + try: + # The two auto-save points of turn 2 (send, brain-done) are refused: + # the row stays at turn 1's state — the incident's stale row. + log, set_armed = _reject_chat_writes(page) + q2 = "How is my Kubernetes cluster set up? (save-long-share-2)" + _ask(page, q2) + + rejected = [e for e in log if e["outcome"] == "rejected"] + assert len(rejected) == 2, f"both turn-2 save points must be refused: {log}" + assert [e["messages"] for e in rejected] == [3, 4], ( + f"the send point held 3 messages, the done point all 4: {log}" + ) + # The row is BEHIND the conversation — the incident's precondition. + assert _row(app_url, cookies, chat_id)["message_count"] == 2 + + # A4: the failure is NOT silent any more — the one-line note is on + # screen (and still no error banner, the phase-55 A2 shape). + expect(page.locator("#send-status")).to_contain_text(SAVE_NOTE) + expect(page.locator("#kb-banner")).to_be_hidden() + + # A2: the Share click saves FIRST, then mints. (The injection comes + # off for the share itself — a share whose upsert fails ABORTS, and + # that branch is marker-pinned in tests/unit/test_save_chat_ui.py.) + set_armed(False) + _grant_clipboard(page, app_url) + _click_share(page) + + upserts = [e for e in log if e["kind"] == "update" and e["outcome"] == "pass"] + mints = [e for e in log if e["kind"] == "mint"] + assert len(mints) == 1, f"exactly one token mint: {log}" + assert upserts, ( + "the linked Share must upsert the current conversation before minting" + ) + assert upserts[-1]["seq"] < mints[-1]["seq"], ( + f"the share's upsert must precede the mint: {log}" + ) + assert upserts[-1]["messages"] == 4, ( + "the share upsert carried the WHOLE conversation, answer included" + ) + + # The row caught up (2 turns = 4 messages) and carries the token. + row = _wait_row_messages(app_url, cookies, _auto_title(q1), 4, page) + assert row.get("share_url"), "the mint produced the share link" + share_url = str(row["share_url"]) + + # The shared page: the FULL conversation — the brain answer of turn 2 + # included (pre-phase-189: one question and nothing else). + anon_ctx = browser.new_context() + anon = anon_ctx.new_page() + anon.set_default_timeout(30_000) + anon.goto(app_url + share_url) + expect(anon.locator("#shared-title")).to_have_text(_auto_title(q1)) + expect(anon.locator(".msg.user .bubble")).to_have_count(2) + bubbles = anon.locator(".msg.brain .bubble") + expect(bubbles).to_have_count(2) + expect(bubbles.nth(1)).to_contain_text(MOCK_ANSWER_MARKER, timeout=20_000) + # The shared answer is turn 2's — the incident lost exactly this. + expect(bubbles.nth(1)).to_contain_text("(save-long-share-2)") + # Zero chips restored for a turn that read nothing (the phase-119 + # shape the shared page keeps). + expect(anon.locator("a.source-chip")).to_have_count(0) + finally: + if anon_ctx is not None: + anon_ctx.close() + _delete_chat(app_url, cookies, chat_id) diff --git a/tests/integration/test_chats_api.py b/tests/integration/test_chats_api.py index 377e656..967f3ee 100644 --- a/tests/integration/test_chats_api.py +++ b/tests/integration/test_chats_api.py @@ -1859,16 +1859,18 @@ def test_post_rejects_more_than_500_messages_and_stores_nothing( assert len(admin_client.get("/api/chats").json()["chats"]) == baseline -def test_post_rejects_sources_over_20_and_stores_nothing( +def test_post_rejects_sources_over_200_and_stores_nothing( client: TestClient, admin_client: TestClient ) -> None: - """One brain message carrying a 21-item ``sources`` list (all valid - ``SourceRef`` shapes) 422 (``ChatMessage.sources`` ``max_items=20`` - — top-N docs + agent reads) and nothing lands.""" + """One brain message carrying a 201-item ``sources`` list (all valid + ``SourceRef`` shapes) 422 (``ChatMessage.sources`` ``max_length=200`` + — phase 189 A1: the cited tier is the agent's READ docs, so the cap + follows ``agent_max_rounds`` (production 50) × calls-per-round, not + ``top_n_docs``) and nothing lands.""" baseline = len(admin_client.get("/api/chats").json()["chats"]) sources = [ {"source": f"src{i}", "path": f"docs{i}.md", "title": f"Doc {i}"} - for i in range(21) + for i in range(201) ] r = client.post( @@ -1885,6 +1887,176 @@ def test_post_rejects_sources_over_20_and_stores_nothing( assert len(admin_client.get("/api/chats").json()["chats"]) == baseline +# ---------- the phase-189 incident payload (owner incident 2026-10-16) ---------- +# +# The shared chat 0101d2ca… (row 8b345ab5…) shows ONLY the question: the +# turn's done frame carried 22 sources (the cited tier is the agent's READ +# docs since phase 119 — merge_cited_docs), 33 tool calls and one thinking +# entry per contiguous thinking run, and the done-time auto-save 422'd at +# the pre-phase caps (sources 20 / tools 50 / segments 50). The quiet +# contract swallowed the failure, nothing retried, and Share minted its +# token on the stale row. These pins are the wire-level regression: the +# same shape saves. + + +def _incident_source(i: int) -> dict[str, Any]: + return {"source": "Homelab", "path": f"notes/doc{i}.md", "title": f"Doc {i}"} + + +def _incident_brain() -> dict[str, Any]: + """The incident turn's brain record: 22 ``sources``, 33 ``tools``, 51 + ``segments`` (the 33 tool entries + 18 thinking entries — one per + contiguous thinking run of a 34-round ``xhigh`` turn).""" + tools: list[dict[str, Any]] = [ + { + "name": "read", + "argument": f"Homelab/notes/doc{i}.md", + "truncated": False, + "chars_shown": None, + "chars_total": None, + "offset": None, + "limit": None, + } + for i in range(33) + ] + segments: list[dict[str, Any]] = [] + for i, tool in enumerate(tools): + if i < 18: # one thinking entry opens the first 18 runs + segments.append({"kind": "thinking", "text": f"thinking run {i}"}) + segments.append({"kind": "tool", **tool}) + assert len([s for s in segments if s["kind"] == "thinking"]) == 18 + return { + "who": "brain", + "text": "The whole investigation, across every document I read.", + "sources": [_incident_source(i) for i in range(22)], + "related": [_incident_source(i) for i in range(2)], + "deflected": False, + "suggestions": [f"Tell me more about doc {i}" for i in range(5)], + "thinking": "".join(f"run {i} " for i in range(18)), + "tools": tools, + "segments": segments, + } + + +def _assert_incident_lists(messages: list[dict[str, Any]]) -> None: + """The brain record of a saved conversation carries all three lists + intact (order included) — nothing silently truncated on the way in.""" + brain = messages[1] + assert len(brain["sources"]) == 22 + assert len(brain["related"]) == 2 + assert len(brain["tools"]) == 33 + assert len(brain["segments"]) == 51 + assert brain["sources"] == [_incident_source(i) for i in range(22)] + assert brain["segments"] == _incident_brain()["segments"] + assert brain["tools"] == _incident_brain()["tools"] + + +def test_post_accepts_the_incident_payload_shape( + client: TestClient, admin_client: TestClient, db: Session +) -> None: + """THE incident regression at the wire: the 22-source / 33-tool / + 51-segment done-time auto-save is **201** (pre-phase-189: 422 into the + quiet failure) and the row round-trips all three lists byte-for-byte + — anonymous, exactly as the browser's auto-save posts it.""" + r = client.post( + "/api/chats", + json={ + "messages": [ + _user("Audit my whole homelab and tell me what needs updating"), + _incident_brain(), + ] + }, + ) + assert r.status_code == 201, f"expected 201, got {r.status_code}: {r.text}" + body = r.json() + _assert_incident_lists(body["messages"]) + + raw = db.execute( + text("SELECT messages FROM saved_chats WHERE id = :id"), + {"id": body["id"]}, + ).scalar_one() + _assert_incident_lists(raw) + + got = admin_client.get(f"/api/chats/{body['id']}") + assert got.status_code == 200 + _assert_incident_lists(got.json()["messages"]) + + +def test_put_accepts_the_incident_payload_shape( + client: TestClient, admin_client: TestClient +) -> None: + """The SAME shape through the re-Save upsert (the path the done-time + auto-save actually rides) is **200** and the row's stored messages + bump to the new array — the pre-turn snapshot is replaced, not + rejected.""" + created = client.post( + "/api/chats", + json={"messages": [_user("Audit my whole homelab and tell me what needs updating")]}, + ).json() + assert created["message_count"] == 1 + + r = client.put( + f"/api/chats/{created['id']}", + json={ + "messages": [ + _user("Audit my whole homelab and tell me what needs updating"), + _incident_brain(), + ] + }, + ) + assert r.status_code == 200, f"expected 200, got {r.status_code}: {r.text}" + _assert_incident_lists(r.json()["messages"]) + + got = admin_client.get(f"/api/chats/{created['id']}") + assert got.status_code == 200 + assert got.json()["message_count"] == 2 # bumped to the new array + _assert_incident_lists(got.json()["messages"]) + + +def test_put_rejects_over_cap_lists_and_leaves_row_unchanged( + client: TestClient, admin_client: TestClient +) -> None: + """The over-cap variants at the NEW boundaries (201 ``tools``, 451 + ``segments``, 101 ``suggestions``) each 422 with the framework's + standard body (phase 83 A2: no custom error copy, no route change) and + the row keeps its stored payload byte-for-byte — the rejected body + never reaches the JSONB.""" + created = client.post( + "/api/chats", json={"messages": _simple_conversation()} + ).json() + kept = _expect(_simple_conversation()) + + over_tools = _incident_brain() + over_tools["tools"] = over_tools["tools"] + [over_tools["tools"][0]] * 168 # 201 + assert len(over_tools["tools"]) == 201 + + over_segments = _incident_brain() + over_segments["segments"] = over_segments["segments"] + [ + {"kind": "thinking", "text": "one too many"} + ] * 400 # 451 + assert len(over_segments["segments"]) == 451 + + over_suggestions = _incident_brain() + over_suggestions["suggestions"] = [f"chip {i}" for i in range(101)] + + for bad, needle in ( + (over_tools, "tools"), + (over_segments, "segments"), + (over_suggestions, "suggestions"), + ): + r = client.put( + f"/api/chats/{created['id']}", + json={"messages": [_user("hi"), bad]}, + ) + assert r.status_code == 422, f"{needle}: expected 422, got {r.status_code}: {r.text}" + assert needle in json.dumps(r.json()), f"{needle}: 422 must name the offender" + + got = admin_client.get(f"/api/chats/{created['id']}") + assert got.status_code == 200 + assert got.json()["messages"] == kept + assert got.json()["message_count"] == 2 + + def test_put_rejects_oversized_message_and_leaves_row_unchanged( client: TestClient, admin_client: TestClient ) -> None: @@ -2137,10 +2309,11 @@ def test_shared_serve_includes_segments(admin_client: TestClient) -> None: assert "segments" not in body["messages"][0] -def test_segments_over_50_entries_rejected(admin_client: TestClient) -> None: - """``max_length=50`` (the phase-83 voice, the ``tools`` cap): 51 - entries 422 at the boundary — far past any realistic turn (the - round cap is 10 → ≤ 20 entries).""" +def test_segments_over_450_entries_rejected(admin_client: TestClient) -> None: + """``max_length=450`` (the phase-189 sizing — ≤ 200 tool entries + ≤ + ~50 thinking runs + margin, replaces the phase-83 "round cap is 10 → + ≤ 20 entries"): 451 entries 422 at the boundary while the + incident-shaped 51 (above) saves.""" r = admin_client.post( "/api/chats", json={ @@ -2148,7 +2321,7 @@ def test_segments_over_50_entries_rejected(admin_client: TestClient) -> None: { "who": "brain", "text": "ok", - "segments": [{"kind": "thinking", "text": "x"}] * 51, + "segments": [{"kind": "thinking", "text": "x"}] * 451, } ] }, diff --git a/tests/unit/test_save_chat_ui.py b/tests/unit/test_save_chat_ui.py index 0940cd4..bd239ab 100644 --- a/tests/unit/test_save_chat_ui.py +++ b/tests/unit/test_save_chat_ui.py @@ -41,6 +41,25 @@ so a silent regression is caught without a browser: slide-down + fade entry via ``.toast.is-visible``, and the reduced-motion override (transform dropped for BOTH states, the opacity fade kept); +* phase 189 task 02 (A2, the owner incident 2026-10-16): the LINKED + ``shareCurrentChat`` branch UPSERTS the current conversation (the + ``persistConversation`` PUT + the 404→recreate fallback + the 201 + re-link) BEFORE minting the share token, and an upsert failure ABORTS + the share (the neutral banner, no token, no toast) — a Share can never + publish a row the conversation has outgrown. The UNLINKED branch + (``POST /api/chats`` with ``share: true``) is byte-identical; +* phase 189 task 03 (A3/A4, the same incident): the ``persisting`` + double-fire guard no longer DROPS an overlapping save point — it sets + ``savePending`` and the in-flight upsert's ``finally`` re-runs + ``persistConversation()`` ONCE after it settles (``persisting = false`` + first, the flag cleared before the re-run — never concurrent, never + stacking). A failed save's one-line note is STICKY (``saveFailedSticky``, + cleared by the next SUCCESS along with ``#send-status``), and a + TRANSIENT failure (the network ``catch`` arm or a 5xx) schedules ONE + retry after 5 s behind the one-shot ``retryScheduled`` guard — a 4xx + NEVER retries (a 422 is deterministic; task 01's cap realign is the + remedy). The A2 contract's SHAPE is unchanged: the same note text, + still no error banner, still silent on success. * phase 55 task 05 (the action row, owner-locked A5): a single ``
`` in index.html wraps BOTH pills as its element children (DOM order New chat → Share), replacing the two @@ -203,7 +222,12 @@ def test_persist_conversation_is_headless_and_quiet() -> None: one-line #send-status note — on BOTH failure paths (non-ok HTTP and network) — with the "next save point retries" promise. The module-level `persisting` flag is the double-fire guard (released - in the finally — never stuck).""" + in the finally — never stuck). + + Phase 189 (A3) turns the DROPPED call into a PENDING one: the guard + arm still returns without issuing a second request (the upsert stays + one-at-a-time), it only remembers the save point for the ``finally`` + re-run pinned by test_persisting_overlap_sets_pending_and_reruns.""" js = _js() body = _fn(js, "persistConversation") assert "showErrorBanner" not in body, "A2: a failed auto-save never raises a banner" @@ -223,14 +247,177 @@ def test_persist_conversation_is_headless_and_quiet() -> None: assert re.search(r"^let persisting = false", js, re.M), ( "the persisting flag is module scope (save points can overlap)" ) - assert "if (persisting) return;" in body, "an in-flight upsert skips the second call" - assert "persisting = true;" in body + # Phase 189 (A3): the guard arm still SKIPS the call (no second + # upsert, no second request) — it only records the save point for the + # finally re-run pinned by test_persisting_overlap_sets_pending_and_reruns. + guard = body.find("if (persisting) {") + guard_return = body.find("return;", guard) + assert -1 < guard < guard_return < body.find("persisting = true;"), ( + "an in-flight upsert skips the second call" + ) + assert body.count("persisting = true;") == 1, "one upsert in flight, ever" finally_idx = body.rfind("finally") assert finally_idx != -1 and "persisting = false" in body[finally_idx:], ( "the flag is released in the finally — never stuck" ) +def test_persisting_overlap_sets_pending_and_reruns() -> None: + """Phase 189 (A3, the owner incident 2026-10-16): a save point that + arrives while an upsert is in flight is PENDING, not dropped. The + guard arm sets ``savePending`` and returns; a fresh run CONSUMES the + flag at the top of its body (so a re-run never schedules a second + one); the ``finally`` re-runs ``persistConversation()`` once, AFTER + ``persisting = false`` (the re-run is a fresh serialized call — + never concurrent with the upsert it trails) and clears the flag + before re-entering. Because the payload is built at CALL time, the + re-run carries the LATEST ``conversation`` — the done-time save a + still-running send-time save used to swallow lands with the brain + record.""" + js = _js() + body = _fn(js, "persistConversation") + # All three phase-189 flags live at module scope (state across the + # overlapping save points), beside the pre-existing `persisting`. + for flag in ("savePending", "saveFailedSticky", "retryScheduled"): + assert re.search(rf"^let {flag} = false;", js, re.M), ( + f"{flag} is module scope like `persisting`" + ) + noop = body.find("if (!conversation.length) return;") + first_fetch = body.find("await fetch(") + guard = body.find("if (persisting) {") + guard_return = body.find("return;", guard) + assert -1 < noop < guard < first_fetch, ( + "the overlap guard still sits before any request" + ) + set_pending = body.find("savePending = true;", guard) + assert guard < set_pending < guard_return, ( + "the in-flight arm marks the save point PENDING instead of dropping it" + ) + # A fresh run consumes the flag — before the payload is built, so the + # re-run is THIS run and cannot schedule a second re-run. + start = body.find("persisting = true;") + consume = body.find("savePending = false;", start) + assert -1 < start < consume < first_fetch, ( + "the run consumes the pending flag at the top of its body" + ) + payload = body.find("JSON.stringify({ messages: conversation })") + assert consume < payload, ( + "the payload is built at call time — the re-run reads the LATEST conversation" + ) + # The finally: release FIRST, then re-run once (never concurrent). + finally_idx = body.rfind("finally") + assert finally_idx > first_fetch, "the release lives in the finally" + tail = body[finally_idx:] + release = tail.find("persisting = false") + recheck = tail.find("if (savePending)", release) + rerun = tail.find("void persistConversation();", recheck) + assert -1 < release < recheck < rerun, ( + "the re-run starts only after the in-flight upsert fully settled" + ) + clear = tail.find("savePending = false;", recheck) + assert recheck < clear < rerun, "the flag is cleared before re-entering — one re-run, not two" + assert tail.count("void persistConversation();") == 1, "exactly one re-run site" + + +def test_failed_save_note_is_sticky_until_success() -> None: + """Phase 189 (A4): a failed auto-save's note does not vanish with the + next paint — both failure arms set ``saveFailedSticky`` after the + EXACT phase-55 note text (the A2 shape is untouched: the same + one-line #send-status note, still NO error banner, still no success + text), and the SUCCESS path clears the flag and resets + ``#send-status``. The clear sits on the success path only — after the + 201 link and the stale-banner clear, never in a failure arm.""" + js = _js() + body = _fn(js, "persistConversation") + note = "Couldn't save automatically — will try on the next message." + assert body.count(note) == 2, "the A2 note text is unchanged, on both failure paths" + assert "showErrorBanner" not in body, "A2 unchanged: a failed auto-save never raises a banner" + assert "Conversation saved." not in body, "A2 unchanged: success stays silent" + # Non-ok arm: note, then the sticky flag. + notok = body.find("if (!res.ok)") + first_note = body.find(note) + first_sticky = body.find("saveFailedSticky = true;", first_note) + assert -1 < notok < first_note < first_sticky, ( + "the non-ok arm notes AND marks the note sticky" + ) + # Network arm (the catch): the same note, the same flag. + catch_idx = body.find("} catch {") + second_note = body.find(note, catch_idx) + second_sticky = body.find("saveFailedSticky = true;", second_note) + assert catch_idx < second_note < second_sticky, ( + "the catch arm notes AND marks the note sticky" + ) + assert body.count("saveFailedSticky = true;") == 2, "both — and only both — failure arms" + # Success: the flag clears and the status line resets, on the success + # path only (after the 201 link + the stale-banner clear). + link = body.find("res.status === 201") + hide = body.find("staleBanner.hidden = true") + clear_guard = body.find("if (saveFailedSticky)") + assert -1 < link < hide < clear_guard < catch_idx, ( + "the stuck note clears on the success path, never on failure" + ) + block = body[clear_guard : body.find("}", clear_guard)] + assert "saveFailedSticky = false;" in block, "the flag resets with the note" + assert 'sendStatus.textContent = "";' in block, "the status line returns to empty" + + +def test_transient_failure_retries_once_after_5s() -> None: + """Phase 189 (A4): a TRANSIENT failure — the network ``catch`` arm or + a 5xx status — schedules ONE retry after 5 000 ms. The one-shot + ``retryScheduled`` guard is set BEFORE the ``setTimeout`` and reset + INSIDE the callback (overlapping failures never stack retries), and + the retry is a plain ``persistConversation()`` — the LATEST + conversation through the same guards (the pending re-run included).""" + js = _js() + body = _fn(js, "persistConversation") + assert body.count("setTimeout(") == 2, "one retry site per transient arm — no more" + for arm_start in (body.find("if (!res.ok)"), body.find("} catch {")): + arm_end = body.find("} finally {", arm_start) + assert -1 < arm_start < arm_end + timeout = body.find("setTimeout(", arm_start) + assert arm_start < timeout < arm_end, "the arm schedules the retry" + guard = body.find("retryScheduled = true;", arm_start) + assert arm_start < guard < timeout, ( + "the one-shot guard is set BEFORE the setTimeout" + ) + delay = body.find(", 5000);", timeout) + assert timeout < delay < arm_end, "the retry waits 5 000 ms" + callback = body[timeout:delay] + assert "retryScheduled = false;" in callback, "the guard resets INSIDE the callback" + assert "void persistConversation();" in callback, "the retry is a plain auto-save" + # The 5xx condition is the ONLY status gate on the non-ok arm. + notok_arm = body[body.find("if (!res.ok)") : body.find("} catch {")] + assert "res.status >= 500" in notok_arm + + +def test_4xx_never_retries() -> None: + """Phase 189 (A4): a 4xx is DETERMINISTIC — a 422 (the incident: the + 22-source payload over the old 20-cap) would 422 again, and task 01's + cap realign is the remedy. The non-ok arm's only scheduling site sits + behind ``res.status >= 500``, so every other non-ok status (404 aside + — the recreate fallback — 409, 422, 400) notes and returns with no + timer; the arm still marks the note sticky (it stays visible).""" + js = _js() + body = _fn(js, "persistConversation") + notok = body.find("if (!res.ok)") + catch_idx = body.find("} catch {") + arm = body[notok:catch_idx] + assert arm.count("setTimeout(") == 1, "the non-ok arm has exactly ONE scheduling site" + gate = arm.find("if (res.status >= 500 && !retryScheduled) {") + timeout = arm.find("setTimeout(") + assert -1 < gate < timeout, "the only retry is gated on a 5xx status" + # Nothing else in the arm keys off a 4xx, and nothing after the gate's + # block runs before the return. + for forbidden in ("status === 4", "status >= 4", "status < 500", "retryScheduled"): + assert forbidden not in arm[timeout + len("setTimeout(") :].replace( + "retryScheduled = false;", "" + ), f"no extra scheduling branch keyed on {forbidden!r}" + assert arm.find("saveFailedSticky = true;") < arm.find("return;"), ( + "a 4xx still leaves the note visible — it just never retries" + ) + assert body.count("setTimeout(") == 2, "the catch arm is the only other scheduling site" + + def test_auto_save_wired_to_the_save_points() -> None: """The headless helper is referenced from the save points: the user send (runTurn's ``!reask`` block — after the localStorage @@ -489,7 +676,14 @@ def test_share_current_chat_save_then_share_branch() -> None: again" — the write surface is public, phase 55 task 01, so a 403 is no longer a sign-in problem for a guest); network → the reachable? banner. The double-click guard releases in the finally — - never stale.""" + never stale. + + Phase 189 (task 02) moves the ORDERING pins to the create-with-share + payload: the linked branch now ALSO holds a POST (its 404→recreate + fallback), so "the first /api/chats POST is the unlinked arm" no + longer reads — ``share: true`` is what identifies that arm (the + upsert-before-mint ordering itself is pinned by + test_share_linked_branch_upserts_before_the_share_mint).""" js = _js() body = _fn(js, "shareCurrentChat") # No-op first: nothing to share → live-region line, no fetch. @@ -506,7 +700,7 @@ def test_share_current_chat_save_then_share_branch() -> None: assert 'JSON.stringify({ messages: conversation, share: true })' in body, ( "the create-with-share payload — the server sets the token in the same commit" ) - post_idx = body.find('fetch("/api/chats", {') + post_idx = body.find('JSON.stringify({ messages: conversation, share: true })') assert -1 < share_fetch < post_idx, "the linked branch precedes the unlinked fallback" created_idx = body.find("currentChatId = String(created.id)", post_idx) assert created_idx != -1, "one action saved AND shared: the conversation links to the row" @@ -529,8 +723,10 @@ def test_share_current_chat_save_then_share_branch() -> None: # is public); the network banner keeps its own line. assert 'showErrorBanner("Couldn\'t share the conversation — is the app reachable?")' in body neutral = "Couldn't share the conversation — try again." - assert body.count(neutral) == 2, ( - "both the linked and the unlinked branch carry the neutral non-ok banner" + assert body.count(neutral) == 3, ( + "three non-ok arms carry the SAME neutral banner: the phase-189 " + "upsert abort, the linked share mint, and the unlinked create " + "(no sign-in wording on any of them)" ) assert "signed in" not in body, "no sign-in wording left in the share handler (task 03)" # The double-click guard releases on EVERY outcome. @@ -571,6 +767,121 @@ def test_share_current_chat_save_then_share_branch() -> None: assert "document.createRange()" in sel and "selectNodeContents(el)" in sel +def test_share_linked_branch_upserts_before_the_share_mint() -> None: + """Phase 189 (task 02, A2): the LINKED branch of ``shareCurrentChat`` + upserts the CURRENT conversation BEFORE minting the token — the owner + incident (2026-10-16: a row holding only the question because the + done-time auto-save 422'd into the phase-55 quiet failure; the Share + minted on that stale row and published a truncated snapshot) is the + why. The upsert carries ``persistConversation``'s semantics: the + ``{ messages: conversation }`` payload + JSON headers, the PUT to + /api/chats/, the 404→recreate fallback (unlink, then POST + /api/chats with the SAME body/headers) and the 201 re-link. The + share mint is the LAST request of the branch (string-ordering pin — + no browser).""" + js = _js() + body = _fn(js, "shareCurrentChat") + branch = body.find("if (currentChatId) {") + else_idx = body.find("} else {") + assert -1 < branch < else_idx + arm = body[branch:else_idx] + put = arm.find('`/api/chats/${currentChatId}`') + mint = arm.find('`/api/chats/${currentChatId}/share`') + assert -1 < put < mint, "the upsert PUT precedes the share mint (save-first)" + # The persistConversation-shaped payload + headers, byte-identical to + # the auto-save's (ONE upsert shape, not two). + persist = _fn(js, "persistConversation") + assert arm.find('const body = JSON.stringify({ messages: conversation });') < put + assert 'const body = JSON.stringify({ messages: conversation });' in persist, ( + "the share upsert reuses the auto-save's payload shape" + ) + assert 'const headers = { "Content-Type": "application/json" };' in arm + assert 'method: "PUT"' in arm, "the linked branch PUTs the linked row" + # The 404→recreate fallback (the same stale-link rule as the auto-save). + notfound = arm.find("res.status === 404") + assert put < notfound < mint, "a PUT 404 (row deleted on History) is handled" + assert "currentChatId = null" in arm[notfound:mint], "the stale link is dropped" + fallback = arm.find('fetch("/api/chats", { method: "POST", headers, body })') + assert notfound < fallback < mint, ( + "the 404 retries as a create with the SAME body/headers" + ) + relink = arm.find("currentChatId = String((await res.json()).id)") + assert fallback < relink < mint, ( + "the 201 re-links the conversation to the recreated row" + ) + # The mint rides AFTER the upsert settled — and it is the branch's + # LAST request (nothing publishes before the row catches up). + assert arm.count("await fetch(") == 3, "PUT, the recreate POST, then the mint" + last_fetch = arm[arm.rfind("await fetch(") :].split("\n")[0] + assert "/share`" in last_fetch, "the share mint is the linked branch's LAST fetch" + # The docstring says WHY (the incident), not just what. + doc = js[: js.find("async function shareCurrentChat()")] + doc = doc[doc.rfind("/*"):] + assert "UPSERTS the current conversation BEFORE minting the token" in doc + assert "truncated snapshot" in doc, "the incident rationale is in the docstring" + + +def test_share_aborts_on_upsert_failure() -> None: + """Phase 189 (task 02, A2): an upsert FAILURE aborts the share. The + failure arm that sits BETWEEN the upsert and the mint raises the + EXACT neutral banner text and ``return``s — no token minted, no + toast, no link handed to the owner (the mint line and every success + path come after it). The double-click guard still releases in the + finally, and a network throw still lands on the reachable? banner + (the outer catch, unchanged).""" + js = _js() + body = _fn(js, "shareCurrentChat") + branch = body.find("if (currentChatId) {") + arm = body[branch : body.find("} else {")] + mint = arm.find('`/api/chats/${currentChatId}/share`') + notok = arm.find("if (!res.ok)") + banner = arm.find( + 'showErrorBanner("Couldn\'t share the conversation — try again.")', notok + ) + assert -1 < notok < banner < mint, ( + "the non-ok upsert raises the neutral banner BEFORE the mint line" + ) + ret = arm.find("return;", banner) + assert notok < ret < mint, "the abort returns — the mint is never reached" + abort = arm[notok:ret] + assert "showToast" not in abort, "the abort toasts nothing (the toast stays success-only)" + assert "share_url" not in abort, "no link is read or handed out on the abort" + assert "copyShareLinkWithFallback" not in abort, "no copy on the abort" + assert "showToast" not in arm[:mint], "no toast anywhere before the mint" + # Both non-ok arms of the linked branch carry the SAME neutral copy: + # the upsert abort and the mint's own failure. + assert arm.count('showErrorBanner("Couldn\'t share the conversation — try again.")') == 2, ( + "the upsert abort + the mint's non-ok arm — the neutral copy, twice in the branch" + ) + finally_idx = body.rfind("finally") + assert finally_idx != -1 and "shareBtn.disabled = false" in body[finally_idx:], ( + "the button releases even on the abort — never stale" + ) + + +def test_share_unlinked_branch_unchanged() -> None: + """Phase 189 (task 02): the UNLINKED branch is byte-identical — one + ``POST /api/chats`` with ``{ messages: conversation, share: true }`` + (the server sets the 128-bit token in the same commit — one action + saves AND shares, owner-locked) and the 201 re-link. It carries no + PUT and no separate share mint: the save-first upsert belongs to the + linked arm only (its POST already saves the CURRENT conversation).""" + js = _js() + body = _fn(js, "shareCurrentChat") + else_idx = body.find("} else {") + copy_idx = body.find("copyShareLinkWithFallback(absoluteShareUrl(shareUrl))") + assert -1 < else_idx < copy_idx + arm = body[else_idx:copy_idx] + assert 'fetch("/api/chats", {' in arm + assert 'method: "POST"' in arm + assert 'JSON.stringify({ messages: conversation, share: true })' in arm + assert "currentChatId = String(created.id)" in arm, "the 201 re-links (saved AND shared)" + assert 'shareUrl = created.share_url' in arm, "the token comes from the same commit" + assert 'method: "PUT"' not in arm, "the unlinked arm never PUTs — unchanged" + assert "/share" not in arm, "the unlinked arm never mints a separate token" + assert "share: true" not in body[:else_idx], "share: true stays out of the linked arm" + + def test_share_button_has_no_reveal_gate() -> None: """Phase 55 task 03: the pill is VISIBLE TO EVERY VISITOR — app.js queries #share-chat-btn and binds the click to shareCurrentChat, but diff --git a/tests/unit/test_schemas.py b/tests/unit/test_schemas.py index 6e02e89..e37163a 100644 --- a/tests/unit/test_schemas.py +++ b/tests/unit/test_schemas.py @@ -21,6 +21,15 @@ boundaries as the saved-chat mirror, and the 422 ``detail`` SHAPE a cap rejection produces is pinned as the server-side half of the phase-164 serializer contract (a LIST OF OBJECTS — the live ``[object Object]`` report). + +Phase 189 (task 01, A1 — the owner incident of 2026-10-16) moves the five +LIST caps from the round-cap-10 era to the real agent configuration +(``sources``/``related``/``tools`` 200, ``suggestions`` 100, ``segments`` +450): the cited tier is the agent's READ docs (``merge_cited_docs``, +phase 119), so its size follows ``BOR_AGENT_MAX_ROUNDS`` (production 50, +the validator unbounded above), and a 22-source done frame used to 422 +its own auto-save. The character caps are UNCHANGED; the incident payload +(22 sources / 33 tools / 40 segments) joins the round-trip pins. """ from __future__ import annotations @@ -54,10 +63,26 @@ from app.schemas import ( #: 600 000 ≈ a full-cap answer at ``BOR_MAX_OUTPUT_TOKENS=131_072`` #: (≈ 524 300 chars at ~4 chars/token) with margin. TEXT_CAP = 600_000 -SOURCES_CAP = 20 -SUGGESTIONS_CAP = 50 +#: Phase 189 (A1, owner incident 2026-10-16): the list caps move from the +#: round-cap-10 era to the real configuration. The cited tier is the +#: agent's READ DOCS since phase 119 (``merge_cited_docs``), so its size +#: follows ``agent_max_rounds`` (production ``BOR_AGENT_MAX_ROUNDS=50``, +#: the validator unbounded above) — 200 is 4× that cap. A 22-source done +#: frame used to 422 its own done-time auto-save (the incident). +SOURCES_CAP = 200 +#: The related-doc tier — the same ``SourceRef`` family, the same cap +#: (``related_max_docs`` is owner-tunable with no upper bound). +RELATED_CAP = 200 +#: 20× the ``suggested_docs`` default of 5 (also tunable unbounded above). +SUGGESTIONS_CAP = 100 CHIP_CAP = 200 -TOOLS_CAP = 50 +#: 4× the production round cap — a round is ONE model request and +#: one-or-more tool calls per round are legal. +TOOLS_CAP = 200 +#: ≤ the ``tools`` cap (200 tool entries) + ≤ ~50 thinking runs (one per +#: round at the production cap) + margin — the phase-158 "50 matches the +#: tools cap" sizing is retired with the round-cap-10 assumption. +SEGMENTS_CAP = 450 SOURCE_CAP = 120 # documents.source String(120) PATH_CAP = 1000 # documents.path String(1000) TITLE_CAP = 500 # documents.title String(500) @@ -94,6 +119,26 @@ def _tool_call() -> dict: } +def _thinking_segment(text: str = "a thinking run") -> dict: + """One contiguous thinking run (the phase-158 timeline entry).""" + return {"kind": "thinking", "text": text} + + +def _tool_segment(index: int) -> dict: + """One tool entry in the timeline — the exact ToolCall mirror, the + phase-146 scope ABSENT (a plain read, the omission serializer).""" + return { + "kind": "tool", + "name": "read", + "argument": f"Homelab/doc{index}.md", + "truncated": False, + "chars_shown": None, + "chars_total": None, + "offset": None, + "limit": None, + } + + def _user_message(text: str = "How did I install k3s?") -> ChatMessage: return ChatMessage.model_validate({"who": "user", "text": text}) @@ -142,7 +187,9 @@ def test_chat_message_thinking_none_still_valid() -> None: # --------------------------------------------------------------------------- -# ChatMessage.sources / .suggestions / .tools +# ChatMessage.sources / .related / .suggestions / .tools / .segments — the +# phase-189 list caps (the character caps above keep their phase-83/164 +# numbers; these five follow the agent's real round configuration) # --------------------------------------------------------------------------- @@ -216,6 +263,56 @@ def test_chat_message_tools_one_over_cap_rejects() -> None: _failed_loc(exc.value, "tools") +def test_chat_message_related_at_cap_validates() -> None: + """Phase 189 (A1): the related-doc tier carries the SAME + ``SourceRef``-family cap as ``sources`` (200) — ``related_max_docs`` + is owner-tunable with no upper bound, so the tier must clear the same + worst case or a long turn 422s its own auto-save.""" + msg = ChatMessage.model_validate( + {"who": "brain", "text": "ok", "related": [_source_ref() for _ in range(RELATED_CAP)]} + ) + assert len(msg.related or []) == RELATED_CAP + assert RELATED_CAP == SOURCES_CAP # one SourceRef-family cap + + +def test_chat_message_related_one_over_cap_rejects() -> None: + with pytest.raises(ValidationError) as exc: + ChatMessage.model_validate( + { + "who": "brain", + "text": "ok", + "related": [_source_ref() for _ in range(RELATED_CAP + 1)], + } + ) + _failed_loc(exc.value, "related") + + +def test_chat_message_segments_at_cap_validates() -> None: + """The list cap at the phase-189 number (450): the worst-case turn at + the production round cap (200 tool entries + ~50 thinking runs) fits + with margin.""" + msg = ChatMessage.model_validate( + { + "who": "brain", + "text": "ok", + "segments": [_thinking_segment() for _ in range(SEGMENTS_CAP)], + } + ) + assert len(msg.segments or []) == SEGMENTS_CAP + + +def test_chat_message_segments_one_over_cap_rejects() -> None: + with pytest.raises(ValidationError) as exc: + ChatMessage.model_validate( + { + "who": "brain", + "text": "ok", + "segments": [_thinking_segment() for _ in range(SEGMENTS_CAP + 1)], + } + ) + _failed_loc(exc.value, "segments") + + # --------------------------------------------------------------------------- # ChatRequest.message (the 4,000-char question cap — phase 104, A3) # --------------------------------------------------------------------------- @@ -701,6 +798,66 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None: assert dumped["share"] is False +def test_incident_long_turn_payload_round_trips() -> None: + """Phase 189 (task 01, owner incident 2026-10-16) EXTENDS the + realistic-payload family with the shape that actually broke: one long + research turn's brain record carrying 22 ``sources`` (the cited tier + is the agent's READ docs since phase 119), 33 ``tools`` and 40 + ``segments`` (the 33 tool entries + one thinking entry per contiguous + thinking run — 7 runs). At the pre-phase caps (``sources`` 20, + ``tools``/``segments`` 50) this record 422'd its own done-time + auto-save, the phase-55 A2 quiet contract swallowed the failure, and + the shared row kept ONLY the question. It now validates and + ``model_dump()``s byte-identically — the omission rules untouched (the + tool entries carry no ``path`` key, so none comes back).""" + sources = [ + {"source": "Homelab", "path": f"notes/doc{i}.md", "title": f"Doc {i}"} + for i in range(22) + ] + # 33 tool entries interleaved with 7 thinking runs (buildSegments' + # chronological shape: one thinking entry per contiguous run). + segments: list[dict] = [] + tools: list[dict] = [] + for run in range(7): + segments.append(_thinking_segment(f"thinking run {run}")) + for _ in range(5 if run < 3 else 4): # 3*5 + 4*4 = 31 + 2 = 33 + segments.append(_tool_segment(len(tools))) + tools.append(_tool_call()) + while len(tools) < 33: # the last run's remaining calls + segments.append(_tool_segment(len(tools))) + tools.append(_tool_call()) + assert len(sources) == 22 and len(tools) == 33 and len(segments) == 40 + assert sum(1 for s in segments if s["kind"] == "thinking") == 7 + + record = { + "who": "brain", + "text": "The whole investigation, across 22 documents.", + "sources": sources, + "related": sources[:2], + "deflected": False, + "suggestions": [f"Show doc {i}" for i in range(5)], + "thinking": "".join(f"run {r} " for r in range(7)), + "tools": tools, + "segments": segments, + "stopped": None, + "failed": None, + "error": None, + "budget_exhausted": None, + } + + saved = SavedChatCreate.model_validate( + {"title": "long research turn", "messages": [record]} + ) + dumped = saved.model_dump()["messages"][0] + assert dumped["sources"] == sources + assert dumped["tools"] == tools + assert dumped["segments"] == segments + # Every entry round-trips through the discriminated union unchanged, + # and the scoped-grep key stays ABSENT (never null) on the plain reads. + assert all("path" not in s for s in dumped["segments"] if s["kind"] == "tool") + assert ChatMessage.model_validate(record).model_dump() == dumped + + def test_realistic_payload_round_trips_through_update_model() -> None: """Same record re-Saved through ``SavedChatUpdate`` (the re-Save upsert surface) — identical stored shape.""" @@ -1064,8 +1221,9 @@ def test_segments_tool_entry_mirrors_toolcall_shape() -> None: def test_segments_caps_mirrored_not_weakened() -> None: """The ToolCall caps bind on the segment (mirrored, not weakened): ``name`` ≤ 100, ``argument`` ≤ 2 000, ``path`` ≤ 2 000, - ``offset``/``limit`` ≥ 1, the list ≤ 50 (the phase-83 voice — the - ``tools`` cap).""" + ``offset``/``limit`` ≥ 1, and the list cap is the phase-189 + ``SEGMENTS_CAP`` (450) — NOT the ``tools`` cap any more (the two + separated when the round-cap-10 sizing was retired).""" bad: list[dict] = [ {"kind": "tool", "name": "n" * (NAME_CAP + 1), "argument": None}, {"kind": "tool", "name": "grep", "argument": "a" * (ARGUMENT_CAP + 1)}, @@ -1084,7 +1242,7 @@ def test_segments_caps_mirrored_not_weakened() -> None: { "who": "brain", "text": "ok", - "segments": [{"kind": "thinking", "text": "x"}] * (TOOLS_CAP + 1), + "segments": [{"kind": "thinking", "text": "x"}] * (SEGMENTS_CAP + 1), } ) _failed_loc(exc.value, "segments") @@ -1205,9 +1363,10 @@ def test_saved_record_answer_text_round_trips_and_stays_optional() -> None: def test_saved_record_answer_text_list_is_capped() -> None: - """The list cap (the phase-83 SEC-05 style, the ``tools`` / - ``segments`` number): 51 blocks reject, 50 fit — far past the - ``show_text_max_blocks`` default of 5.""" + """The list cap (the phase-83 SEC-05 style): 51 blocks reject, 50 fit + — far past the ``show_text_max_blocks`` default of 5. The cap stays 50 + through phase 189 (it follows ``show_text_max_blocks``, not the round + count).""" blocks = [_verbatim_block(locator=f"Homelab/a{i}.md") for i in range(51)] with pytest.raises(ValidationError): ChatMessage.model_validate(