Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Details
Build and Push Containers / build-and-push-db (push) Successful in 14s
Details

Browse Source 
## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
ducoterra
2026-10-11 23:38:18 -04:00
parent c6223c93dc
commit 149e753952
45 changed files with 1947 additions and 544 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/178_split_thinking_levels/00_phase.md → .agents/phases/complete/178_split_thinking_levels/00_phase.md
View File
.agents/phases/todo/178_split_thinking_levels/01_settings_catalog_env.md → .agents/phases/complete/178_split_thinking_levels/01_settings_catalog_env.md
View File
.agents/phases/todo/178_split_thinking_levels/02_admin_row_contract.md → .agents/phases/complete/178_split_thinking_levels/02_admin_row_contract.md
View File
.agents/phases/todo/178_split_thinking_levels/03_llm_extra_body_split.md → .agents/phases/complete/178_split_thinking_levels/03_llm_extra_body_split.md
View File
.agents/phases/todo/178_split_thinking_levels/04_per_turn_log_field.md → .agents/phases/complete/178_split_thinking_levels/04_per_turn_log_field.md
View File
.agents/phases/todo/178_split_thinking_levels/05_e2e_admin_settings_rows.md → .agents/phases/complete/178_split_thinking_levels/05_e2e_admin_settings_rows.md
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__00_phase.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__00_phase.a1.md
+20
View File
@@ -0,0 +1,20 @@
## Phase 178 — final verification pass: 100% green
**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation
**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.
**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.
**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).
**Next pending phase:** none — 178 is the last phase in `todo/`.
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__00_phase.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__01_settings_catalog_env.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__01_settings_catalog_env.a1.md
+22
View File
@@ -0,0 +1,22 @@
All complete and verified. `image-1.png` is a pre-existing untracked file (created Oct 11 09:59, before this task — not touched by me).
**Report**
**Implemented (phase 178, task 01):**
- `app/config.py`: `llm_thinking_level` → `llm_chat_thinking_level` (default `"low"`), new `llm_summary_thinking_level` (default `""`), house docstrings, no validator
- `app/core/settings_catalog.py`: one row → two rows ("Chat thinking level" / "Summary thinking level", `LLM (aipi)`)
- `.env.example`: `BOR_LLM_CHAT_THINKING_LEVEL=low` + `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty)
- `app/rag/llm.py` / `app/api/chat.py`: mechanical rename (docstrings, `_thinking_extra_body`, per-turn log arg) — both LLM paths still ride the chat level, byte-identical behavior
- Tests renamed/extended in place: `test_config.py` (3→6 tests), `test_settings_catalog.py` (both-rows test, completeness pin intact), `test_llm_client.py`, `test_chat_gate.py`, `test_admin_settings_api.py`, `test_admin_settings_live.py`, e2e `test_thinking_level.py` (incl. the now-correct "Chat thinking level" label pin)
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (chat `low`/summary `""` defaults + env overrides)
- `uv run pytest --cov=app --cov-report=term-missing` → 4164 passed, 99% coverage (>90%)
- `uv run ruff check . && uv run pyright` → clean (0 errors, 0 warnings)
- `grep -rn "llm_thinking_level\b|BOR_LLM_THINKING_LEVEL\b" app/ tests/ .env.example README.md` → nothing (old name gone everywhere)
**Docs:** README env table updated (one `BOR_LLM_THINKING_LEVEL` row → the two renamed/split rows); no AGENTS.md impact.
**Decisions:** A1/A2 honored — rename orphans stored old-key overrides (no migration); summary default is empty = absent parameter, never literal "none". Wire split deferred to task 03 as designed.
**Next pending task:** `02_admin_row_contract.md`
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__01_settings_catalog_env.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__01_settings_catalog_env.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
............................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__02_admin_row_contract.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__02_admin_row_contract.a1.md
+19
View File
@@ -0,0 +1,19 @@
**Task 02 complete — report**
**Implemented (test-only, `app/` untouched):**
- `tests/integration/test_admin_settings_api.py`: added `test_llm_summary_thinking_level_row_contract` — summary row in `LLM (aipi)` (label "Summary thinking level", `str`, `BOR_LLM_SUMMARY_THINKING_LEVEL`, effective `""` with `delenv`), PUT raw storage (`"high"`), empty storable as the off-switch, DELETE reset to `""`, anonymous 403 pin, and the independence pin (summary `"high"` → chat still `"low"`, verified via `get_settings()` and the GET item).
- `tests/integration/test_admin_settings_live.py`: added `test_llm_summary_thinking_level_live_apply_no_restart` — `delenv` → `""`, `_save "high"` → live `high` (no restart), `_save ""` → `""`, DELETE → `""`; also updated the module docstring to name the summary twin.
- Removed a stale `__pycache__/test_thinking_level.*.pyc` artifact so the `llm_thinking_level\b` grep pin is clean.
**Results:**
- `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v` → 24 passed
- `… -k "thinking"` on each file → 2 passed each (4 total: both row contracts + both live-apply twins)
- `uv run pytest --cov=app --cov-report=term-missing` → 4166 passed, 0 failed, coverage 99% (>90%)
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
- `grep -rn "llm_thinking_level\b" tests/ app/ .env.example` → empty
**Docs:** no doc impact (README already current from task 01; no user-visible changes in a test-only task).
**Notable:** the independence pin `delenv`s both env vars for a machine-independent base (`.env` sets no thinking vars, so `source == "default"` is deterministic).
**Next pending task:** `.agents/phases/todo/178_split_thinking_levels/03_llm_extra_body_split.md`
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__02_admin_row_contract.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__02_admin_row_contract.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.............................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__03_llm_extra_body_split.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__03_llm_extra_body_split.a1.md
+21
View File
@@ -0,0 +1,21 @@
All complete and green. The README's env-var table (L1164–1165, written in task 01) already describes the split accurately — no doc changes needed.
## Report
**Implemented (task 03 — the wire split):**
- `_thinking_extra_body` is now a pure function of the level string (no `Settings` read); trimmed non-empty → `{"reasoning_effort": <value>}`, empty/whitespace → `None` (parameter absent, never a literal "none")
- One-shot `chat()` path (via `_chat_once`) passes `llm_summary_thinking_level` — document summaries, folder summaries, KB overview, `check_models` probe (off by default, no special-casing)
- `chat_stream()` passes `llm_chat_thinking_level` (default `low` unchanged); embeddings path untouched
- Module docstring finalized with the split wording incl. the probe consequence
- `tests/unit/test_llm_client.py`: `chat()` family re-pointed at the summary kwarg (`high` / trim / `""` / whitespace-absent, full pre-172 kwargs shape), `chat_stream()` family kept, two cross-pins (defaults-inverted: chat `high` + summary `""` → only the stream carries it; defaults: stream `low`, one-shot no `extra_body`); embeddings test untouched
**Gates (all 100% green):**
- `uv run pytest tests/unit/test_llm_client.py -k "thinking" -v` → 15 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 4169 passed, 0 failed; `app/` coverage **99%** (>90%)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
**Docs:** no impact — README env table (task 01) already matches the new behavior; AGENTS.md has no thinking-level lines.
**Decisions:** one client-per-cross-pin helper (`_make_both_client`) since the fake `completions` serves both `create()` shapes; test names carry `thinking` so the task's `-k "thinking"` gate selects the whole family.
**Next pending task:** `04_per_turn_log_field.md`
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__03_llm_extra_body_split.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__03_llm_extra_body_split.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__04_per_turn_log_field.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__04_per_turn_log_field.a1.md
+20
View File
@@ -0,0 +1,20 @@
All gates pass. Done.
## Report — Task 04: per-turn log line pinned to the chat level
**Implemented:**
- `app/api/chat.py` docstring finalized: per-turn line records `thinking=<level>` (effective `BOR_LLM_CHAT_THINKING_LEVEL`, trimmed, `-` when empty) before `thinking_chars` (phase 172, A6; phase 178, A5 — the **chat** level; the one-shot summary level never appears on the line).
- Inline comment at the log site finalized: "phase 178 (A5): the effective chat thinking level, trimmed, `-` when empty — the level this turn ran at; the summary level is a one-shot concern and gains no field here." No functional change to the line.
- `tests/unit/test_chat_gate.py`: docstring updated to name `BOR_LLM_CHAT_THINKING_LEVEL` / chat level (phase 178, A5); added the **cross-pin** (chat `low` + summary `high` → `thinking=low`, `thinking=high` absent) and the **inverse off-pin** (chat `""` + summary `high` → `thinking=-`, `thinking=high` absent).
**Test / lint / coverage (all green):**
- `uv run pytest tests/unit/test_chat_gate.py -k "log" -v` → 1 passed (full log-line family: default `low`, `high` override, `""`→`-`, trim→`-`, field order, both cross-pins)
- `grep -n "BOR_LLM_SUMMARY_THINKING_LEVEL" app/api/chat.py` → no output (exit 1)
- `uv run pytest -q` → exit 0 (100% green); `uv run pytest --cov=app` → `app/` TOTAL **99%** (>90%)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
**Docs:** no doc impact (docstring/comment + tests only; no new env var, command, feature, or API).
**Deviation:** the task's work item 2 specified docstring wording containing `BOR_LLM_SUMMARY_THINKING_LEVEL`, but the completion criterion requires that grep to return nothing in `app/api/chat.py`. The verifiable grep gate was prioritized — the docstring reads "the one-shot **summary level** never appears on the line" (identical meaning, name omitted).
**Next pending task:** `05_e2e_admin_settings_rows.md`
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__04_per_turn_log_field.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__04_per_turn_log_field.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__05_e2e_admin_settings_rows.a1.err
View File
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__05_e2e_admin_settings_rows.a1.md
+8
View File
@@ -0,0 +1,8 @@
**Task 05 complete — report**
- **Implemented:** `tests/e2e/test_thinking_level.py` → `test_split_thinking_levels.py` (phase 178 Playwright suite, 3 tests): both rows render free-form (`type="text"`, `LLM (aipi)` band, labels/env-var displays, chat `low`/summary empty with `env` badges, old `llm_thinking_level` row absent, "like chat model" reference pair); per-key independent save (`think-soft`/`high`) with catalogue + raw `app_settings` pins and reload persistence (DB badges, no restart); per-row reset to env defaults (summary empty / chat `low`, env badges, house toasts, no raw rows). Env pins: `BOR_SETTINGS_DB=1`, `BOR_LLM_CHAT_THINKING_LEVEL=low`, `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty SET), `BOR_LLM_CHAT_MODEL`.
- **Notable fix:** the suite's direct-SQL `TRUNCATE app_settings` does not invalidate the app's `get_settings()` lru-cache (the admin API is the only invalidating mutation surface) — test 3 typed the same values test 2 saved, saw the stale value as "served", registered no dirty state. Fixed in-suite: `_clean` now follows each truncate with the app's own `DELETE /api/admin/settings` (zero rows, cache rebuilds env-only). No app-code change. Also pinned the reset-rows settle (DB badge before `to_be_enabled`) to kill the save-click race.
- **Gates (all green):** `uv run pytest` → 4169 passed; `uv run pytest --cov=app --cov-report=term-missing` → 4169 passed, TOTAL **99%** (>90%); `uv run ruff check . && uv run pyright` → 0 errors; `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation (run ×3, stable); regression `test_admin_settings_panel.py` → 12 passed, `test_admin_settings_layout.py` → 5 passed.
- **Docs:** README already updated by tasks 01–04 (both env vars documented, no stale `BOR_LLM_THINKING_LEVEL` refs); no AGENTS.md impact.
- **Deviations (harness-owned):** no commit / no phase-dir move made — left in the working tree per my rules. `TODO.md` still holds one *external* owner item added 22:40 today ("long git clone freezes the server?") — predates/outsides this phase, L3 is cleared and not re-added; left the item untouched. Stray `image-1.png` (repo root, 09:59, pre-phase) left alone.
- **Next pending task:** none — task 05 was the last of phase 178.
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__05_e2e_admin_settings_rows.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/178_split_thinking_levels/178_split_thinking_levels__05_e2e_admin_settings_rows.a1.validate
+136
View File
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.env.example
+2 -1
View File
@@ -58,7 +58,8 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
BOR_LLM_CHAT_MODEL=turbo
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
BOR_LLM_CHAT_THINKING_LEVEL=low # the chat turn's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
BOR_LLM_SUMMARY_THINKING_LEVEL= # the one-shot summary thinking level (document/folder summaries, KB overview); empty (default) = the parameter is not sent
# BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off
# BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120)
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls)
README.md
+2 -1
View File
@@ -1161,7 +1161,8 @@ startup (the DB rows shadow the env, `.env` is never rewritten).
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` |
| `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model |
| `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries |
| `BOR_LLM_THINKING_LEVEL` | `low` | The model's thinking level (pi-style, free-form — no allowlist); rides every chat-completions request as `reasoning_effort`; empty = the parameter is not sent |
| `BOR_LLM_CHAT_THINKING_LEVEL` | `low` | The chat turn's thinking level (pi-style, free-form — no allowlist); rides the streaming chat-completions requests as `reasoning_effort`; empty = the parameter is not sent |
| `BOR_LLM_SUMMARY_THINKING_LEVEL` | *(empty)* | The one-shot summary thinking level (document/folder summaries, KB overview, the model probe); rides the one-shot `chat()` calls as `reasoning_effort`; empty (default) = the parameter is not sent |
| `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) |
| `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off |
| `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries |
TODO.md
+2
View File
@@ -1 +1,3 @@
# TODO
- [ ] A long git clone seems to freeze the server? Investigate to find out if this is true.
app/api/chat.py
+8 -3
View File
@@ -15,8 +15,9 @@ turn's thinking is counted in the per-turn log line
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
``thinking`` frames (the pieces are still counted). The per-turn log
line records ``thinking=<level>`` (the effective
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6).
``BOR_LLM_CHAT_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6; phase 178, A5 — the **chat**
level; the one-shot summary level never appears on the line).
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
@@ -1286,7 +1287,11 @@ async def chat(
settings.relevance_threshold,
plan.deflected,
source_paths,
settings.llm_thinking_level.strip() or "-", # phase 172 (A6)
# phase 178 (A5): the effective chat thinking
# level, trimmed, `-` when empty — the level this
# turn ran at; the summary level is a one-shot
# concern and gains no field here.
settings.llm_chat_thinking_level.strip() or "-",
thinking_chars,
holder.tool_calls,
holder.rounds,
app/config.py
+15 -6
View File
@@ -105,14 +105,23 @@ class Settings(BaseSettings):
#: Served by the same OpenAI-compatible endpoint — no new model
#: management. Called via ``LLMClient.chat()``.
llm_summary_model: str = "lite"
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
#: The chat turn's thinking level (phase 178,
#: ``BOR_LLM_CHAT_THINKING_LEVEL``; the phase-172 knob, renamed) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to
#: aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
#: parameter is not sent at all.
llm_thinking_level: str = "low"
#: truth). Default ``low``. Rides the streaming chat-completions
#: requests to aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py`` ``chat_stream``); the EMPTY value is the off
#: switch — the parameter is not sent at all.
llm_chat_thinking_level: str = "low"
#: The one-shot summary thinking level (phase 178,
#: ``BOR_LLM_SUMMARY_THINKING_LEVEL``) — rides the one-shot
#: ``chat()`` calls (document summaries, folder summaries, KB
#: overview, the ``check_models`` probe) as ``reasoning_effort``.
#: Default EMPTY = the off switch (phase-172 A2): the parameter is
#: not sent at all; the TODO's "none" means absence, never a
#: literal "none" string.
llm_summary_thinking_level: str = ""
#: Operator kill-switch for the ``thinking`` SSE events (phase 17,
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
#: pieces are still counted for the per-turn log line but never
app/core/settings_catalog.py
+5 -2
View File
@@ -151,8 +151,11 @@ CATALOG: dict[str, SettingMeta] = {
"llm_summary_model": SettingMeta(
"llm_summary_model", "LLM (aipi)", "LLM summary model"
),
"llm_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level"
"llm_chat_thinking_level": SettingMeta(
"llm_chat_thinking_level", "LLM (aipi)", "Chat thinking level"
),
"llm_summary_thinking_level": SettingMeta(
"llm_summary_thinking_level", "LLM (aipi)", "Summary thinking level"
),
"stream_thinking": SettingMeta(
"stream_thinking", "LLM (aipi)", "Stream thinking"
app/rag/llm.py
+30 -18
View File
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty). The operator's thinking level
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
as the top-level ``reasoning_effort`` parameter (via the SDK's
``extra_body``); an empty value is the off switch — the parameter is
absent, the request is byte-identical to pre-phase-172, and the
embeddings path never carries it.
(an answer can in principle be empty). The operator's thinking levels
(phase 178) ride chat-completions requests as the top-level
``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
default ``low``; the phase-172 knob, renamed) rides streaming
``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
default EMPTY = off) rides ``chat`` — document summaries, folder
summaries, the KB overview, and the ``check_models`` probe (a
reachability ping, off by default, no special-casing). An empty value is
the off switch — the parameter is absent (never a literal ``"none"``),
the request is byte-identical to pre-phase-172, and the embeddings path
never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
def _thinking_extra_body(level: str) -> dict[str, str] | None:
"""The thinking level as the chat-completions extra body (phase 178).
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
OpenAI-standard top-level parameter, phase-172 A1); empty /
whitespace → ``None`` (the off switch — the parameter is ABSENT
from the request, phase-172 A2; never a literal ``"none"`` or
``"off"``).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
trimmed = level.strip()
return {"reasoning_effort": trimmed} if trimmed else None
class EmbeddingError(RuntimeError):
@@ -429,7 +436,7 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else.
"""
extra = _thinking_extra_body(self.settings)
extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {}
)
@@ -440,8 +447,11 @@ class LLMClient:
temperature=0.2,
max_tokens=2048,
stream=False,
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
# Phase 178 (A3/A2): the **summary** level rides as
# reasoning_effort (document summaries, folder summaries,
# KB overview, the ``check_models`` probe); the key is
# absent (never None) when off — the default (empty)
# keeps one-shot requests byte-identical to pre-phase-172.
**extra_kwargs,
)
except LLMError:
@@ -622,8 +632,10 @@ class LLMClient:
}
if tools is not None:
kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None:
# Phase 172 (A1/A2): the thinking level rides as
if (
extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
) is not None:
# Phase 178 (A3/A2): the **chat** level rides as
# reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None
tests/e2e/test_split_thinking_levels.py
+532
View File
@@ -0,0 +1,532 @@
"""Phase 178 E2E (Playwright): the SPLIT thinking levels — a Chat
thinking level row and a Summary thinking level row in the admin
Settings panel — the owner's acceptance for TWO independently
tunable, pi-style LLM reasoning knobs, proven in a real browser.
Source: ``TODO.md`` L3 — "Need a separate summary thinking level and
chat thinking level. Default thinking level for summary should be
none. Chat should stay at low"
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_both_rows_render_free_form_with_defaults`` — the rows:
the ``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Chat thinking level** with the
``BOR_LLM_CHAT_THINKING_LEVEL`` env-var display shows ``low``
with the ``env`` badge (chat stays at low, TODO L3), a row
labelled **Summary thinking level** with the
``BOR_LLM_SUMMARY_THINKING_LEVEL`` display shows an **empty**
input with the ``env`` badge (the TODO's "none" = the phase-172
off switch, LOCKED A2 — the parameter is ABSENT, never a
literal "none"), BOTH inputs are ``type="text"`` (free-form —
NOT ``number``/``select``, the A4 contract: no allowlist, the
endpoint is the source of truth), the ``llm_chat_model`` row's
input is the same control (the "like chat model" reference, one
assertion pair), and ``tr[data-key="llm_thinking_level"]`` is
**absent** (the old phase-172 key is gone, LOCKED A1).
2. ``test_saved_levels_persist_without_restart`` — the save (the
phase-168 §7.4 Save lifecycle per row: dirty → save → ``DB``
badge): ``think-soft`` (a free-form custom — NOT a known pi
level, A4) is typed into the summary row and ``high`` into the
chat row (the dirty hints appear), then BOTH are saved; the
API catalogue (``_catalog_item``) AND the raw ``app_settings``
rows carry the saved values for EACH key independently (the
independence pin — the two rows' controls are distinct and the
saved values do not cross), and a reload of ``/settings.html``
still shows ``think-soft`` / ``high`` with the ``DB`` badges
(no restart, the phase-168 contract).
3. ``test_reset_returns_to_env_defaults`` — the per-row Reset (the
per-row ``DELETE``): after the saves, both Resets are live;
clicking the summary Reset lands the value at **empty** with
the ``env`` badge and clicking the chat Reset lands ``low``
with the ``env`` badge (the overlay reset, task 02's contract,
in the browser); the status lines show the house "… is back to
the environment value." toast for each.
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens), and — because the
direct SQL truncate does NOT invalidate the app's settings cache
(the admin API is the only mutation surface that does) — follows
it with the app's own clear-all (``DELETE /api/admin/settings``;
without it, a test that types the SAME value its predecessor
saved sees the stale value as the served one, registers no dirty
state, and the Save button stays disabled). The module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_CHAT_THINKING_LEVEL=low`` AND
``BOR_LLM_SUMMARY_THINKING_LEVEL=`` (the empty string SET — so
both badges read ``env``, not ``default``, and an operator's
local ``.env`` cannot leak a different level into the suite;
the house leak-guard pattern) together with the "like chat
model" reference (``BOR_LLM_CHAT_MODEL``). A module-scope ``.env``
mtime guard pins LOCKED A1: no code path the suite drives opens
the env file for writing.
ASSUMPTION A5 (phase-locked): the per-turn-line assertion
(``thinking=<chat level>``) and the wire split (``chat_stream()``
→ chat level, ``chat()`` → summary level) live in the
unit/integration layer (tasks 03/04) — this suite proves the
settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The chat row's saved value (test 2) — a known pi level, the
#: operator's "bump the chat turn" case.
SAVED_CHAT_LEVEL = "high"
#: The summary row's saved value (test 2) — NOT a known pi level
#: (proves the A4 contract: no allowlist, the endpoint is the
#: source of truth).
SAVED_SUMMARY_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-178
pins, with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the rows' save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 178 (task 05): pin BOTH thinking levels to their CODE
# DEFAULTS — each env var is SET (so the badge reads ``env``,
# not ``default``) and an operator's local .env cannot leak a
# different level into the suite's app. The summary level is
# set to EMPTY (the TODO's "none" = the off switch, A2).
env["BOR_LLM_CHAT_THINKING_LEVEL"] = Settings.model_fields[
"llm_chat_thinking_level"
].default
env["BOR_LLM_SUMMARY_THINKING_LEVEL"] = Settings.model_fields[
"llm_summary_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: BOTH thinking
levels pinned to their code defaults (chat ``low``, summary
empty = the A2 off switch) and ``BOR_SETTINGS_DB=1`` (the
module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the rows are wiped
between tests) and the single-row ``ui_settings`` (the
middleware reads it for EVERY page), and delete this suite's
issued tokens (label-scoped on ``e2e-`` — never a TRUNCATE: the
shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
def _invalidate_app_cache(app_url: str) -> None:
"""The direct SQL truncate does NOT invalidate the app's
``get_settings()`` lru-cache (the admin API is the only
mutation surface that invalidates — the phase-168 contract)
— so after the truncate, hit the app's own clear-all
(``DELETE /api/admin/settings``): zero rows to delete, but the
invalidate call rebuilds the cache env-only. Without it, a test
that types the SAME value its predecessor saved sees the STALE
value as the served one (``markDirty`` finds no diff, the Save
button stays disabled)."""
client = _admin_client(app_url)
r = client.delete(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, f"clear-all → {r.status_code}: {r.text[:200]}"
@pytest.fixture(autouse=True)
def _clean(app_server: str, db_ready: None) -> Iterator[None]:
_clean_settings_state()
_invalidate_app_cache(app_server)
yield
_clean_settings_state()
_invalidate_app_cache(app_server)
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
def _raw_value(key: str) -> str | None:
"""The raw ``app_settings`` row for ``key`` (the overlay's D1
env-native string form), or None when no override is stored."""
with SessionLocal() as db:
return db.execute(
text("SELECT value FROM app_settings WHERE key = :k"), {"k": key}
).scalar_one_or_none()
def _raw_count(key: str) -> int:
with SessionLocal() as db:
return db.execute(
text("SELECT count(*) FROM app_settings WHERE key = :k"), {"k": key}
).scalar_one()
# ---------------------------------------------------------------------------
# 1. Both rows render free-form with defaults (chat ``low``,
# summary empty = the A2 off switch; the old key is gone)
# ---------------------------------------------------------------------------
def test_both_rows_render_free_form_with_defaults(
page: Page, app_url: str, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
client = _admin_client(app_url)
# The LLM (aipi) group band is visible (the phase-171 band
# contract — both rows sit in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# The CHAT row: the label, the env-var display (the catalogue
# pattern), the free-form text control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth)…
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
expect(chat_row).to_be_visible()
expect(chat_row.locator(".cell-clip")).to_have_text("Chat thinking level")
expect(chat_row.locator(".settings-env")).to_have_text("BOR_LLM_CHAT_THINKING_LEVEL")
assert chat_row.locator(".settings-control").get_attribute("type") == "text"
# …chat stays at low (TODO L3): the env-pinned ``low`` with the
# env badge (the env var is set — no app_settings row, no
# code-default badge).
expect(chat_row.locator(".settings-control")).to_have_value("low")
expect(chat_row.locator(".settings-src")).to_have_text("env")
expect(chat_row.locator(".settings-reset")).to_be_disabled()
# The SUMMARY row: the label, the env-var display, the free-form
# text control…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
expect(summary_row).to_be_visible()
expect(summary_row.locator(".cell-clip")).to_have_text("Summary thinking level")
expect(summary_row.locator(".settings-env")).to_have_text(
"BOR_LLM_SUMMARY_THINKING_LEVEL"
)
assert summary_row.locator(".settings-control").get_attribute("type") == "text"
# …and the TODO's "none" = the off switch (A2): an EMPTY input
# with the env badge (the env var is SET to the empty string —
# the badge reads env, not default).
expect(summary_row.locator(".settings-control")).to_have_value("")
expect(summary_row.locator(".settings-src")).to_have_text("env")
expect(summary_row.locator(".settings-reset")).to_be_disabled()
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair
# covering both thinking rows).
chat_model_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert chat_model_type == "text"
assert (
chat_row.locator(".settings-control").get_attribute("type")
== summary_row.locator(".settings-control").get_attribute("type")
== chat_model_type
)
# The old phase-172 key is GONE (LOCKED A1 — the rename, not
# add-on): no row for llm_thinking_level anywhere in the panel.
assert page.locator('tr[data-key="llm_thinking_level"]').count() == 0
# The server agrees with the panel: both defaults, both sourced
# from the env (the module pins), stored raw on no row.
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "env" and chat_item["value"] == "low"
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "env" and summary_item["value"] == ""
assert _raw_count("llm_chat_thinking_level") == 0
assert _raw_count("llm_summary_thinking_level") == 0
# ---------------------------------------------------------------------------
# 2. The saves: dirty → save → DB badge, EACH key's value persists
# independently and survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_levels_persist_without_restart(
page: Page, app_url: str, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type the summary's free-form custom
# value (A4 — NOT a known pi level: no allowlist, the endpoint
# is the source of truth) and the chat's bump — two dirty
# rows, the hints appear…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
summary_row.locator(".settings-control").fill(SAVED_SUMMARY_LEVEL)
chat_row.locator(".settings-control").fill(SAVED_CHAT_LEVEL)
expect(page.locator("#settings-dirty")).to_have_text("2 unsaved changes")
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then BOTH badges flip to DB (the
# re-GET settled the rows).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 2 settings.", timeout=30_000
)
expect(summary_row.locator(".settings-src")).to_have_text("DB")
expect(chat_row.locator(".settings-src")).to_have_text("DB")
# The effective values are the saved ones for EACH key
# independently (the independence pin — the two rows' controls
# are distinct and the saved values do not cross): the API
# catalogue (what the LLM layer reads per request via
# get_settings)…
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "db" and summary_item["value"] == SAVED_SUMMARY_LEVEL
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "db" and chat_item["value"] == SAVED_CHAT_LEVEL
# …and the raw app_settings rows are stored RAW (the overlay's
# D1 env-native string form), one per key.
assert _raw_value("llm_summary_thinking_level") == SAVED_SUMMARY_LEVEL
assert _raw_value("llm_chat_thinking_level") == SAVED_CHAT_LEVEL
# No restart: the DB values survive a reload of /settings.html
# (the panel re-reads the settled server state), the summary
# row's custom value and the chat row's bump each still on
# their OWN row with the DB badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_summary_thinking_level"] .settings-control')
).to_have_value(SAVED_SUMMARY_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_summary_thinking_level"] .settings-src')
).to_have_text("DB")
expect(
page.locator('tr[data-key="llm_chat_thinking_level"] .settings-control')
).to_have_value(SAVED_CHAT_LEVEL)
expect(
page.locator('tr[data-key="llm_chat_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Resets: each row goes back to ITS environment default (the
# per-row DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_defaults(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish saved overrides first (the Resets are disabled with
# nothing to reset — the phase-168 render contract): save a
# value on each row so both Resets go live…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
summary_row.locator(".settings-control").fill(SAVED_SUMMARY_LEVEL)
chat_row.locator(".settings-control").fill(SAVED_CHAT_LEVEL)
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 2 settings.", timeout=30_000
)
# The re-GET settles the rows (DB badges) before the Resets go
# live (the phase-168 render contract — the button tracks the
# SETTLED source, not the click).
expect(summary_row.locator(".settings-src")).to_have_text("DB")
expect(chat_row.locator(".settings-src")).to_have_text("DB")
expect(summary_row.locator(".settings-reset")).to_be_enabled()
expect(chat_row.locator(".settings-reset")).to_be_enabled()
# The summary Reset (the per-row DELETE): the status line…
summary_row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_SUMMARY_THINKING_LEVEL is back to the environment value.",
timeout=30_000,
)
# …the value lands at EMPTY (the env var is set to the empty
# string — the env badge, not the code-default badge) and the
# row's Reset goes live-free again.
expect(summary_row.locator(".settings-control")).to_have_value("")
expect(summary_row.locator(".settings-src")).to_have_text("env")
expect(summary_row.locator(".settings-reset")).to_be_disabled()
# The chat Reset (the per-row DELETE): the status line…
chat_row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_CHAT_THINKING_LEVEL is back to the environment value.",
timeout=30_000,
)
# …the value reads ``low`` again (chat stays at low, TODO L3)
# with the env badge, and the row's Reset is disabled.
expect(chat_row.locator(".settings-control")).to_have_value("low")
expect(chat_row.locator(".settings-src")).to_have_text("env")
expect(chat_row.locator(".settings-reset")).to_be_disabled()
# The server agrees: both overrides are gone, the effective
# values are the env defaults, and no raw row survives.
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "env" and summary_item["value"] == ""
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "env" and chat_item["value"] == "low"
assert _raw_count("llm_summary_thinking_level") == 0
assert _raw_count("llm_chat_thinking_level") == 0
tests/e2e/test_thinking_level.py
-405
View File
@@ -1,405 +0,0 @@
"""Phase 172 E2E (Playwright): the Thinking level row in the admin Settings
panel — the owner's acceptance for a pi-style, operator-tunable LLM
reasoning knob, proven in a real browser.
Source: ``TODO.md`` L3 — "Brain of Reese needs to support thinking levels
like pi does. should be a free-form text entry like chat model. Default
should be 'low'. Update .env and admin settings."
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_level.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_row_renders_free_form_with_default_low`` — the row: the
``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Thinking level** with the
``BOR_LLM_THINKING_LEVEL`` env-var display (the catalogue
pattern) is present, its input is a **free-form text** input
(``type="text"`` — NOT ``number``/``select``, the A4
contract), it shows ``low`` with the ``env`` badge, and the
``llm_chat_model`` row's input is the same ``type="text"``
control (the "like chat model" reference, in one assertion
pair).
2. ``test_saved_level_persists_without_restart`` — the save
(the phase-168 §7.4 Save lifecycle: dirty → save → ``DB``
badge): ``high`` is typed (the dirty hint appears), then the
free-form custom value ``think-soft`` is typed once (no
allowlist — A4) and saved; the effective value reads
``think-soft`` (the API catalogue AND the raw
``app_settings`` row), and a reload of ``/settings.html``
still shows ``think-soft`` with the ``DB`` badge (no
restart, the phase-168 contract).
3. ``test_reset_returns_to_env_default`` — the per-row Reset
(the per-row ``DELETE``): after a save, the Reset is live;
clicking it lands the status line and the value reads
``low`` again with the ``env`` badge (the overlay reset,
task 02's contract, in the browser).
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens); the module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_THINKING_LEVEL`` (and the
"like chat model" reference, ``BOR_LLM_CHAT_MODEL``) to their
CODE DEFAULTS — the env var is set (so the badge reads ``env``,
not ``default``) and an operator's local ``.env`` cannot leak a
different level into the suite's app (the house leak-guard
pattern). A module-scope ``.env`` mtime guard pins LOCKED A1:
no code path the suite drives opens the env file for writing.
ASSUMPTION A6 (phase-locked): the per-turn-line assertion
(``thinking=<value>``) lives in the unit/integration layer
(task 04) — the mock LLM turn does not exercise the wire, and
this suite proves the settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The save's custom free-form value (test 2) — NOT a known pi
#: level (proves the A4 contract: no allowlist, the endpoint is
#: the source of truth).
SAVED_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-172 pins,
with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the row's save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 172 (task 05): pin the thinking level to the CODE
# DEFAULT — the env var is SET (so the badge reads ``env``, not
# ``default``) and an operator's local .env cannot leak a
# different level into the suite's app.
env["BOR_LLM_THINKING_LEVEL"] = Settings.model_fields[
"llm_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: the thinking level
pinned to the code default ``low`` and ``BOR_SETTINGS_DB=1``
(the module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the new key's rows
are wiped between tests) and the single-row ``ui_settings``
(the middleware reads it for EVERY page), and delete this
suite's issued tokens (label-scoped on ``e2e-`` — never a
TRUNCATE: the shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
_clean_settings_state()
yield
_clean_settings_state()
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
# ---------------------------------------------------------------------------
# 1. The row renders free-form with default low (the "like chat
# model" reference, in one assertion pair)
# ---------------------------------------------------------------------------
def test_row_renders_free_form_with_default_low(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
# The LLM (aipi) group band is visible (the phase-171 band
# contract — the row sits in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# …and the row is present with its label…
row = page.locator('tr[data-key="llm_thinking_level"]')
expect(row).to_be_visible()
expect(row.locator(".cell-clip")).to_have_text("Thinking level")
# …and the env-var display (the catalogue pattern).
expect(row.locator(".settings-env")).to_have_text("BOR_LLM_THINKING_LEVEL")
# The input is a FREE-FORM TEXT control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth).
assert row.locator(".settings-control").get_attribute("type") == "text"
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair).
thinking_type = row.locator(".settings-control").get_attribute("type")
chat_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert thinking_type == chat_type == "text"
# The default: the env-pinned ``low`` with the env badge (the
# env var is set — no app_settings row, no code-default badge).
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# Nothing is saved yet: the row's Reset is disabled (no saved
# override — the phase-168 render contract).
expect(row.locator(".settings-reset")).to_be_disabled()
# ---------------------------------------------------------------------------
# 2. The save: dirty → save → DB badge, the effective value is the
# free-form custom, and it survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_level_persists_without_restart(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type ``high`` — the dirty hint
# appears (one unsaved change on the row)…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
# …then type the free-form custom value once (A4 — NOT a known
# pi level: no allowlist, the endpoint is the source of truth).
row.locator(".settings-control").fill(SAVED_LEVEL)
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then the badge flips to DB (the
# re-GET settled the row).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
# The effective value reads ``think-soft`` — the API catalogue
# (what the LLM layer reads per request via get_settings)…
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "db" and item["value"] == SAVED_LEVEL
# …and the raw app_settings row is stored RAW (the overlay's
# D1 env-native string form).
with SessionLocal() as db:
raw = db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar_one()
assert raw == SAVED_LEVEL, f"the row must be stored raw (got {raw!r})"
# No restart: the DB value survives a reload of /settings.html
# (the panel re-reads the settled server state) with the DB
# badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-control')
).to_have_value(SAVED_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Reset: control goes back to the environment (the per-row
# DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_default(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish a saved override first (the Reset is disabled with
# nothing to reset — the phase-168 render contract): save
# ``high`` so the row's Reset goes live…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
expect(row.locator(".settings-reset")).to_be_enabled()
# The per-row Reset (the per-row DELETE): the status line…
row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_THINKING_LEVEL is back to the environment value.", timeout=30_000
)
# …the value reads ``low`` again with the env badge (the env
# var is set — not the code-default badge)…
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# …and the server agrees: the override is gone and the
# effective value is the env value again.
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "env" and item["value"] == "low"
with SessionLocal() as db:
assert (
db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar()
== 0
), "the DELETE must remove the row"
tests/integration/test_admin_settings_api.py
+122 -18
View File
@@ -518,11 +518,13 @@ def test_catalog_covers_every_settings_field(admin_client) -> None:
assert set(CATALOG) == set(Settings.model_fields)
def test_llm_thinking_level_row_contract(
def test_llm_chat_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the new ``llm_thinking_level`` row behaves
exactly like ``llm_chat_model`` — the machinery is generic over
"""Phase 178 (task 01; task 02 extends for the summary row): the
renamed ``llm_chat_thinking_level`` row (the phase-172 knob, the
rename per locked A1) behaves exactly like ``llm_chat_model`` —
the machinery is generic over
the catalogue (D4 / A5), so the per-key contract is: (a) present in
the GET body (group ``LLM (aipi)``, ``str``, effective ``low``
with no row in ``app_settings``, ``env_var`` named); (b) PUT stores
@@ -538,20 +540,20 @@ def test_llm_thinking_level_row_contract(
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, type, env var, effective ``low`` with NO row
# in app_settings, not secret, not restart-required (the LLM layer
# reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_thinking_level")
item = _item(r.json(), "llm_chat_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Thinking level"
assert item["label"] == "Chat thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_THINKING_LEVEL"
assert item["env_var"] == "BOR_LLM_CHAT_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == "low"
@@ -562,48 +564,150 @@ def test_llm_thinking_level_row_contract(
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"})
r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_thinking_level == "high" # live, no restart
assert get_settings().llm_chat_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable (the off switch, A2/A4 — the
# API stores as-is; no allowlist, no value validation): the row
# holds the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""})
r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_thinking_level == ""
assert get_settings().llm_chat_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``low`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_thinking_level")
r = admin_client.delete(f"{API}/llm_chat_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == "low"
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT count(*) FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"})
r = anon.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
def test_llm_summary_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the ``llm_summary_thinking_level`` row (the
new one-shot knob) behaves like the chat row — the machinery is
generic over the catalogue (D4 / A4) — with the summary specifics:
(a) present in the GET body (group ``LLM (aipi)``, ``str``, label
``"Summary thinking level"``, ``env_var``
``BOR_LLM_SUMMARY_THINKING_LEVEL``, effective ``""`` with no row in
``app_settings``); (b) PUT stores the RAW string in
``app_settings`` and the value resolves live WITHOUT a restart;
(c) the EMPTY value is storable (the summary off switch, A2 — the
API stores as-is, no allowlist, an empty stored value is never
rejected as "no value"); (d) DELETE resets to the env value
(``""`` — the code default); (e) the non-admin 403 gate applies;
(f) INDEPENDENCE — the two overlays resolve independently (the
whole point of the split): a saved summary value never changes the
chat row's effective value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): BOTH knobs at their code defaults, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, label, type, env var, effective ``""`` with NO
# row in app_settings, not secret, not restart-required (the LLM
# layer reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_summary_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Summary thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_SUMMARY_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == ""
assert item["source"] == "default" # the env var is unset, no row
assert item["has_override"] is False
assert item["db_value"] is None
assert item["default"] == ""
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_summary_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable — the summary off switch (A2; the
# API stores as-is; no allowlist, no value validation): an empty
# stored value must not be rejected as "no value"; the row holds
# the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_summary_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``""`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_summary_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_summary_thinking_level == ""
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
# (f) INDEPENDENCE — the whole point of the split: save the summary
# row "high" again; the chat row's effective value is STILL "low"
# (the two overlays resolve independently — a saved summary value
# never touches the chat level).
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert get_settings().llm_summary_thinking_level == "high"
assert get_settings().llm_chat_thinking_level == "low" # untouched
chat_item = _item(admin_client.get(API).json(), "llm_chat_thinking_level")
assert chat_item["value"] == "low" # the panel agrees, too
admin_client.delete(f"{API}/llm_summary_thinking_level") # no row survives
def test_docs_token_is_write_only_masked_end_to_end(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
tests/integration/test_admin_settings_live.py
+61 -16
View File
@@ -10,9 +10,10 @@ snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_thinking_level`` — the value the LLM
layer reads per request via ``get_settings()``; the off switch is
live-settable, A2).
phase-172 thinking level (``llm_chat_thinking_level`` — the value the
LLM layer reads per request via ``get_settings()``; the off switch is
live-settable, A2) and its phase-178 summary twin (task 02:
``llm_summary_thinking_level``, default ``""`` = off, A2).
The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` —
@@ -164,44 +165,88 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
# ---------------------------------------------------------------------------
# the thinking level: save/reset resolve the live value (phase 172, task 02)
# the chat thinking level: save/reset resolve the live value
# (phase 178, task 01 — the phase-172 test, renamed)
# ---------------------------------------------------------------------------
def test_llm_thinking_level_live_apply_no_restart(
def test_llm_chat_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the thinking level follows the overlay
"""Phase 178 (task 01): the chat thinking level follows the overlay
live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE
call after the write) resolves
``get_settings().llm_thinking_level`` WITHOUT a restart (the proof
the LLM layer — task 03 reads the setting per request via
``get_settings().llm_chat_thinking_level`` WITHOUT a restart (the
proof the LLM layer — task 03 reads the setting per request via
``get_settings()`` — picks the override up live), the EMPTY value
resolves live too (the off switch is live-settable, A2/A4), and a
reset (the DELETE) returns the env value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
_save(db, "llm_thinking_level", "high")
assert get_settings().llm_chat_thinking_level == "low"
_save(db, "llm_chat_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_thinking_level == "high" # NO restart
assert get_settings().llm_chat_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_thinking_level", "")
_save(db, "llm_chat_thinking_level", "")
invalidate_settings()
assert get_settings().llm_thinking_level == ""
assert get_settings().llm_chat_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'")
text("DELETE FROM app_settings WHERE key = 'llm_chat_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# ---------------------------------------------------------------------------
# the summary thinking level: save/reset resolve the live value
# (phase 178, task 02 — the chat twin's summary counterpart)
# ---------------------------------------------------------------------------
def test_llm_summary_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the summary thinking level follows the
overlay live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE call
after the write) resolves ``get_settings().llm_summary_thinking_level``
WITHOUT a restart (the proof the LLM layer — task 03 reads the
setting per request via ``get_settings()`` — picks the override up
live), the EMPTY value resolves live too (the off switch is
live-settable, A2/A4), and a reset (the DELETE) returns the env
value (``""`` — the code default; the chat twin's reset returns
``"low"``, the two levels are independent).
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
_save(db, "llm_summary_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_summary_thinking_level", "")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_summary_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# ---------------------------------------------------------------------------
tests/unit/test_chat_gate.py
+45 -5
View File
@@ -1550,8 +1550,13 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
seeded suggestion tier's size (``len(plan.suggested_docs)``).
Phase 172 (A6): the line gains ``thinking=<level>`` immediately
BEFORE ``thinking_chars=N`` — the effective (overlay-resolved)
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off
switch)."""
``BOR_LLM_CHAT_THINKING_LEVEL`` (the chat level), trimmed,
``-`` when empty (the off switch).
Phase 178 (A5): the field stays pinned to the CHAT level — the
level the turn actually ran at; the one-shot summary level
(``llm_summary_thinking_level``, a different knob) is never read
by the line and must not leak into the line's ``thinking=``
value (cross-pin + inverse off-pin below)."""
_session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)]
@@ -1606,7 +1611,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue]
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level="high"), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=high thinking_chars=" in line
@@ -1615,7 +1620,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue]
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level=""), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
@@ -1625,7 +1630,42 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue]
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level=" "), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
# Phase 178 (A5) cross-pin: the line is pinned to the CHAT level
# — with the one-shot summary level set to a DIFFERENT value,
# the line still shows the chat level; the summary's "high"
# does not appear as the line's ``thinking=`` value.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
relevance_threshold=0.30,
llm_chat_thinking_level="low",
llm_summary_thinking_level="high",
),
)
line = _last_log_line()
assert "thinking=low thinking_chars=" in line
assert "thinking=high" not in line
# The inverse off-pin: the chat off-switch logs ``-`` even
# though the one-shot summary knob is set — only the chat level
# (the level this turn ran at) rides the line.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
relevance_threshold=0.30,
llm_chat_thinking_level="",
llm_summary_thinking_level="high",
),
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
assert "thinking=high" not in line
tests/unit/test_config.py
+56 -23
View File
@@ -105,32 +105,65 @@ def test_llm_summary_model_env_override(monkeypatch) -> None:
assert s.llm_summary_model == "mini"
def test_llm_thinking_level_default_is_low(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 172: the model's thinking level defaults to ``low``
(``BOR_LLM_THINKING_LEVEL`` unset)."""
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_thinking_level == "low"
def test_llm_thinking_level_env_override(monkeypatch) -> None:
"""Phase 172 (A4): free-form like ``llm_chat_model`` —
``BOR_LLM_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_thinking_level == "high"
def test_llm_thinking_level_empty_env_value_stays_empty(
def test_llm_chat_thinking_level_default_is_low(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 172 (A2): the EMPTY value is the off switch — it must not
be coerced back to the default (trimming happens only at the wire,
task 03; no normalization at load)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "")
"""Phase 178 (the phase-172 knob, renamed): the chat turn's
thinking level defaults to ``low`` (``BOR_LLM_CHAT_THINKING_LEVEL``
unset)."""
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_thinking_level == ""
assert s.llm_chat_thinking_level == "low"
def test_llm_chat_thinking_level_env_override(monkeypatch) -> None:
"""Phase 178 (A4, kept): free-form like ``llm_chat_model`` —
``BOR_LLM_CHAT_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_CHAT_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_chat_thinking_level == "high"
def test_llm_chat_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2, kept): the EMPTY value is the off switch — it
must not be coerced back to the default (trimming happens only at
the wire; no normalization at load)."""
monkeypatch.setenv("BOR_LLM_CHAT_THINKING_LEVEL", "")
s = _settings()
assert s.llm_chat_thinking_level == ""
def test_llm_summary_thinking_level_default_is_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2): the one-shot summary thinking level defaults to
EMPTY (``BOR_LLM_SUMMARY_THINKING_LEVEL`` unset) — the TODO's
"none" means absence, never a literal "none" string."""
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_summary_thinking_level == ""
def test_llm_summary_thinking_level_env_override(monkeypatch) -> None:
"""Phase 178: free-form like the chat level —
``BOR_LLM_SUMMARY_THINKING_LEVEL=high`` resolves to ``"high"``
(no allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_SUMMARY_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_summary_thinking_level == "high"
def test_llm_summary_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2, kept): the EMPTY value is the off switch — it
must not be coerced to anything at load (no normalization)."""
monkeypatch.setenv("BOR_LLM_SUMMARY_THINKING_LEVEL", "")
s = _settings()
assert s.llm_summary_thinking_level == ""
def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None:
tests/unit/test_llm_client.py
+116 -33
View File
@@ -1324,48 +1324,62 @@ def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch)
assert sleeps == [1.25, 1.25, 1.25]
# ---------- thinking level passthrough (phase 172, task 03) ----------
# ---------- thinking level passthrough (phase 172; the split, 178 A3) ----------
# chat() (one-shot) rides the SUMMARY level; chat_stream() rides the CHAT
# level; an empty (trimmed) level is the off switch (A2 — the parameter
# is absent, never a literal "none").
def test_chat_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): a non-empty level rides the one-shot request as
the top-level ``reasoning_effort`` parameter via the SDK's
# --- the one-shot path: the summary level (phase 178, A3) ---
def test_chat_sends_thinking_level_extra_body_when_summary_level_set() -> None:
"""Phase 178 (A3): a non-empty summary level rides the one-shot
request as the top-level ``reasoning_effort`` parameter via the SDK's
``extra_body``."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low"
_FakeCompletion("ok"), llm_summary_thinking_level="high"
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_default_level_low_is_always_sent() -> None:
"""The default (``low``) is always sent (A2): a client built without
naming the setting still carries the parameter."""
def test_chat_default_summary_thinking_off_is_pre_phase172_byte_identical() -> None:
"""The default summary level is EMPTY (the TODO's "none", A2): a
client built without naming either setting sends NO ``extra_body`` —
one-shot requests are byte-identical to pre-phase-172."""
llm, completions = _make_chat_client(_FakeCompletion("ok"))
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_trims_the_level_before_sending() -> None:
"""TRIM: a hand-edited ``.env`` value with trailing whitespace
(``"low "``) is sent trimmed — the raw stored string never reaches
the wire with surrounding whitespace."""
def test_chat_trims_the_summary_thinking_level_before_sending() -> None:
"""TRIM: a hand-edited ``.env`` value with surrounding whitespace
(``" high "``) is sent trimmed — the raw stored string never reaches
the wire with whitespace."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low "
_FakeCompletion("ok"), llm_summary_thinking_level=" high "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): a whitespace value is the off switch —
``extra_body`` is ABSENT (never None, never ``"off"``) and the
request kwargs keep the pre-phase-172 shape."""
def test_chat_empty_summary_thinking_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): an empty value is the off switch —
``extra_body`` is ABSENT (never None, never a literal ``"none"``) and
the request kwargs keep the pre-phase-172 shape."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level=" "
_FakeCompletion("ok"), llm_summary_thinking_level=""
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
@@ -1379,30 +1393,42 @@ def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None:
}
def test_chat_stream_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): the chat turn rides the parameter too."""
def test_chat_whitespace_summary_thinking_level_is_the_off_switch() -> None:
"""Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_summary_thinking_level=" "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
def test_chat_stream_sends_thinking_level_extra_body_when_level_set() -> None:
"""Phase 178 (A3): the chat turn rides the parameter from the
**chat** level."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level="high"
[_chunk("x")], llm_chat_thinking_level="high"
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_trims_the_level_before_sending() -> None:
def test_chat_stream_trims_the_thinking_level_before_sending() -> None:
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" high "
[_chunk("x")], llm_chat_thinking_level=" high "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): the stream's off switch — ``extra_body`` absent
(never None) and the kwargs dict keeps the pre-phase-172 shape."""
def test_chat_stream_empty_thinking_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 178 (A2): the stream's off switch — ``extra_body`` absent
(never None, never a literal ``"none"``) and the kwargs dict keeps
the pre-phase-172 shape."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=""
[_chunk("x")], llm_chat_thinking_level=""
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
@@ -1416,22 +1442,79 @@ def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None:
}
def test_chat_stream_whitespace_level_is_the_off_switch() -> None:
def test_chat_stream_whitespace_thinking_level_is_the_off_switch() -> None:
"""Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" "
[_chunk("x")], llm_chat_thinking_level=" "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs
# --- the cross-pins (phase 178, A3 — the whole point of the split) ---
def _make_both_client(
chunks: list,
completion: _FakeCompletion,
**settings_kwargs: Any,
) -> tuple[LLMClient, _FakeCompletions]:
"""One client for BOTH paths (a shared fake ``completions`` serves the
streaming and the non-streaming ``create()`` alike) so the two levels
can be observed on one client."""
completions = _FakeCompletions(chunks, None, completion)
fake_openai = SimpleNamespace(chat=SimpleNamespace(completions=completions))
llm = LLMClient(_settings(**settings_kwargs))
llm._client = fake_openai # pyright: ignore[reportAttributeAccessIssue]
return llm, completions
def test_split_thinking_levels_inverted_defaults_chat_high_summary_off() -> None:
"""Phase 178 (A3): the defaults INVERTED — the chat level ``high``
rides ``chat_stream`` while the (empty) summary level keeps ``chat``
free of ``extra_body``. The two knobs are independent on the wire."""
llm, completions = _make_both_client(
[_chunk("x")],
_FakeCompletion("ok"),
llm_chat_thinking_level="high",
llm_summary_thinking_level="",
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
def test_split_thinking_levels_default_client_stream_low_summary_off() -> None:
"""The DEFAULTS (chat ``low`` + summary ``""``) — the stream carries
``reasoning_effort="low"`` while the one-shot request is
byte-identical to pre-phase-172 (no ``extra_body``; the TODO's
"summary should be none")."""
llm, completions = _make_both_client([_chunk("x")], _FakeCompletion("ok"))
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "low"}
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_embeddings_body_never_carries_reasoning_effort() -> None:
"""Phase 172 (A3): the embeddings path is untouched — with the level
set, every POST body stays the minimal ``{model, input}`` payload;
``reasoning_effort`` never appears in it."""
service = _FakeEmbeddingsService()
llm, http = _make_client(service, llm_thinking_level="high")
llm, http = _make_client(service, llm_chat_thinking_level="high")
asyncio.run(llm.embed(["a", "b"]))
vec = asyncio.run(llm.embed_one("c"))
assert len(vec) == 768
tests/unit/test_settings_catalog.py
+19 -13
View File
@@ -96,19 +96,25 @@ def test_env_var_is_bare_key_upper_prefixed() -> None:
assert env_var(key).startswith("BOR_")
def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None:
"""Phase 172 (task 01): the new ``llm_thinking_level`` pair sits in
``LLM (aipi)`` with the ``Thinking level`` label and is plain —
not ``secret`` (a free-form text value, like ``llm_chat_model``),
not ``restart_required`` (the LLM client reads the setting per
request via ``get_settings()`` — live), overridable (the overlay is
generic over ``str`` fields)."""
meta = CATALOG["llm_thinking_level"]
assert meta.group == "LLM (aipi)"
assert meta.label == "Thinking level"
assert "llm_thinking_level" not in SECRETS
assert "llm_thinking_level" not in RESTART_REQUIRED
assert "llm_thinking_level" not in NON_OVERRIDABLE
def test_llm_thinking_levels_are_plain_live_overridable_rows() -> None:
"""Phase 178 (task 01): the two thinking-level rows (the phase-172
knob, renamed + the new summary level) sit in ``LLM (aipi)`` with
the ``Chat thinking level`` / ``Summary thinking level`` labels and
are plain — not ``secret`` (free-form text values, like
``llm_chat_model``), not ``restart_required`` (the LLM client
reads the settings per request via ``get_settings()`` — live),
overridable (the overlay is generic over ``str`` fields)."""
for key, label in (
("llm_chat_thinking_level", "Chat thinking level"),
("llm_summary_thinking_level", "Summary thinking level"),
):
meta = CATALOG[key]
assert meta.group == "LLM (aipi)"
assert meta.label == label
assert meta.secret is False
assert key not in SECRETS
assert key not in RESTART_REQUIRED
assert key not in NON_OVERRIDABLE
def test_docs_user_is_a_plain_live_overridable_row() -> None: