Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 172_thinking_level
Build and Push Containers / build-and-push-app (push) Successful in 1m43s
Details
Build and Push Containers / build-and-push-db (push) Successful in 12s
Details

Browse Source 
All verification complete. Final report:

**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.**

- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`.
- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands).
- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol.
- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
This commit is contained in:
ducoterra
2026-10-10 13:19:10 -04:00
parent b21b05eaee
commit f27106a638
43 changed files with 1689 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/172_thinking_level/00_phase.md → .agents/phases/complete/172_thinking_level/00_phase.md
View File
.agents/phases/todo/172_thinking_level/01_setting_and_env.md → .agents/phases/complete/172_thinking_level/01_setting_and_env.md
View File
.agents/phases/todo/172_thinking_level/02_admin_row_contract.md → .agents/phases/complete/172_thinking_level/02_admin_row_contract.md
View File
.agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md → .agents/phases/complete/172_thinking_level/03_llm_extra_body_passthrough.md
View File
.agents/phases/todo/172_thinking_level/04_per_turn_log_field.md → .agents/phases/complete/172_thinking_level/04_per_turn_log_field.md
View File
.agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md → .agents/phases/complete/172_thinking_level/05_e2e_admin_settings_row.md
View File
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md
+9
View File
@@ -0,0 +1,9 @@
All verification complete. Final report:
**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.**
- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`.
- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands).
- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol.
- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md
+22
View File
@@ -0,0 +1,22 @@
Task 01 complete — everything green.
**Implemented**
- `app/config.py`: `llm_thinking_level: str = "low"` with house docstring (env var, default, `reasoning_effort` wire mapping, empty = off switch), placed after `llm_summary_model` before `stream_thinking`; no validator/coercion (A4/A2)
- `.env.example`: `BOR_LLM_THINKING_LEVEL=low` with one-line comment, directly under `BOR_LLM_CHAT_MODEL=turbo` (L60→L61)
- `app/core/settings_catalog.py`: `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` after the `llm_summary_model` entry — plain (not secret, not restart-required, overridable)
- Tests: 3 new cases in `tests/unit/test_config.py` (default `low`, env override `high`, empty env stays `""`); 1 new case in `tests/unit/test_settings_catalog.py` (group/label + not in the derived SECRETS/RESTART_REQUIRED/NON_OVERRIDABLE sets)
**Results**
- `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` → 5 passed
- `uv run pytest` → 4073 passed (6 initial failures were only the DB being down; `podman compose up -d db` → all green)
- `uv run pytest --cov=app --cov-report=term-missing` → 99% on `app/` (>90% gate)
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
- `GET /api/admin/settings` verified via TestClient: row present in `LLM (aipi)`, `str`, effective `low` (source `default`), overridable, not secret/restart
**Docs**
- `README.md`: added `BOR_LLM_THINKING_LEVEL` row to the LLM env-var table; `AGENTS.md`: no doc impact (no env list there)
**Decisions**
- None beyond the phase design; `settings.js` untouched (A5 — the row falls out of the catalogue)
**Next pending task:** `02_admin_row_contract.md`
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
......................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6393 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Report — phase 172, task 02 (admin row API contract for `llm_thinking_level`)**
Implemented (tests only, per task — no `app/` code touched):
- `tests/integration/test_admin_settings_api.py::test_llm_thinking_level_row_contract` — (a) GET: present, group `LLM (aipi)`, `str`, `env_var` `BOR_LLM_THINKING_LEVEL`, effective `low`, not secret/restart-required; (b) PUT `"high"` → 200, row stored **raw** in `app_settings`, live effective `high`; (c) PUT `""` → 200 storable (off switch, no allowlist); (d) DELETE → row gone, back to `low`; (e) non-admin 403 asserted once (full matrix already pinned). Comment notes the shape/coverage tests pin presence generically (suite is not key-parameterized).
- `tests/integration/test_admin_settings_live.py::test_llm_thinking_level_live_apply_no_restart` — house `_save` + `invalidate_settings()` pattern: `high` live without restart, empty resolves live (A2/A4), DELETE resets to `low`; module docstring updated.
- Deterministic base in both: `monkeypatch.delenv` + code default `low` (kill-switch test idiom, machine-independent).
Gates (all green):
- `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` → 20 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 4075 passed, app/ coverage 99% (>90%)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
Docs: no impact — test-only task; the env var is documented in `.env.example` (task 01), and README/AGENTS.md don't enumerate individual settings.
Next pending task: `03_llm_extra_body_passthrough.md`.
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6393 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md
+14
View File
@@ -0,0 +1,14 @@
Task 03 complete — all gates green.
**Report**
- **Implemented (task 03 — wire passthrough):**
- `_thinking_extra_body()` module-level helper in `app/rag/llm.py`: trimmed non-empty level → `{"reasoning_effort": <value>}`; empty/whitespace → `None` (off switch)
- `chat()` (via `_chat_once`): `extra_body` spread into `create()` **only when present** — key absent (never `None`) when off; pyright-safe typed `dict[str, Any]`
- `chat_stream()`: `kwargs["extra_body"]` set after the `tools` block, same absent-when-off pattern
- Module docstring: one sentence on the phase-172 wire contract (empty = absent, embeddings never carry it)
- 9 new unit tests in `tests/unit/test_llm_client.py`: `chat()` low/default/whitespace, `chat_stream()` high/empty/whitespace, TRIM (`"low "` → `"low"`), byte-identical kwargs-shape pins for the off switch, and the embeddings POST body never containing `reasoning_effort`
- **Results:** `uv run pytest --cov=app` → **4084 passed**, 99% on `app/` (>90% gate); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors; `uv run pytest tests/unit/test_llm_client.py -v --no-cov` → 81 passed
- **Docs:** no doc impact — README L1156 already documents `BOR_LLM_THINKING_LEVEL` incl. the wire mapping (task 01's doc work); AGENTS.md unaffected
- **Decisions:** task-spec inline `**` spread failed pyright (empty-dict inference, 33 errors) — split into an annotated `extra_kwargs: dict[str, Any]`, semantics identical (key absent when off)
- **Next pending task:** `04_per_turn_log_field.md`
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md
+14
View File
@@ -0,0 +1,14 @@
All gates green. The change is a log-line-only observability field — `README.md` documents the `thinking` SSE events (untouched UI surface) but not the per-turn line's field list, and `AGENTS.md` rule 10 refers to the line generically, so no doc edits are needed.
**Report — task 04 (per-turn log field)**
- `app/api/chat.py`: per-turn log line gains `thinking=%s` immediately before `thinking_chars=%d`, argument `settings.llm_thinking_level.strip() or "-"` (effective overlay-resolved value; existing fields un-reordered, `query_log` row untouched)
- `app/api/chat.py`: one-line module-docstring note in the Thinking paragraph (house pattern) — `thinking=<level>` (effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172, A6)
- `tests/unit/test_chat_gate.py`: extended the existing per-turn-line field-order test — asserts `thinking=low` (default), `thinking=high` (effective non-default), `thinking=-` (empty and whitespace-only, pinning the trim), adjacency to `thinking_chars`, and `thinking=` inserted into the full field-order tuple
- `uv run pytest --cov=app` → 4084 passed, `app/` coverage 99% (>90% gate)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- `grep -rn "thinking=" app/api/chat.py` → exactly once in the format string (L1275); no `query_log` schema change
- Docs: no README/AGENTS.md impact (log-line field is server-observability only; both files reference the line generically)
- No deviations; A6 honored exactly
**Next pending task:** `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (dedicated Playwright suite), regression set, full gates, atomic commit.
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md
+12
View File
@@ -0,0 +1,12 @@
All gates green. TODO.md is already cleared by the conversion commit (unmodified), so the phase commit is phase files + code only, as the task anticipated. The commit itself and the `todo/ → complete/` move are the harness's to make at phase end (per protocol, I leave every change in the working tree).
## Report
- **Implemented:** `tests/e2e/test_thinking_level.py` — the phase's dedicated Playwright suite (3 tests), mirroring the phase-168 harness (module `_boot_app` with mock LLM + `BOR_SETTINGS_DB=1` opt-in, `BOR_LLM_THINKING_LEVEL=low` pinned to the code default, `app_settings`/`ui_settings` autouse wipe, `.env` mtime guard, `_admin_client`):
- `test_row_renders_free_form_with_default_low` — `LLM (aipi)` band visible; "Thinking level" row with `BOR_LLM_THINKING_LEVEL` env-var display; `type="text"` free-form control (not number/select); `low` + `env` badge; `llm_chat_model` control asserted as the same `type="text"` (one assertion pair)
- `test_saved_level_persists_without_restart` — dirty → save → `DB` badge; custom value `think-soft` saved raw (API effective + raw `app_settings` row); survives reload without restart
- `test_reset_returns_to_env_default` — per-row Reset (DELETE) after a save → status line, `low` + `env` badge, row gone
- **Gates (all green):** `uv run pytest -v` → 4084 passed; `--cov=app` → **99%** (>90%); `uv run ruff check . && uv run pyright` → clean; `tests/e2e/test_thinking_level.py --no-cov` → 3 passed in isolation; regression `test_admin_settings_panel.py` → 12 passed, `test_admin_settings_layout.py` → 5 passed (both in isolation)
- **Docs:** no doc changes needed — `README.md` already carries the `BOR_LLM_THINKING_LEVEL` row (tasks 01–04); AGENTS.md has no per-setting content
- **Decisions:** the module app pins `BOR_LLM_THINKING_LEVEL=low` (env set → `env` badge, house leak-guard) and `BOR_LLM_CHAT_MODEL` to its code default; no KB seeding (A6 — the settings surface only, per-turn line is task 04's integration contract); commit + phase-dir move deferred to the harness
- **Next pending task:** none — task 05 is the phase's final task; phase 172 is complete
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.env.example
+1
View File
@@ -58,6 +58,7 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
BOR_LLM_CHAT_MODEL=turbo
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
# BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off
# BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120)
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls)
README.md
+1
View File
@@ -1153,6 +1153,7 @@ never the plaintext).
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` |
| `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model |
| `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries |
| `BOR_LLM_THINKING_LEVEL` | `low` | The model's thinking level (pi-style, free-form — no allowlist); rides every chat-completions request as `reasoning_effort`; empty = the parameter is not sent |
| `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) |
| `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off |
| `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries |
app/api/chat.py
+7 -3
View File
@@ -13,7 +13,10 @@ model's reasoning arrives ahead of the answer and is streamed as
``thinking`` events before the ``delta`` events of the same turn. Each
turn's thinking is counted in the per-turn log line
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
``thinking`` frames (the pieces are still counted).
``thinking`` frames (the pieces are still counted). The per-turn log
line records ``thinking=<level>`` (the effective
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6).
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
@@ -1269,8 +1272,8 @@ async def chat(
"question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d "
"suggested=%d tuning=%d "
"kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r "
"thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d "
"retries=%d scaffold_stripped=%d",
"thinking=%s thinking_chars=%d tool_calls=%d rounds=%d "
"cap_hit=%d total_ms=%d retries=%d scaffold_stripped=%d",
request.message,
embed_ms,
plan.top_score,
@@ -1283,6 +1286,7 @@ async def chat(
settings.relevance_threshold,
plan.deflected,
source_paths,
settings.llm_thinking_level.strip() or "-", # phase 172 (A6)
thinking_chars,
holder.tool_calls,
holder.rounds,
app/config.py
+8
View File
@@ -105,6 +105,14 @@ class Settings(BaseSettings):
#: Served by the same OpenAI-compatible endpoint — no new model
#: management. Called via ``LLMClient.chat()``.
llm_summary_model: str = "lite"
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to
#: aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
#: parameter is not sent at all.
llm_thinking_level: str = "low"
#: Operator kill-switch for the ``thinking`` SSE events (phase 17,
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
#: pieces are still counted for the per-turn log line but never
app/core/settings_catalog.py
+3
View File
@@ -150,6 +150,9 @@ CATALOG: dict[str, SettingMeta] = {
"llm_summary_model": SettingMeta(
"llm_summary_model", "LLM (aipi)", "LLM summary model"
),
"llm_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level"
),
"stream_thinking": SettingMeta(
"stream_thinking", "LLM (aipi)", "Stream thinking"
),
app/rag/llm.py
+29 -1
View File
@@ -10,7 +10,13 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty).
(an answer can in principle be empty). The operator's thinking level
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
as the top-level ``reasoning_effort`` parameter (via the SDK's
``extra_body``); an empty value is the off switch — the parameter is
absent, the request is byte-identical to pre-phase-172, and the
embeddings path never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different
@@ -40,6 +46,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
class EmbeddingError(RuntimeError):
"""The embeddings endpoint failed (network, HTTP, or malformed reply)."""
@@ -412,6 +429,10 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else.
"""
extra = _thinking_extra_body(self.settings)
extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {}
)
try:
resp = await self._client.chat.completions.create(
model=model,
@@ -419,6 +440,9 @@ class LLMClient:
temperature=0.2,
max_tokens=2048,
stream=False,
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
**extra_kwargs,
)
except LLMError:
raise
@@ -598,6 +622,10 @@ class LLMClient:
}
if tools is not None:
kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None:
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None
try:
stream = cast(
tests/e2e/test_thinking_level.py
+405
View File
@@ -0,0 +1,405 @@
"""Phase 172 E2E (Playwright): the Thinking level row in the admin Settings
panel — the owner's acceptance for a pi-style, operator-tunable LLM
reasoning knob, proven in a real browser.
Source: ``TODO.md`` L3 — "Brain of Reese needs to support thinking levels
like pi does. should be a free-form text entry like chat model. Default
should be 'low'. Update .env and admin settings."
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_level.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_row_renders_free_form_with_default_low`` — the row: the
``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Thinking level** with the
``BOR_LLM_THINKING_LEVEL`` env-var display (the catalogue
pattern) is present, its input is a **free-form text** input
(``type="text"`` — NOT ``number``/``select``, the A4
contract), it shows ``low`` with the ``env`` badge, and the
``llm_chat_model`` row's input is the same ``type="text"``
control (the "like chat model" reference, in one assertion
pair).
2. ``test_saved_level_persists_without_restart`` — the save
(the phase-168 §7.4 Save lifecycle: dirty → save → ``DB``
badge): ``high`` is typed (the dirty hint appears), then the
free-form custom value ``think-soft`` is typed once (no
allowlist — A4) and saved; the effective value reads
``think-soft`` (the API catalogue AND the raw
``app_settings`` row), and a reload of ``/settings.html``
still shows ``think-soft`` with the ``DB`` badge (no
restart, the phase-168 contract).
3. ``test_reset_returns_to_env_default`` — the per-row Reset
(the per-row ``DELETE``): after a save, the Reset is live;
clicking it lands the status line and the value reads
``low`` again with the ``env`` badge (the overlay reset,
task 02's contract, in the browser).
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens); the module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_THINKING_LEVEL`` (and the
"like chat model" reference, ``BOR_LLM_CHAT_MODEL``) to their
CODE DEFAULTS — the env var is set (so the badge reads ``env``,
not ``default``) and an operator's local ``.env`` cannot leak a
different level into the suite's app (the house leak-guard
pattern). A module-scope ``.env`` mtime guard pins LOCKED A1:
no code path the suite drives opens the env file for writing.
ASSUMPTION A6 (phase-locked): the per-turn-line assertion
(``thinking=<value>``) lives in the unit/integration layer
(task 04) — the mock LLM turn does not exercise the wire, and
this suite proves the settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The save's custom free-form value (test 2) — NOT a known pi
#: level (proves the A4 contract: no allowlist, the endpoint is
#: the source of truth).
SAVED_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-172 pins,
with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the row's save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 172 (task 05): pin the thinking level to the CODE
# DEFAULT — the env var is SET (so the badge reads ``env``, not
# ``default``) and an operator's local .env cannot leak a
# different level into the suite's app.
env["BOR_LLM_THINKING_LEVEL"] = Settings.model_fields[
"llm_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: the thinking level
pinned to the code default ``low`` and ``BOR_SETTINGS_DB=1``
(the module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the new key's rows
are wiped between tests) and the single-row ``ui_settings``
(the middleware reads it for EVERY page), and delete this
suite's issued tokens (label-scoped on ``e2e-`` — never a
TRUNCATE: the shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
_clean_settings_state()
yield
_clean_settings_state()
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
# ---------------------------------------------------------------------------
# 1. The row renders free-form with default low (the "like chat
# model" reference, in one assertion pair)
# ---------------------------------------------------------------------------
def test_row_renders_free_form_with_default_low(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
# The LLM (aipi) group band is visible (the phase-171 band
# contract — the row sits in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# …and the row is present with its label…
row = page.locator('tr[data-key="llm_thinking_level"]')
expect(row).to_be_visible()
expect(row.locator(".cell-clip")).to_have_text("Thinking level")
# …and the env-var display (the catalogue pattern).
expect(row.locator(".settings-env")).to_have_text("BOR_LLM_THINKING_LEVEL")
# The input is a FREE-FORM TEXT control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth).
assert row.locator(".settings-control").get_attribute("type") == "text"
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair).
thinking_type = row.locator(".settings-control").get_attribute("type")
chat_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert thinking_type == chat_type == "text"
# The default: the env-pinned ``low`` with the env badge (the
# env var is set — no app_settings row, no code-default badge).
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# Nothing is saved yet: the row's Reset is disabled (no saved
# override — the phase-168 render contract).
expect(row.locator(".settings-reset")).to_be_disabled()
# ---------------------------------------------------------------------------
# 2. The save: dirty → save → DB badge, the effective value is the
# free-form custom, and it survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_level_persists_without_restart(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type ``high`` — the dirty hint
# appears (one unsaved change on the row)…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
# …then type the free-form custom value once (A4 — NOT a known
# pi level: no allowlist, the endpoint is the source of truth).
row.locator(".settings-control").fill(SAVED_LEVEL)
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then the badge flips to DB (the
# re-GET settled the row).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
# The effective value reads ``think-soft`` — the API catalogue
# (what the LLM layer reads per request via get_settings)…
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "db" and item["value"] == SAVED_LEVEL
# …and the raw app_settings row is stored RAW (the overlay's
# D1 env-native string form).
with SessionLocal() as db:
raw = db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar_one()
assert raw == SAVED_LEVEL, f"the row must be stored raw (got {raw!r})"
# No restart: the DB value survives a reload of /settings.html
# (the panel re-reads the settled server state) with the DB
# badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-control')
).to_have_value(SAVED_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Reset: control goes back to the environment (the per-row
# DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_default(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish a saved override first (the Reset is disabled with
# nothing to reset — the phase-168 render contract): save
# ``high`` so the row's Reset goes live…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
expect(row.locator(".settings-reset")).to_be_enabled()
# The per-row Reset (the per-row DELETE): the status line…
row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_THINKING_LEVEL is back to the environment value.", timeout=30_000
)
# …the value reads ``low`` again with the env badge (the env
# var is set — not the code-default badge)…
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# …and the server agrees: the override is gone and the
# effective value is the env value again.
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "env" and item["value"] == "low"
with SessionLocal() as db:
assert (
db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar()
== 0
), "the DELETE must remove the row"
tests/integration/test_admin_settings_api.py
+86
View File
@@ -516,3 +516,89 @@ def test_catalog_covers_every_settings_field(admin_client) -> None:
assert r.status_code == 200
assert {item["key"] for item in r.json()["items"]} == set(Settings.model_fields)
assert set(CATALOG) == set(Settings.model_fields)
def test_llm_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the new ``llm_thinking_level`` row behaves
exactly like ``llm_chat_model`` — the machinery is generic over
the catalogue (D4 / A5), so the per-key contract is: (a) present in
the GET body (group ``LLM (aipi)``, ``str``, effective ``low``
with no row in ``app_settings``, ``env_var`` named); (b) PUT stores
the RAW string in ``app_settings`` (the overlay's env-native
string form) and the response reports the new effective value;
(c) the EMPTY value is storable (the off switch, A2/A4 — the API
stores as-is, no allowlist); (d) DELETE resets to the env value;
(e) the non-admin 403 gate applies (asserted once here; the full
four-route matrix is ``test_auth_matrix_all_four_routes``).
``test_catalogue_shape_and_sources_on_empty_table`` +
``test_catalog_covers_every_settings_field`` already pin the row's
PRESENCE generically — these cases pin the per-key behavior.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
# (a) the row: group, type, env var, effective ``low`` with NO row
# in app_settings, not secret, not restart-required (the LLM layer
# reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == "low"
assert item["source"] in ("default", "env") # no row in app_settings
assert item["has_override"] is False
assert item["db_value"] is None
assert item["default"] == "low"
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable (the off switch, A2/A4 — the
# API stores as-is; no allowlist, no value validation): the row
# holds the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``low`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == "low"
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_thinking_level == "low"
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
tests/integration/test_admin_settings_live.py
+45 -1
View File
@@ -9,7 +9,10 @@ per-use read, ``app/api/chat.py``'s ``plan_turn`` gate), the shared
snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
the ``scripts/``-consumer surface (``import_extension_set``).
the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_thinking_level`` — the value the LLM
layer reads per request via ``get_settings()``; the off switch is
live-settable, A2).
The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` —
@@ -160,6 +163,47 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
assert llm.settings.llm_chat_model == "gpt-second"
# ---------------------------------------------------------------------------
# the thinking level: save/reset resolve the live value (phase 172, task 02)
# ---------------------------------------------------------------------------
def test_llm_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the thinking level follows the overlay
live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE
call after the write) resolves
``get_settings().llm_thinking_level`` WITHOUT a restart (the proof
the LLM layer — task 03 reads the setting per request via
``get_settings()`` — picks the override up live), the EMPTY value
resolves live too (the off switch is live-settable, A2/A4), and a
reset (the DELETE) returns the env value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
_save(db, "llm_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_thinking_level", "")
invalidate_settings()
assert get_settings().llm_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
# ---------------------------------------------------------------------------
# the chat concurrency semaphore follows the saved cap
# ---------------------------------------------------------------------------
tests/unit/test_chat_gate.py
+47 -7
View File
@@ -1547,22 +1547,33 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
``suggested=N`` immediately AFTER ``summary_hits=N`` — the field
order of every existing field is untouched (the phase-114
``retries=N scaffold_stripped=N`` tail stays last). The value is the
seeded suggestion tier's size (``len(plan.suggested_docs)``)."""
seeded suggestion tier's size (``len(plan.suggested_docs)``).
Phase 172 (A6): the line gains ``thinking=<level>`` immediately
BEFORE ``thinking_chars=N`` — the effective (overlay-resolved)
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off
switch)."""
_session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)]
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever(chunks))
with caplog.at_level("INFO", logger="app.chat"):
_ask(client, "How is my Kubernetes cluster set up?")
def _last_log_line() -> str:
caplog.clear()
with caplog.at_level("INFO", logger="app.chat"):
_ask(client, "How is my Kubernetes cluster set up?")
log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert log_lines
return log_lines[-1]
log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert log_lines
line = log_lines[-1]
# The default (the fixture's settings carry the field default):
# thinking=low, immediately before thinking_chars.
line = _last_log_line()
assert "thinking=low thinking_chars=" in line
# The new slot: suggested=N right after summary_hits=N (one suggested
# doc — the single fixture doc — and one summary hit on it).
assert "summary_hits=1 suggested=1" in line
# The full field order (the phase-114 tail stays last).
# The full field order (the phase-114 tail stays last; phase 172
# inserts thinking= between sources= and thinking_chars=).
order = (
"question=",
"embed_ms=",
@@ -1576,6 +1587,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
"threshold=",
"deflected=",
"sources=",
"thinking=",
"thinking_chars=",
"tool_calls=",
"rounds=",
@@ -1589,3 +1601,31 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
pos = line.find(field)
assert pos > idx, f"{field} out of order in the per-turn log line"
idx = pos
# The effective value: a non-default setting rides the line as-is.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=high thinking_chars=" in line
# The off switch: an empty setting logs ``-`` ...
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
# ... and so does a whitespace-only one (A6: the TRIMMED value
# decides, the log field mirrors the wire's trim).
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
tests/unit/test_config.py
+28
View File
@@ -105,6 +105,34 @@ def test_llm_summary_model_env_override(monkeypatch) -> None:
assert s.llm_summary_model == "mini"
def test_llm_thinking_level_default_is_low(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 172: the model's thinking level defaults to ``low``
(``BOR_LLM_THINKING_LEVEL`` unset)."""
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_thinking_level == "low"
def test_llm_thinking_level_env_override(monkeypatch) -> None:
"""Phase 172 (A4): free-form like ``llm_chat_model`` —
``BOR_LLM_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_thinking_level == "high"
def test_llm_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 172 (A2): the EMPTY value is the off switch — it must not
be coerced back to the default (trimming happens only at the wire,
task 03; no normalization at load)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "")
s = _settings()
assert s.llm_thinking_level == ""
def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 30: document content sent to the ``lite`` model is capped at
``BOR_SUMMARY_MAX_CHARS`` (default 12 000 chars per call)."""
tests/unit/test_llm_client.py
+118
View File
@@ -1324,6 +1324,124 @@ def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch)
assert sleeps == [1.25, 1.25, 1.25]
# ---------- thinking level passthrough (phase 172, task 03) ----------
def test_chat_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): a non-empty level rides the one-shot request as
the top-level ``reasoning_effort`` parameter via the SDK's
``extra_body``."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low"
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_default_level_low_is_always_sent() -> None:
"""The default (``low``) is always sent (A2): a client built without
naming the setting still carries the parameter."""
llm, completions = _make_chat_client(_FakeCompletion("ok"))
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_trims_the_level_before_sending() -> None:
"""TRIM: a hand-edited ``.env`` value with trailing whitespace
(``"low "``) is sent trimmed — the raw stored string never reaches
the wire with surrounding whitespace."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): a whitespace value is the off switch —
``extra_body`` is ABSENT (never None, never ``"off"``) and the
request kwargs keep the pre-phase-172 shape."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level=" "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_stream_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): the chat turn rides the parameter too."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level="high"
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_trims_the_level_before_sending() -> None:
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" high "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): the stream's off switch — ``extra_body`` absent
(never None) and the kwargs dict keeps the pre-phase-172 shape."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=""
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs
assert set(completions.kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_stream_whitespace_level_is_the_off_switch() -> None:
"""Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs
def test_embeddings_body_never_carries_reasoning_effort() -> None:
"""Phase 172 (A3): the embeddings path is untouched — with the level
set, every POST body stays the minimal ``{model, input}`` payload;
``reasoning_effort`` never appears in it."""
service = _FakeEmbeddingsService()
llm, http = _make_client(service, llm_thinking_level="high")
asyncio.run(llm.embed(["a", "b"]))
vec = asyncio.run(llm.embed_one("c"))
assert len(vec) == 768
assert http.bodies # the requests actually happened
for body in http.bodies:
assert set(body) == {"model", "input"}
assert "reasoning_effort" not in body
assert "extra_body" not in body
# ---------- chat_stream_retried (phase 67, task 01) ----------
_RETRY_MSGS: list[dict[str, str]] = [{"role": "user", "content": "q"}]
tests/unit/test_settings_catalog.py
+15
View File
@@ -93,6 +93,21 @@ def test_env_var_is_bare_key_upper_prefixed() -> None:
assert env_var(key).startswith("BOR_")
def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None:
"""Phase 172 (task 01): the new ``llm_thinking_level`` pair sits in
``LLM (aipi)`` with the ``Thinking level`` label and is plain —
not ``secret`` (a free-form text value, like ``llm_chat_model``),
not ``restart_required`` (the LLM client reads the setting per
request via ``get_settings()`` — live), overridable (the overlay is
generic over ``str`` fields)."""
meta = CATALOG["llm_thinking_level"]
assert meta.group == "LLM (aipi)"
assert meta.label == "Thinking level"
assert "llm_thinking_level" not in SECRETS
assert "llm_thinking_level" not in RESTART_REQUIRED
assert "llm_thinking_level" not in NON_OVERRIDABLE
def test_labels_nonempty_and_unique() -> None:
labels = [meta.label for meta in CATALOG.values()]
assert all(label.strip() for label in labels), "empty label in the catalogue"