diff --git a/.agents/phases/todo/172_thinking_level/00_phase.md b/.agents/phases/complete/172_thinking_level/00_phase.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/00_phase.md rename to .agents/phases/complete/172_thinking_level/00_phase.md diff --git a/.agents/phases/todo/172_thinking_level/01_setting_and_env.md b/.agents/phases/complete/172_thinking_level/01_setting_and_env.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/01_setting_and_env.md rename to .agents/phases/complete/172_thinking_level/01_setting_and_env.md diff --git a/.agents/phases/todo/172_thinking_level/02_admin_row_contract.md b/.agents/phases/complete/172_thinking_level/02_admin_row_contract.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/02_admin_row_contract.md rename to .agents/phases/complete/172_thinking_level/02_admin_row_contract.md diff --git a/.agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md b/.agents/phases/complete/172_thinking_level/03_llm_extra_body_passthrough.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md rename to .agents/phases/complete/172_thinking_level/03_llm_extra_body_passthrough.md diff --git a/.agents/phases/todo/172_thinking_level/04_per_turn_log_field.md b/.agents/phases/complete/172_thinking_level/04_per_turn_log_field.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/04_per_turn_log_field.md rename to .agents/phases/complete/172_thinking_level/04_per_turn_log_field.md diff --git a/.agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md b/.agents/phases/complete/172_thinking_level/05_e2e_admin_settings_row.md similarity index 100% rename from .agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md rename to .agents/phases/complete/172_thinking_level/05_e2e_admin_settings_row.md diff --git a/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md new file mode 100644 index 0000000..9e5fc82 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md @@ -0,0 +1,9 @@ +All verification complete. Final report: + +**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.** + +- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=` (`-` when empty) before `thinking_chars`. +- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings. +- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands). +- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol. +- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success). diff --git a/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.validate new file mode 100644 index 0000000..d88500e --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 59%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +.................................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6400 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md new file mode 100644 index 0000000..9472214 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md @@ -0,0 +1,22 @@ +Task 01 complete — everything green. + +**Implemented** +- `app/config.py`: `llm_thinking_level: str = "low"` with house docstring (env var, default, `reasoning_effort` wire mapping, empty = off switch), placed after `llm_summary_model` before `stream_thinking`; no validator/coercion (A4/A2) +- `.env.example`: `BOR_LLM_THINKING_LEVEL=low` with one-line comment, directly under `BOR_LLM_CHAT_MODEL=turbo` (L60→L61) +- `app/core/settings_catalog.py`: `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` after the `llm_summary_model` entry — plain (not secret, not restart-required, overridable) +- Tests: 3 new cases in `tests/unit/test_config.py` (default `low`, env override `high`, empty env stays `""`); 1 new case in `tests/unit/test_settings_catalog.py` (group/label + not in the derived SECRETS/RESTART_REQUIRED/NON_OVERRIDABLE sets) + +**Results** +- `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` → 5 passed +- `uv run pytest` → 4073 passed (6 initial failures were only the DB being down; `podman compose up -d db` → all green) +- `uv run pytest --cov=app --cov-report=term-missing` → 99% on `app/` (>90% gate) +- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings +- `GET /api/admin/settings` verified via TestClient: row present in `LLM (aipi)`, `str`, effective `low` (source `default`), overridable, not secret/restart + +**Docs** +- `README.md`: added `BOR_LLM_THINKING_LEVEL` row to the LLM env-var table; `AGENTS.md`: no doc impact (no env list there) + +**Decisions** +- None beyond the phase design; `settings.js` untouched (A5 — the row falls out of the catalogue) + +**Next pending task:** `02_admin_row_contract.md` diff --git a/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.validate new file mode 100644 index 0000000..5a63592 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 30%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 76%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 83%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 90%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 98%] +......................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6393 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md new file mode 100644 index 0000000..9aad983 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md @@ -0,0 +1,15 @@ +**Report — phase 172, task 02 (admin row API contract for `llm_thinking_level`)** + +Implemented (tests only, per task — no `app/` code touched): +- `tests/integration/test_admin_settings_api.py::test_llm_thinking_level_row_contract` — (a) GET: present, group `LLM (aipi)`, `str`, `env_var` `BOR_LLM_THINKING_LEVEL`, effective `low`, not secret/restart-required; (b) PUT `"high"` → 200, row stored **raw** in `app_settings`, live effective `high`; (c) PUT `""` → 200 storable (off switch, no allowlist); (d) DELETE → row gone, back to `low`; (e) non-admin 403 asserted once (full matrix already pinned). Comment notes the shape/coverage tests pin presence generically (suite is not key-parameterized). +- `tests/integration/test_admin_settings_live.py::test_llm_thinking_level_live_apply_no_restart` — house `_save` + `invalidate_settings()` pattern: `high` live without restart, empty resolves live (A2/A4), DELETE resets to `low`; module docstring updated. +- Deterministic base in both: `monkeypatch.delenv` + code default `low` (kill-switch test idiom, machine-independent). + +Gates (all green): +- `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` → 20 passed +- `uv run pytest --cov=app --cov-report=term-missing` → 4075 passed, app/ coverage 99% (>90%) +- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings + +Docs: no impact — test-only task; the env var is documented in `.env.example` (task 01), and README/AGENTS.md don't enumerate individual settings. + +Next pending task: `03_llm_extra_body_passthrough.md`. diff --git a/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.validate new file mode 100644 index 0000000..96b786d --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 30%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 53%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 60%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 67%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 83%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 90%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 97%] +........................................................................ [ 98%] +........................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 246 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6393 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md new file mode 100644 index 0000000..504099b --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md @@ -0,0 +1,14 @@ +Task 03 complete — all gates green. + +**Report** + +- **Implemented (task 03 — wire passthrough):** + - `_thinking_extra_body()` module-level helper in `app/rag/llm.py`: trimmed non-empty level → `{"reasoning_effort": }`; empty/whitespace → `None` (off switch) + - `chat()` (via `_chat_once`): `extra_body` spread into `create()` **only when present** — key absent (never `None`) when off; pyright-safe typed `dict[str, Any]` + - `chat_stream()`: `kwargs["extra_body"]` set after the `tools` block, same absent-when-off pattern + - Module docstring: one sentence on the phase-172 wire contract (empty = absent, embeddings never carry it) + - 9 new unit tests in `tests/unit/test_llm_client.py`: `chat()` low/default/whitespace, `chat_stream()` high/empty/whitespace, TRIM (`"low "` → `"low"`), byte-identical kwargs-shape pins for the off switch, and the embeddings POST body never containing `reasoning_effort` +- **Results:** `uv run pytest --cov=app` → **4084 passed**, 99% on `app/` (>90% gate); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors; `uv run pytest tests/unit/test_llm_client.py -v --no-cov` → 81 passed +- **Docs:** no doc impact — README L1156 already documents `BOR_LLM_THINKING_LEVEL` incl. the wire mapping (task 01's doc work); AGENTS.md unaffected +- **Decisions:** task-spec inline `**` spread failed pyright (empty-dict inference, 33 errors) — split into an annotated `extra_kwargs: dict[str, Any]`, semantics identical (key absent when off) +- **Next pending task:** `04_per_turn_log_field.md` diff --git a/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.validate new file mode 100644 index 0000000..d88500e --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 59%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +.................................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6400 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md new file mode 100644 index 0000000..cde3a7d --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md @@ -0,0 +1,14 @@ +All gates green. The change is a log-line-only observability field — `README.md` documents the `thinking` SSE events (untouched UI surface) but not the per-turn line's field list, and `AGENTS.md` rule 10 refers to the line generically, so no doc edits are needed. + +**Report — task 04 (per-turn log field)** + +- `app/api/chat.py`: per-turn log line gains `thinking=%s` immediately before `thinking_chars=%d`, argument `settings.llm_thinking_level.strip() or "-"` (effective overlay-resolved value; existing fields un-reordered, `query_log` row untouched) +- `app/api/chat.py`: one-line module-docstring note in the Thinking paragraph (house pattern) — `thinking=` (effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172, A6) +- `tests/unit/test_chat_gate.py`: extended the existing per-turn-line field-order test — asserts `thinking=low` (default), `thinking=high` (effective non-default), `thinking=-` (empty and whitespace-only, pinning the trim), adjacency to `thinking_chars`, and `thinking=` inserted into the full field-order tuple +- `uv run pytest --cov=app` → 4084 passed, `app/` coverage 99% (>90% gate) +- `uv run ruff check .` → clean; `uv run pyright` → 0 errors +- `grep -rn "thinking=" app/api/chat.py` → exactly once in the format string (L1275); no `query_log` schema change +- Docs: no README/AGENTS.md impact (log-line field is server-observability only; both files reference the line generically) +- No deviations; A6 honored exactly + +**Next pending task:** `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (dedicated Playwright suite), regression set, full gates, atomic commit. diff --git a/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.validate new file mode 100644 index 0000000..d88500e --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 59%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +.................................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6400 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.err b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md new file mode 100644 index 0000000..ba8913b --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md @@ -0,0 +1,12 @@ +All gates green. TODO.md is already cleared by the conversion commit (unmodified), so the phase commit is phase files + code only, as the task anticipated. The commit itself and the `todo/ → complete/` move are the harness's to make at phase end (per protocol, I leave every change in the working tree). + +## Report + +- **Implemented:** `tests/e2e/test_thinking_level.py` — the phase's dedicated Playwright suite (3 tests), mirroring the phase-168 harness (module `_boot_app` with mock LLM + `BOR_SETTINGS_DB=1` opt-in, `BOR_LLM_THINKING_LEVEL=low` pinned to the code default, `app_settings`/`ui_settings` autouse wipe, `.env` mtime guard, `_admin_client`): + - `test_row_renders_free_form_with_default_low` — `LLM (aipi)` band visible; "Thinking level" row with `BOR_LLM_THINKING_LEVEL` env-var display; `type="text"` free-form control (not number/select); `low` + `env` badge; `llm_chat_model` control asserted as the same `type="text"` (one assertion pair) + - `test_saved_level_persists_without_restart` — dirty → save → `DB` badge; custom value `think-soft` saved raw (API effective + raw `app_settings` row); survives reload without restart + - `test_reset_returns_to_env_default` — per-row Reset (DELETE) after a save → status line, `low` + `env` badge, row gone +- **Gates (all green):** `uv run pytest -v` → 4084 passed; `--cov=app` → **99%** (>90%); `uv run ruff check . && uv run pyright` → clean; `tests/e2e/test_thinking_level.py --no-cov` → 3 passed in isolation; regression `test_admin_settings_panel.py` → 12 passed, `test_admin_settings_layout.py` → 5 passed (both in isolation) +- **Docs:** no doc changes needed — `README.md` already carries the `BOR_LLM_THINKING_LEVEL` row (tasks 01–04); AGENTS.md has no per-setting content +- **Decisions:** the module app pins `BOR_LLM_THINKING_LEVEL=low` (env set → `env` badge, house leak-guard) and `BOR_LLM_CHAT_MODEL` to its code default; no KB seeding (A6 — the settings surface only, per-turn line is task 04's integration contract); commit + phase-dir move deferred to the harness +- **Next pending task:** none — task 05 is the phase's final task; phase 172 is complete diff --git a/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md.status b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.validate b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.validate new file mode 100644 index 0000000..d88500e --- /dev/null +++ b/.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.validate @@ -0,0 +1,134 @@ +........................................................................ [ 1%] +........................................................................ [ 3%] +........................................................................ [ 5%] +........................................................................ [ 7%] +........................................................................ [ 8%] +........................................................................ [ 10%] +........................................................................ [ 12%] +........................................................................ [ 14%] +........................................................................ [ 15%] +........................................................................ [ 17%] +........................................................................ [ 19%] +........................................................................ [ 21%] +........................................................................ [ 22%] +........................................................................ [ 24%] +........................................................................ [ 26%] +........................................................................ [ 28%] +........................................................................ [ 29%] +........................................................................ [ 31%] +........................................................................ [ 33%] +........................................................................ [ 35%] +........................................................................ [ 37%] +........................................................................ [ 38%] +........................................................................ [ 40%] +........................................................................ [ 42%] +........................................................................ [ 44%] +........................................................................ [ 45%] +........................................................................ [ 47%] +........................................................................ [ 49%] +........................................................................ [ 51%] +........................................................................ [ 52%] +........................................................................ [ 54%] +........................................................................ [ 56%] +........................................................................ [ 58%] +........................................................................ [ 59%] +........................................................................ [ 61%] +........................................................................ [ 63%] +........................................................................ [ 65%] +........................................................................ [ 66%] +........................................................................ [ 68%] +........................................................................ [ 70%] +........................................................................ [ 72%] +........................................................................ [ 74%] +........................................................................ [ 75%] +........................................................................ [ 77%] +........................................................................ [ 79%] +........................................................................ [ 81%] +........................................................................ [ 82%] +........................................................................ [ 84%] +........................................................................ [ 86%] +........................................................................ [ 88%] +........................................................................ [ 89%] +........................................................................ [ 91%] +........................................................................ [ 93%] +........................................................................ [ 95%] +........................................................................ [ 96%] +........................................................................ [ 98%] +.................................................... [100%] +=============================== warnings summary =============================== +.venv/lib/python3.13/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.13.13-final-0 _______________ + +Name Stmts Miss Cover +-------------------------------------------------- +app/__init__.py 1 0 100% +app/api/__init__.py 0 0 100% +app/api/admin_settings.py 108 0 100% +app/api/auth.py 52 0 100% +app/api/chat.py 280 1 99% +app/api/chat_images.py 50 0 100% +app/api/chats.py 110 0 100% +app/api/config.py 13 0 100% +app/api/doc_drafts.py 99 0 100% +app/api/docs.py 234 1 99% +app/api/git_sources.py 259 0 100% +app/api/health.py 10 0 100% +app/api/steering.py 42 0 100% +app/api/suggestions.py 33 0 100% +app/api/sync.py 204 0 100% +app/api/tokens.py 40 0 100% +app/api/ui_settings.py 55 0 100% +app/config.py 286 0 100% +app/core/__init__.py 0 0 100% +app/core/auth.py 45 0 100% +app/core/caching.py 124 0 100% +app/core/compression.py 59 0 100% +app/core/debugging.py 29 2 93% +app/core/docs_push.py 39 0 100% +app/core/errors.py 5 0 100% +app/core/http_cache.py 22 0 100% +app/core/logging.py 13 0 100% +app/core/rate_limit.py 44 0 100% +app/core/response_budget.py 63 0 100% +app/core/security_headers.py 20 0 100% +app/core/settings_catalog.py 18 0 100% +app/core/settings_overlay.py 111 0 100% +app/core/theming.py 38 0 100% +app/core/tokens.py 44 0 100% +app/db.py 22 0 100% +app/main.py 86 0 100% +app/models.py 141 0 100% +app/rag/__init__.py 0 0 100% +app/rag/agent.py 454 1 99% +app/rag/archive_upload.py 134 0 100% +app/rag/chunker.py 220 4 98% +app/rag/doc_dates.py 18 0 100% +app/rag/folder_summaries.py 134 0 100% +app/rag/git_sources.py 77 0 100% +app/rag/importer.py 841 4 99% +app/rag/llm.py 253 1 99% +app/rag/md_images.py 63 0 100% +app/rag/overview.py 78 0 100% +app/rag/pdf.py 192 0 100% +app/rag/prompts.py 156 0 100% +app/rag/retriever.py 252 3 99% +app/rag/scaffolding.py 55 0 100% +app/rag/source_removal.py 41 0 100% +app/rag/sources_meta.py 16 0 100% +app/rag/suggestions.py 27 0 100% +app/rag/summarizer.py 58 0 100% +app/schemas.py 532 0 100% +-------------------------------------------------- +TOTAL 6400 17 99% +coverage gate: app/ 99% (>90%) OK +All checks passed! +0 errors, 0 warnings, 0 informations +WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414). +Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` + +validation OK diff --git a/.env.example b/.env.example index 5def59e..1996c2f 100644 --- a/.env.example +++ b/.env.example @@ -58,6 +58,7 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1 BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed" BOR_LLM_CHAT_MODEL=turbo +BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent # BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off # BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120) # BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls) diff --git a/README.md b/README.md index 24cc8f1..205965c 100644 --- a/README.md +++ b/README.md @@ -1153,6 +1153,7 @@ never the plaintext). | `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` | | `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model | | `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries | +| `BOR_LLM_THINKING_LEVEL` | `low` | The model's thinking level (pi-style, free-form — no allowlist); rides every chat-completions request as `reasoning_effort`; empty = the parameter is not sent | | `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) | | `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off | | `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries | diff --git a/app/api/chat.py b/app/api/chat.py index 4e1f9dd..bf99d7c 100644 --- a/app/api/chat.py +++ b/app/api/chat.py @@ -13,7 +13,10 @@ model's reasoning arrives ahead of the answer and is streamed as ``thinking`` events before the ``delta`` events of the same turn. Each turn's thinking is counted in the per-turn log line (``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the -``thinking`` frames (the pieces are still counted). +``thinking`` frames (the pieces are still counted). The per-turn log +line records ``thinking=`` (the effective +``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before +``thinking_chars`` (phase 172, A6). Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when ``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0`` @@ -1269,8 +1272,8 @@ async def chat( "question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d " "suggested=%d tuning=%d " "kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r " - "thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d " - "retries=%d scaffold_stripped=%d", + "thinking=%s thinking_chars=%d tool_calls=%d rounds=%d " + "cap_hit=%d total_ms=%d retries=%d scaffold_stripped=%d", request.message, embed_ms, plan.top_score, @@ -1283,6 +1286,7 @@ async def chat( settings.relevance_threshold, plan.deflected, source_paths, + settings.llm_thinking_level.strip() or "-", # phase 172 (A6) thinking_chars, holder.tool_calls, holder.rounds, diff --git a/app/config.py b/app/config.py index 33d0c83..04f203a 100644 --- a/app/config.py +++ b/app/config.py @@ -105,6 +105,14 @@ class Settings(BaseSettings): #: Served by the same OpenAI-compatible endpoint — no new model #: management. Called via ``LLMClient.chat()``. llm_summary_model: str = "lite" + #: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) — + #: pi-style reasoning effort, free-form text exactly like + #: ``llm_chat_model`` (no allowlist — the endpoint is the source of + #: truth). Default ``low``. Rides every chat-completions request to + #: aipi as the top-level ``reasoning_effort`` parameter + #: (``app/rag/llm.py``); the EMPTY value is the off switch — the + #: parameter is not sent at all. + llm_thinking_level: str = "low" #: Operator kill-switch for the ``thinking`` SSE events (phase 17, #: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking #: pieces are still counted for the per-turn log line but never diff --git a/app/core/settings_catalog.py b/app/core/settings_catalog.py index 2d6274a..b20852a 100644 --- a/app/core/settings_catalog.py +++ b/app/core/settings_catalog.py @@ -150,6 +150,9 @@ CATALOG: dict[str, SettingMeta] = { "llm_summary_model": SettingMeta( "llm_summary_model", "LLM (aipi)", "LLM summary model" ), + "llm_thinking_level": SettingMeta( + "llm_thinking_level", "LLM (aipi)", "Thinking level" + ), "stream_thinking": SettingMeta( "stream_thinking", "LLM (aipi)", "Stream thinking" ), diff --git a/app/rag/llm.py b/app/rag/llm.py index ab5b205..50da49c 100644 --- a/app/rag/llm.py +++ b/app/rag/llm.py @@ -10,7 +10,13 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm wire convention, verified live 2026-08-23) **before** the answer's ``delta.content`` chunks, and reasoning counts against ``max_tokens`` -(an answer can in principle be empty). +(an answer can in principle be empty). The operator's thinking level +(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every +chat-completions request (one-shot ``chat`` and streaming ``chat_stream``) +as the top-level ``reasoning_effort`` parameter (via the SDK's +``extra_body``); an empty value is the off switch — the parameter is +absent, the request is byte-identical to pre-phase-172, and the +embeddings path never carries it. Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768 dimensions when the table is created, so a model that returns a different @@ -40,6 +46,17 @@ if TYPE_CHECKING: logger = logging.getLogger("app.llm") +def _thinking_extra_body(settings: Settings) -> dict[str, str] | None: + """The phase-172 thinking level as the chat-completions extra body. + + Non-empty (trimmed) value → ``{"reasoning_effort": }`` (the + OpenAI-standard top-level parameter, A1); empty/whitespace → ``None`` + (the off switch — the parameter is ABSENT from the request, A2). + """ + level = settings.llm_thinking_level.strip() + return {"reasoning_effort": level} if level else None + + class EmbeddingError(RuntimeError): """The embeddings endpoint failed (network, HTTP, or malformed reply).""" @@ -412,6 +429,10 @@ class LLMClient: ``finish_reason``) for an empty reply — the one failure class :meth:`chat` retries — and :class:`LLMError` for everything else. """ + extra = _thinking_extra_body(self.settings) + extra_kwargs: dict[str, Any] = ( + {"extra_body": extra} if extra is not None else {} + ) try: resp = await self._client.chat.completions.create( model=model, @@ -419,6 +440,9 @@ class LLMClient: temperature=0.2, max_tokens=2048, stream=False, + # Phase 172 (A1/A2): the thinking level rides as + # reasoning_effort; the key is absent (never None) when off. + **extra_kwargs, ) except LLMError: raise @@ -598,6 +622,10 @@ class LLMClient: } if tools is not None: kwargs["tools"] = tools + if (extra := _thinking_extra_body(self.settings)) is not None: + # Phase 172 (A1/A2): the thinking level rides as + # reasoning_effort; the key is absent (never None) when off. + kwargs["extra_body"] = extra stream: AsyncStream[ChatCompletionChunk] | None = None try: stream = cast( diff --git a/tests/e2e/test_thinking_level.py b/tests/e2e/test_thinking_level.py new file mode 100644 index 0000000..80810ca --- /dev/null +++ b/tests/e2e/test_thinking_level.py @@ -0,0 +1,405 @@ +"""Phase 172 E2E (Playwright): the Thinking level row in the admin Settings +panel — the owner's acceptance for a pi-style, operator-tunable LLM +reasoning knob, proven in a real browser. + +Source: ``TODO.md`` L3 — "Brain of Reese needs to support thinking levels +like pi does. should be a free-form text entry like chat model. Default +should be 'low'. Update .env and admin settings." + +Run in isolation (DB must be up: ``podman compose up -d db``): + + uv run pytest tests/e2e/test_thinking_level.py -v --no-cov + +Test → contract mapping (one story, one phase, one isolated file): + +1. ``test_row_renders_free_form_with_default_low`` — the row: the + ``LLM (aipi)`` group band is visible (the phase-171 band + contract), a row labelled **Thinking level** with the + ``BOR_LLM_THINKING_LEVEL`` env-var display (the catalogue + pattern) is present, its input is a **free-form text** input + (``type="text"`` — NOT ``number``/``select``, the A4 + contract), it shows ``low`` with the ``env`` badge, and the + ``llm_chat_model`` row's input is the same ``type="text"`` + control (the "like chat model" reference, in one assertion + pair). +2. ``test_saved_level_persists_without_restart`` — the save + (the phase-168 §7.4 Save lifecycle: dirty → save → ``DB`` + badge): ``high`` is typed (the dirty hint appears), then the + free-form custom value ``think-soft`` is typed once (no + allowlist — A4) and saved; the effective value reads + ``think-soft`` (the API catalogue AND the raw + ``app_settings`` row), and a reload of ``/settings.html`` + still shows ``think-soft`` with the ``DB`` badge (no + restart, the phase-168 contract). +3. ``test_reset_returns_to_env_default`` — the per-row Reset + (the per-row ``DELETE``): after a save, the Reset is live; + clicking it lands the status line and the value reads + ``low`` again with the ``env`` badge (the overlay reset, + task 02's contract, in the browser). + +DB isolation: the shared e2e Postgres keeps ``app_settings`` +(the overlay rows — a leftover row would retune every later +suite's app through the conftest pin's failure mode), +``ui_settings`` (the single row the caching middleware reads +for EVERY served page) and ``api_tokens`` rows across suites. +An autouse fixture truncates ``app_settings`` and +``ui_settings`` and deletes the ``e2e-``-labeled tokens before +AND after every test (never a TRUNCATE on ``api_tokens`` — the +shared DB may hold the owner's real tokens); the module teardown +pins ``app_settings`` empty. The module app opts BACK IN over +the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_ +SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in +pattern) and pins ``BOR_LLM_THINKING_LEVEL`` (and the +"like chat model" reference, ``BOR_LLM_CHAT_MODEL``) to their +CODE DEFAULTS — the env var is set (so the badge reads ``env``, +not ``default``) and an operator's local ``.env`` cannot leak a +different level into the suite's app (the house leak-guard +pattern). A module-scope ``.env`` mtime guard pins LOCKED A1: +no code path the suite drives opens the env file for writing. + +ASSUMPTION A6 (phase-locked): the per-turn-line assertion +(``thinking=``) lives in the unit/integration layer +(task 04) — the mock LLM turn does not exercise the wire, and +this suite proves the settings surface only. +""" +from __future__ import annotations + +import json +import os +import subprocess +import sys +from collections.abc import Iterator +from typing import Any + +import httpx +import pytest +from playwright.sync_api import Page, expect +from sqlalchemy import text + +from app.config import Settings +from app.db import SessionLocal +from e2e.auth_helpers import login +from e2e.conftest import ( + ADMIN_PASSWORD, + APP_PORT, + REPO, + SESSION_SECRET, + USE_REAL_LLM, + _wait_http, +) + +APP_URL = f"http://127.0.0.1:{APP_PORT}" + +#: The save's custom free-form value (test 2) — NOT a known pi +#: level (proves the A4 contract: no allowlist, the endpoint is +#: the source of truth). +SAVED_LEVEL = "think-soft" + + +# --------------------------------------------------------------------------- +# Per-module app env (the phase-168 tuning pattern) +# --------------------------------------------------------------------------- + + +def _boot_app() -> subprocess.Popen[bytes]: + """The one module app: the shared env block + the phase-172 pins, + with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``).""" + global _MOCK_PORT + env = dict(os.environ) + env.pop("DEBUGPY", None) + env["BOR_ENVIRONMENT"] = "e2e" + env["BOR_STATIC_DIR"] = str(REPO / "frontend") + env["BOR_LLM_BASE_URL"] = ( + "https://aipi.reeseapps.com/v1" + if USE_REAL_LLM + else f"http://127.0.0.1:{_MOCK_PORT}/v1" + ) + # The E2E-calibrated gate pins (the conftest pattern — the mock's + # token-overlap distribution, explicit so an operator's local + # .env cannot leak a different gate into the suite's app). + env["BOR_RELEVANCE_THRESHOLD"] = "0.30" + env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15" + env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15" + env["BOR_LLM_RETRY_DELAY"] = "0" + env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default) + env.setdefault( + "BOR_DATABASE_URL", + "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese", + ) + # Phase 16: admin auth must be set or create_app() refuses to boot. + env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD + env["BOR_SESSION_SECRET"] = SESSION_SECRET + env["BOR_DOCS_REPO"] = "" + env["BOR_SUGGESTIONS"] = json.dumps( + Settings.model_fields["suggestions"].default + ) + # Leak guards: the branding base and the "like chat model" + # reference row stay the CODE defaults (derived from the class + # fields, never drift from app/config.py). + env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default + env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default + env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default + env["BOR_GIT_SOURCES"] = "" + env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default + # Phase 168 (the panel suite's pattern): the module opts BACK IN + # over the conftest's global BOR_SETTINGS_DB=0 pin (the + # test_pictorial_pdf.py pattern) — the row's save/reset + # contract needs the overlay on. + env["BOR_SETTINGS_DB"] = "1" + # Phase 172 (task 05): pin the thinking level to the CODE + # DEFAULT — the env var is SET (so the badge reads ``env``, not + # ``default``) and an operator's local .env cannot leak a + # different level into the suite's app. + env["BOR_LLM_THINKING_LEVEL"] = Settings.model_fields[ + "llm_thinking_level" + ].default + return subprocess.Popen( + [sys.executable, "-m", "uvicorn", "app.main:app", + "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"], + cwd=REPO, + env=env, + ) + + +def _stop_app(proc: subprocess.Popen[bytes]) -> None: + proc.terminate() + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() + + +@pytest.fixture(scope="module") +def app_server(mock_llm: int) -> Iterator[str]: + """The real app under test — per-module env: the thinking level + pinned to the code default ``low`` and ``BOR_SETTINGS_DB=1`` + (the module opts back in over the conftest pin).""" + global _MOCK_PORT + _MOCK_PORT = mock_llm + proc = _boot_app() + try: + _wait_http(f"{APP_URL}/api/health") + yield APP_URL + finally: + _stop_app(proc) + + +@pytest.fixture(scope="module") +def app_url(app_server: str) -> str: + return app_server + + +# --------------------------------------------------------------------------- +# DB isolation + helpers +# --------------------------------------------------------------------------- + + +def _clean_settings_state() -> None: + """Fresh settings + theme + token state per test: truncate + ``app_settings`` (a leftover row is the cross-suite poison this + phase's overlay contract exists to prevent — the new key's rows + are wiped between tests) and the single-row ``ui_settings`` + (the middleware reads it for EVERY page), and delete this + suite's issued tokens (label-scoped on ``e2e-`` — never a + TRUNCATE: the shared DB may hold the owner's real tokens).""" + with SessionLocal() as db: + db.execute(text("TRUNCATE app_settings")) + db.execute(text("TRUNCATE ui_settings")) + db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'")) + db.commit() + + +@pytest.fixture(autouse=True) +def _clean(db_ready: None) -> Iterator[None]: + _clean_settings_state() + yield + _clean_settings_state() + # The module's final state: app_settings EMPTY (the cross-suite + # poison guard — the next suite's app must boot with no + # overrides). + with SessionLocal() as db: + assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0 + + +@pytest.fixture(scope="module", autouse=True) +def _env_never_written() -> Iterator[None]: + """LOCKED A1, mtime-pinned: no code path the suite drives opens + ``.env`` for writing (the file is gitignored — a ``git status`` + check cannot see it, so ``stat``'s mtime is the assertion).""" + env_file = REPO / ".env" + before = env_file.stat().st_mtime_ns if env_file.exists() else None + yield + after = env_file.stat().st_mtime_ns if env_file.exists() else None + assert before == after, "the app under test must never write .env" + + +def _cookies(page: Page) -> dict[str, str]: + """The session cookies the browser context holds (the test's API + side sees exactly what that browser sees).""" + return { + c["name"]: c["value"] + for c in page.context.cookies() + if "name" in c and "value" in c + } + + +def _admin_client(app_url: str) -> httpx.Client: + """The API side of the form login (the 204 + the signed session + cookie in the jar).""" + client = httpx.Client(timeout=60.0) + r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD}) + assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}" + return client + + +def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]: + r = client.get(f"{app_url}/api/admin/settings", timeout=30.0) + assert r.status_code == 200, r.text[:200] + for item in r.json()["items"]: + if item["key"] == key: + return item + raise AssertionError(f"{key} missing from the catalogue") + + +# --------------------------------------------------------------------------- +# 1. The row renders free-form with default low (the "like chat +# model" reference, in one assertion pair) +# --------------------------------------------------------------------------- + + +def test_row_renders_free_form_with_default_low(page: Page, app_url: str, db_ready: None) -> None: + page.set_default_timeout(30_000) + login(page, app_url, next="/settings.html") + expect(page.locator("#settings-content")).to_be_visible(timeout=15_000) + expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on + + # The LLM (aipi) group band is visible (the phase-171 band + # contract — the row sits in it)… + expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible() + # …and the row is present with its label… + row = page.locator('tr[data-key="llm_thinking_level"]') + expect(row).to_be_visible() + expect(row.locator(".cell-clip")).to_have_text("Thinking level") + # …and the env-var display (the catalogue pattern). + expect(row.locator(".settings-env")).to_have_text("BOR_LLM_THINKING_LEVEL") + + # The input is a FREE-FORM TEXT control (A4: type="text" — NOT + # number/select; no allowlist, the endpoint is the source of + # truth). + assert row.locator(".settings-control").get_attribute("type") == "text" + # The "like chat model" reference: the llm_chat_model row's + # input is the SAME type="text" control (one assertion pair). + thinking_type = row.locator(".settings-control").get_attribute("type") + chat_type = page.locator( + 'tr[data-key="llm_chat_model"] .settings-control' + ).get_attribute("type") + assert thinking_type == chat_type == "text" + + # The default: the env-pinned ``low`` with the env badge (the + # env var is set — no app_settings row, no code-default badge). + expect(row.locator(".settings-control")).to_have_value("low") + expect(row.locator(".settings-src")).to_have_text("env") + # Nothing is saved yet: the row's Reset is disabled (no saved + # override — the phase-168 render contract). + expect(row.locator(".settings-reset")).to_be_disabled() + + +# --------------------------------------------------------------------------- +# 2. The save: dirty → save → DB badge, the effective value is the +# free-form custom, and it survives a reload WITHOUT a restart +# --------------------------------------------------------------------------- + + +def test_saved_level_persists_without_restart(page: Page, app_url: str, db_ready: None) -> None: + page.set_default_timeout(30_000) + login(page, app_url, next="/settings.html") + expect(page.locator("#settings-content")).to_be_visible(timeout=15_000) + client = _admin_client(app_url) + + # The §7.4 Save lifecycle: type ``high`` — the dirty hint + # appears (one unsaved change on the row)… + row = page.locator('tr[data-key="llm_thinking_level"]') + row.locator(".settings-control").fill("high") + expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes") + # …then type the free-form custom value once (A4 — NOT a known + # pi level: no allowlist, the endpoint is the source of truth). + row.locator(".settings-control").fill(SAVED_LEVEL) + expect(page.locator("#settings-save")).to_be_enabled() + + # Save: the status line, then the badge flips to DB (the + # re-GET settled the row). + page.click("#settings-save") + expect(page.locator("#settings-status")).to_contain_text( + "Saved 1 settings.", timeout=30_000 + ) + expect(row.locator(".settings-src")).to_have_text("DB") + + # The effective value reads ``think-soft`` — the API catalogue + # (what the LLM layer reads per request via get_settings)… + item = _catalog_item(client, app_url, "llm_thinking_level") + assert item["source"] == "db" and item["value"] == SAVED_LEVEL + # …and the raw app_settings row is stored RAW (the overlay's + # D1 env-native string form). + with SessionLocal() as db: + raw = db.execute( + text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'") + ).scalar_one() + assert raw == SAVED_LEVEL, f"the row must be stored raw (got {raw!r})" + + # No restart: the DB value survives a reload of /settings.html + # (the panel re-reads the settled server state) with the DB + # badge still on. + page.goto(app_url + "/settings.html") + expect(page.locator("#settings-content")).to_be_visible(timeout=15_000) + expect( + page.locator('tr[data-key="llm_thinking_level"] .settings-control') + ).to_have_value(SAVED_LEVEL, timeout=15_000) + expect( + page.locator('tr[data-key="llm_thinking_level"] .settings-src') + ).to_have_text("DB") + + +# --------------------------------------------------------------------------- +# 3. The Reset: control goes back to the environment (the per-row +# DELETE — task 02's contract, in the browser) +# --------------------------------------------------------------------------- + + +def test_reset_returns_to_env_default(page: Page, app_url: str, db_ready: None) -> None: + page.set_default_timeout(30_000) + login(page, app_url, next="/settings.html") + expect(page.locator("#settings-content")).to_be_visible(timeout=15_000) + client = _admin_client(app_url) + + # Establish a saved override first (the Reset is disabled with + # nothing to reset — the phase-168 render contract): save + # ``high`` so the row's Reset goes live… + row = page.locator('tr[data-key="llm_thinking_level"]') + row.locator(".settings-control").fill("high") + page.click("#settings-save") + expect(page.locator("#settings-status")).to_contain_text( + "Saved 1 settings.", timeout=30_000 + ) + expect(row.locator(".settings-src")).to_have_text("DB") + expect(row.locator(".settings-reset")).to_be_enabled() + + # The per-row Reset (the per-row DELETE): the status line… + row.locator(".settings-reset").click() + expect(page.locator("#settings-status")).to_contain_text( + "BOR_LLM_THINKING_LEVEL is back to the environment value.", timeout=30_000 + ) + # …the value reads ``low`` again with the env badge (the env + # var is set — not the code-default badge)… + expect(row.locator(".settings-control")).to_have_value("low") + expect(row.locator(".settings-src")).to_have_text("env") + # …and the server agrees: the override is gone and the + # effective value is the env value again. + item = _catalog_item(client, app_url, "llm_thinking_level") + assert item["source"] == "env" and item["value"] == "low" + with SessionLocal() as db: + assert ( + db.execute( + text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'") + ).scalar() + == 0 + ), "the DELETE must remove the row" diff --git a/tests/integration/test_admin_settings_api.py b/tests/integration/test_admin_settings_api.py index 7c610bb..a7c7094 100644 --- a/tests/integration/test_admin_settings_api.py +++ b/tests/integration/test_admin_settings_api.py @@ -516,3 +516,89 @@ def test_catalog_covers_every_settings_field(admin_client) -> None: assert r.status_code == 200 assert {item["key"] for item in r.json()["items"]} == set(Settings.model_fields) assert set(CATALOG) == set(Settings.model_fields) + + +def test_llm_thinking_level_row_contract( + admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch +) -> None: + """Phase 172 (task 02): the new ``llm_thinking_level`` row behaves + exactly like ``llm_chat_model`` — the machinery is generic over + the catalogue (D4 / A5), so the per-key contract is: (a) present in + the GET body (group ``LLM (aipi)``, ``str``, effective ``low`` + with no row in ``app_settings``, ``env_var`` named); (b) PUT stores + the RAW string in ``app_settings`` (the overlay's env-native + string form) and the response reports the new effective value; + (c) the EMPTY value is storable (the off switch, A2/A4 — the API + stores as-is, no allowlist); (d) DELETE resets to the env value; + (e) the non-admin 403 gate applies (asserted once here; the full + four-route matrix is ``test_auth_matrix_all_four_routes``). + ``test_catalogue_shape_and_sources_on_empty_table`` + + ``test_catalog_covers_every_settings_field`` already pin the row's + PRESENCE generically — these cases pin the per-key behavior. + """ + # A deterministic base (machine-independent, the kill-switch test's + # idiom): the code default, no row in the table. + monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False) + invalidate_settings() + assert get_settings().llm_thinking_level == "low" + + # (a) the row: group, type, env var, effective ``low`` with NO row + # in app_settings, not secret, not restart-required (the LLM layer + # reads the setting per request — live). + r = admin_client.get(API) + assert r.status_code == 200, r.text + item = _item(r.json(), "llm_thinking_level") + assert item["group"] == "LLM (aipi)" + assert item["label"] == "Thinking level" + assert item["type"] == "str" + assert item["env_var"] == "BOR_LLM_THINKING_LEVEL" + assert item["secret"] is False + assert item["restart_required"] is False + assert item["value"] == "low" + assert item["source"] in ("default", "env") # no row in app_settings + assert item["has_override"] is False + assert item["db_value"] is None + assert item["default"] == "low" + + # (b) PUT stores the RAW string (the overlay's D1 — env-native + # string form) and the response reports the new effective value. + r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"}) + assert r.status_code == 200, r.text + assert r.json()["value"] == "high" + assert r.json()["source"] == "db" + assert r.json()["has_override"] is True + assert r.json()["db_value"] == "high" + assert db.execute( + text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'") + ).scalar() == "high" # stored raw, exactly as sent + assert get_settings().llm_thinking_level == "high" # live, no restart + + # (c) the EMPTY value is storable (the off switch, A2/A4 — the + # API stores as-is; no allowlist, no value validation): the row + # holds the empty string and the effective value is ``""``. + r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""}) + assert r.status_code == 200, r.text + assert r.json()["value"] == "" + assert r.json()["has_value"] is False # the effective value is empty + assert db.execute( + text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'") + ).scalar() == "" # stored as-is (the off switch) + assert get_settings().llm_thinking_level == "" + + # (d) DELETE: the row is gone and the effective value is back to + # the env value (``low`` — the base pinned above). + r = admin_client.delete(f"{API}/llm_thinking_level") + assert r.status_code == 200, r.text + assert r.json()["value"] == "low" + assert r.json()["has_override"] is False + assert db.execute( + text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'") + ).scalar() == 0 # the row is gone + assert get_settings().llm_thinking_level == "low" + + # (e) the non-admin 403 gate still applies (asserted once for this + # key — the router-wide dependency is pinned on ALL four routes in + # ``test_auth_matrix_all_four_routes``). + anon = TestClient(fastapi_app) + r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"}) + assert r.status_code == 403 and r.json() == {"detail": "admin only"} diff --git a/tests/integration/test_admin_settings_live.py b/tests/integration/test_admin_settings_live.py index a591510..bde2f15 100644 --- a/tests/integration/test_admin_settings_live.py +++ b/tests/integration/test_admin_settings_live.py @@ -9,7 +9,10 @@ per-use read, ``app/api/chat.py``'s ``plan_turn`` gate), the shared snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart (``BOR_SETTINGS_DB=0`` with rows present → env values are back) and -the ``scripts/``-consumer surface (``import_extension_set``). +the ``scripts/``-consumer surface (``import_extension_set``), and the +phase-172 thinking level (``llm_thinking_level`` — the value the LLM +layer reads per request via ``get_settings()``; the off switch is +live-settable, A2). The rows are inserted directly through the ``db`` session (the admin API is task 04); the save itself is ``invalidate_settings()`` — @@ -160,6 +163,47 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None: assert llm.settings.llm_chat_model == "gpt-second" +# --------------------------------------------------------------------------- +# the thinking level: save/reset resolve the live value (phase 172, task 02) +# --------------------------------------------------------------------------- + + +def test_llm_thinking_level_live_apply_no_restart( + db: Session, monkeypatch: pytest.MonkeyPatch +) -> None: + """Phase 172 (task 02): the thinking level follows the overlay + live — a save (the PUT's write, minus the API: ``_save`` + + ``invalidate_settings()`` is exactly what task 04's PUT/DELETE + call after the write) resolves + ``get_settings().llm_thinking_level`` WITHOUT a restart (the proof + the LLM layer — task 03 reads the setting per request via + ``get_settings()`` — picks the override up live), the EMPTY value + resolves live too (the off switch is live-settable, A2/A4), and a + reset (the DELETE) returns the env value. + """ + # A deterministic base (machine-independent, the kill-switch test's + # idiom): the code default, no row in the table. + monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False) + invalidate_settings() + assert get_settings().llm_thinking_level == "low" + _save(db, "llm_thinking_level", "high") + invalidate_settings() + assert get_settings().llm_thinking_level == "high" # NO restart + # The off switch is live-settable too (A2/A4): the empty value + # resolves through the overlay, not stripped or coerced. + _save(db, "llm_thinking_level", "") + invalidate_settings() + assert get_settings().llm_thinking_level == "" + # The reset returns the env value (no process restart in between — + # the LLM layer re-reads per request, task 03). + db.execute( + text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'") + ) + db.commit() + invalidate_settings() + assert get_settings().llm_thinking_level == "low" + + # --------------------------------------------------------------------------- # the chat concurrency semaphore follows the saved cap # --------------------------------------------------------------------------- diff --git a/tests/unit/test_chat_gate.py b/tests/unit/test_chat_gate.py index 40a35e3..cf7ad2a 100644 --- a/tests/unit/test_chat_gate.py +++ b/tests/unit/test_chat_gate.py @@ -1547,22 +1547,33 @@ def test_endpoint_log_line_records_suggested_after_summary_hits( ``suggested=N`` immediately AFTER ``summary_hits=N`` — the field order of every existing field is untouched (the phase-114 ``retries=N scaffold_stripped=N`` tail stays last). The value is the - seeded suggestion tier's size (``len(plan.suggested_docs)``).""" + seeded suggestion tier's size (``len(plan.suggested_docs)``). + Phase 172 (A6): the line gains ``thinking=`` immediately + BEFORE ``thinking_chars=N`` — the effective (overlay-resolved) + ``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off + switch).""" _session, _llm = gate_env doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT") chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)] monkeypatch.setattr(chat_api, "retrieve", _fake_retriever(chunks)) - with caplog.at_level("INFO", logger="app.chat"): - _ask(client, "How is my Kubernetes cluster set up?") + def _last_log_line() -> str: + caplog.clear() + with caplog.at_level("INFO", logger="app.chat"): + _ask(client, "How is my Kubernetes cluster set up?") + log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()] + assert log_lines + return log_lines[-1] - log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()] - assert log_lines - line = log_lines[-1] + # The default (the fixture's settings carry the field default): + # thinking=low, immediately before thinking_chars. + line = _last_log_line() + assert "thinking=low thinking_chars=" in line # The new slot: suggested=N right after summary_hits=N (one suggested # doc — the single fixture doc — and one summary hit on it). assert "summary_hits=1 suggested=1" in line - # The full field order (the phase-114 tail stays last). + # The full field order (the phase-114 tail stays last; phase 172 + # inserts thinking= between sources= and thinking_chars=). order = ( "question=", "embed_ms=", @@ -1576,6 +1587,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits( "threshold=", "deflected=", "sources=", + "thinking=", "thinking_chars=", "tool_calls=", "rounds=", @@ -1589,3 +1601,31 @@ def test_endpoint_log_line_records_suggested_after_summary_hits( pos = line.find(field) assert pos > idx, f"{field} out of order in the per-turn log line" idx = pos + + # The effective value: a non-default setting rides the line as-is. + monkeypatch.setattr( + chat_api, + "get_settings", + lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue] + ) + line = _last_log_line() + assert "thinking=high thinking_chars=" in line + + # The off switch: an empty setting logs ``-`` ... + monkeypatch.setattr( + chat_api, + "get_settings", + lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue] + ) + line = _last_log_line() + assert "thinking=- thinking_chars=" in line + + # ... and so does a whitespace-only one (A6: the TRIMMED value + # decides, the log field mirrors the wire's trim). + monkeypatch.setattr( + chat_api, + "get_settings", + lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue] + ) + line = _last_log_line() + assert "thinking=- thinking_chars=" in line diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py index db55311..cd670b5 100644 --- a/tests/unit/test_config.py +++ b/tests/unit/test_config.py @@ -105,6 +105,34 @@ def test_llm_summary_model_env_override(monkeypatch) -> None: assert s.llm_summary_model == "mini" +def test_llm_thinking_level_default_is_low(monkeypatch: pytest.MonkeyPatch) -> None: + """Phase 172: the model's thinking level defaults to ``low`` + (``BOR_LLM_THINKING_LEVEL`` unset).""" + monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False) + s = _settings() + assert s.llm_thinking_level == "low" + + +def test_llm_thinking_level_env_override(monkeypatch) -> None: + """Phase 172 (A4): free-form like ``llm_chat_model`` — + ``BOR_LLM_THINKING_LEVEL=high`` resolves to ``"high"`` (no + allowlist, no validator).""" + monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "high") + s = _settings() + assert s.llm_thinking_level == "high" + + +def test_llm_thinking_level_empty_env_value_stays_empty( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Phase 172 (A2): the EMPTY value is the off switch — it must not + be coerced back to the default (trimming happens only at the wire, + task 03; no normalization at load).""" + monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "") + s = _settings() + assert s.llm_thinking_level == "" + + def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None: """Phase 30: document content sent to the ``lite`` model is capped at ``BOR_SUMMARY_MAX_CHARS`` (default 12 000 chars per call).""" diff --git a/tests/unit/test_llm_client.py b/tests/unit/test_llm_client.py index 39e1179..7e60f68 100644 --- a/tests/unit/test_llm_client.py +++ b/tests/unit/test_llm_client.py @@ -1324,6 +1324,124 @@ def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch) assert sleeps == [1.25, 1.25, 1.25] +# ---------- thinking level passthrough (phase 172, task 03) ---------- + + +def test_chat_sends_reasoning_effort_extra_body_when_level_set() -> None: + """Phase 172 (A1): a non-empty level rides the one-shot request as + the top-level ``reasoning_effort`` parameter via the SDK's + ``extra_body``.""" + llm, completions = _make_chat_client( + _FakeCompletion("ok"), llm_thinking_level="low" + ) + asyncio.run(llm.chat([{"role": "user", "content": "q"}])) + assert completions.chat_kwargs is not None + assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} + + +def test_chat_default_level_low_is_always_sent() -> None: + """The default (``low``) is always sent (A2): a client built without + naming the setting still carries the parameter.""" + llm, completions = _make_chat_client(_FakeCompletion("ok")) + asyncio.run(llm.chat([{"role": "user", "content": "q"}])) + assert completions.chat_kwargs is not None + assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} + + +def test_chat_trims_the_level_before_sending() -> None: + """TRIM: a hand-edited ``.env`` value with trailing whitespace + (``"low "``) is sent trimmed — the raw stored string never reaches + the wire with surrounding whitespace.""" + llm, completions = _make_chat_client( + _FakeCompletion("ok"), llm_thinking_level="low " + ) + asyncio.run(llm.chat([{"role": "user", "content": "q"}])) + assert completions.chat_kwargs is not None + assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} + + +def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None: + """Phase 172 (A2): a whitespace value is the off switch — + ``extra_body`` is ABSENT (never None, never ``"off"``) and the + request kwargs keep the pre-phase-172 shape.""" + llm, completions = _make_chat_client( + _FakeCompletion("ok"), llm_thinking_level=" " + ) + asyncio.run(llm.chat([{"role": "user", "content": "q"}])) + assert completions.chat_kwargs is not None + assert "extra_body" not in completions.chat_kwargs + assert set(completions.chat_kwargs) == { + "model", + "messages", + "temperature", + "max_tokens", + "stream", + } + + +def test_chat_stream_sends_reasoning_effort_extra_body_when_level_set() -> None: + """Phase 172 (A1): the chat turn rides the parameter too.""" + llm, completions = _make_stream_client( + [_chunk("x")], llm_thinking_level="high" + ) + asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) + assert completions.kwargs is not None + assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"} + + +def test_chat_stream_trims_the_level_before_sending() -> None: + llm, completions = _make_stream_client( + [_chunk("x")], llm_thinking_level=" high " + ) + asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) + assert completions.kwargs is not None + assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"} + + +def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None: + """Phase 172 (A2): the stream's off switch — ``extra_body`` absent + (never None) and the kwargs dict keeps the pre-phase-172 shape.""" + llm, completions = _make_stream_client( + [_chunk("x")], llm_thinking_level="" + ) + asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) + assert completions.kwargs is not None + assert "extra_body" not in completions.kwargs + assert set(completions.kwargs) == { + "model", + "messages", + "temperature", + "max_tokens", + "stream", + } + + +def test_chat_stream_whitespace_level_is_the_off_switch() -> None: + """Whitespace-only (``" "``) is empty for wire purposes (A2).""" + llm, completions = _make_stream_client( + [_chunk("x")], llm_thinking_level=" " + ) + asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) + assert completions.kwargs is not None + assert "extra_body" not in completions.kwargs + + +def test_embeddings_body_never_carries_reasoning_effort() -> None: + """Phase 172 (A3): the embeddings path is untouched — with the level + set, every POST body stays the minimal ``{model, input}`` payload; + ``reasoning_effort`` never appears in it.""" + service = _FakeEmbeddingsService() + llm, http = _make_client(service, llm_thinking_level="high") + asyncio.run(llm.embed(["a", "b"])) + vec = asyncio.run(llm.embed_one("c")) + assert len(vec) == 768 + assert http.bodies # the requests actually happened + for body in http.bodies: + assert set(body) == {"model", "input"} + assert "reasoning_effort" not in body + assert "extra_body" not in body + + # ---------- chat_stream_retried (phase 67, task 01) ---------- _RETRY_MSGS: list[dict[str, str]] = [{"role": "user", "content": "q"}] diff --git a/tests/unit/test_settings_catalog.py b/tests/unit/test_settings_catalog.py index 0e4ee79..083f3f0 100644 --- a/tests/unit/test_settings_catalog.py +++ b/tests/unit/test_settings_catalog.py @@ -93,6 +93,21 @@ def test_env_var_is_bare_key_upper_prefixed() -> None: assert env_var(key).startswith("BOR_") +def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None: + """Phase 172 (task 01): the new ``llm_thinking_level`` pair sits in + ``LLM (aipi)`` with the ``Thinking level`` label and is plain — + not ``secret`` (a free-form text value, like ``llm_chat_model``), + not ``restart_required`` (the LLM client reads the setting per + request via ``get_settings()`` — live), overridable (the overlay is + generic over ``str`` fields).""" + meta = CATALOG["llm_thinking_level"] + assert meta.group == "LLM (aipi)" + assert meta.label == "Thinking level" + assert "llm_thinking_level" not in SECRETS + assert "llm_thinking_level" not in RESTART_REQUIRED + assert "llm_thinking_level" not in NON_OVERRIDABLE + + def test_labels_nonempty_and_unique() -> None: labels = [meta.label for meta in CATALOG.values()] assert all(label.strip() for label in labels), "empty label in the catalogue"