Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Experimental/brain-of-reese:e9ae6e2ba38d39ef441df30df3d72f74ebfcf6d8
Branches Tags
Experimental/brain-of-reese:main
..
compare: Experimental/brain-of-reese:f27106a638335e7df641311766ed253fd7b1ceb4
Branches Tags
Experimental/brain-of-reese:main
2 Commits
e9ae6e2ba3 .. f27106a638
Author SHA1 Message Date
ducoterra f27106a638 phase: 172_thinking_level
Build and Push Containers / build-and-push-app (push) Successful in 1m43s
Details
Build and Push Containers / build-and-push-db (push) Successful in 12s
Details 
All verification complete. Final report:

**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.**

- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`.
- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands).
- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol.
- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
 2026-10-10 13:19:10 -04:00
ducoterra b21b05eaee chore(agent): phase roadmap from TODO.md, phase 172 (thinking level) 
LLM-Generated: true
 2026-10-10 10:28:46 -04:00
43 changed files with 1914 additions and 12 deletions
Expand all files Collapse all files
.agents/phases/complete/172_thinking_level/00_phase.md
+50
View File
@@ -0,0 +1,50 @@
# Phase 172 — Thinking level: a pi-style, operator-tunable LLM reasoning knob
**Source:** `TODO.md` L3 — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-10)
**Context:** `app/config.py` — the LLM section (L97–125: `llm_base_url`, `llm_api_key`, `llm_chat_model` L101, `llm_embed_model`, `llm_summary_model` L107, `stream_thinking` L112, `llm_retries` / `llm_retry_delay` / `llm_timeout` L122 — the house pattern this setting joins), `.env.example` L57–65 (the LLM block — `BOR_LLM_CHAT_MODEL=turbo` L60), `app/core/settings_catalog.py` L137–161 (the `LLM (aipi)` catalogue group; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes the new field a build failure until it has a `SettingMeta`), `app/core/settings_overlay.py` (the generic DB-over-env overlay — a plain `str` field rides it with zero extra code: `PUT` stores the raw string, `invalidate_settings()` re-reads), `app/rag/llm.py` — `LLMClient.chat()` (L446, the one-shot `chat.completions.create` L416, `temperature=0.2`, 2048 tokens) and `LLMClient.chat_stream()` (L511, the kwargs dict L592–600: `model` / `messages` / `temperature=0.4` / `max_tokens` / `stream`), `app/api/chat.py` — the per-turn log line (L1268–1291: `…thinking_chars=%d tool_calls=%d…`), `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: a new `str` catalogue entry renders as a free-form text row automatically, exactly like `llm_chat_model`), `tests/unit/test_config.py`, `tests/unit/test_llm_client.py` (the mocked-SDK-client family), `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (the set/reset/live-apply contract), `tests/e2e/test_admin_settings_panel.py` (the phase-168 suite — the pattern to copy).
## Objective
The operator can pick the model's thinking level — pi-style, free-form text, default `low` — from `.env` (`BOR_LLM_THINKING_LEVEL`) or the admin Settings panel, and the value rides every chat-completions request to aipi as the OpenAI-standard `reasoning_effort` parameter, so the same knob works as the model's reasoning support evolves. The empty value is the off switch (the parameter is not sent at all).
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — one `Settings` field (task 01).** `llm_thinking_level: str = "low"` in the LLM section of `app/config.py` (placed directly after the `llm_summary_model` block, before `stream_thinking` — model-identity settings stay together). Free-form, exactly like `llm_chat_model` (A4: **no allowlist, no validator** — any non-empty text is accepted; the endpoint is the source of truth). The docstring names the env var (`BOR_LLM_THINKING_LEVEL`), the default, the wire mapping (D2) and the empty-off rule (D3). `.env.example` gains `BOR_LLM_THINKING_LEVEL=low` directly under `BOR_LLM_CHAT_MODEL=turbo` (L60) with a one-line comment (the house style of L61–63).
- **D2 — the wire (task 03).** The value is sent as a **top-level chat-completions parameter** `reasoning_effort` via the OpenAI SDK's `extra_body` (`extra_body={"reasoning_effort": "<value>"}` — the SDK merges `extra_body` into the request JSON body). Applied in BOTH `LLMClient.chat()` (the one-shot: document summaries, KB overview) and `LLMClient.chat_stream()` (the chat turn) — A3. The embeddings path (`_post_embeddings` / `embed` / `embed_one`) is **untouched**. `ASSUMPTION: A1 — aipi accepts the OpenAI-standard `reasoning_effort` top-level parameter (pi's `reasoningEffort`); if aipi exposes a different key (e.g. `chat_template_kwargs`), only task 03's key changes — the rest of the phase is key-agnostic by design.**
- **D3 — the off switch (tasks 03/04).** The **trimmed** value decides: non-empty → the parameter is sent with the trimmed value; empty/whitespace → `extra_body` is absent from the request (byte-identical to today's payload). Nothing is sent as `"off"` or `null` — absence is the off.
- **D4 — the admin row (task 02).** One `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` entry in `CATALOG` (placed after `llm_summary_model` L150–151, before `llm_retries` L156). Nothing else: the panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes, and the DB-over-env overlay are all **generic over the catalogue** — the row, its free-form text input, its save/reset and its live-apply fall out with zero extra code (A5). The completeness pin is the enforcement: a missing entry fails the build.
- **D5 — the per-turn line (task 04).** The per-turn log line (PLAN §9, `app/api/chat.py` L1268) gains `thinking=<value>` immediately before the existing `thinking_chars=%d` — the effective (overlay-resolved) value, trimmed, or `-` when empty, so "which level did this turn run at" stays one `grep` away (AGENTS rule 10). No `query_log` schema change (the table is retrieval-scoped; the line is the home of turn knobs).
- **NOT touched:** `frontend/assets/settings.js` (generated from the catalogue), `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events — different concern, stays as-is), every other setting.
## Dependencies
— (none: the app is fully functional at the head of `complete/`; this phase builds on the committed phase-168/171 settings machinery)
## Tasks
1. `01_setting_and_env.md` — `Settings.llm_thinking_level: str = "low"` + the `CATALOG` row (the completeness pin keeps the build green) + `BOR_LLM_THINKING_LEVEL=low` in `.env.example`; the config/catalog unit tests.
2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for the new key (stored raw, live without restart, empty storable as the off switch).
3. `03_llm_extra_body_passthrough.md` — `extra_body={"reasoning_effort": …}` on `chat()` + `chat_stream()` when trimmed-non-empty; absent when empty; the mocked-client unit tests.
4. `04_per_turn_log_field.md` — `thinking=<value>` (or `-`) on the per-turn log line; the log-line test.
5. `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_config.py` — the default is `low`, the env override works, an empty env value yields an empty string (no coercion, no allowlist — D3/A4). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (the new field has its `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat()` and `chat_stream()` send `extra_body={"reasoning_effort": <trimmed value>}` when the setting is non-empty, send **no** `extra_body` when it is empty/whitespace, and the embeddings path is unaffected (the mocked-SDK-client family).
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — the row appears in `GET /api/admin/settings` (group, `str`, effective `low`); `PUT` stores the raw string, the empty value is storable (the off switch), the overlay resolves live without a restart; `DELETE` resets to env.
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_thinking_level.py`, run **in isolation** — the row is visible in the `LLM (aipi)` group with default `low`, is a free-form text input (type `high`, save, survives reload), reset-to-env restores `low`, and the `test_admin_settings_panel.py` regression set stays green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest -k "config and thinking" -v` shows the default `low` + the env override; `.env.example` contains `BOR_LLM_THINKING_LEVEL=low` under `BOR_LLM_CHAT_MODEL`.
- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, default `low`); `PUT` / `DELETE` round-trip per the overlay contract.
- [ ] With the setting non-empty, a `chat_stream()` / `chat()` request carries `reasoning_effort` (unit-asserted via the mocked client); with it empty, the request kwargs contain no `extra_body` (byte-identical to today).
- [ ] The per-turn log line carries `thinking=<value>` (`-` when empty) next to `thinking_chars`.
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation.
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort`); the phase dir moves to `.agents/phases/complete/`.
## Locked decisions (owner-confirmed 2026-10-10, roadmap confirmation)
- **A1 — the wire key is `reasoning_effort`** (the OpenAI-standard top-level chat-completions parameter, pi's `reasoningEffort`), sent via the SDK's `extra_body`. If aipi exposes a different key, only task 03 changes.
- **A2 — the off switch is absence:** empty/whitespace value → the parameter is not sent at all; the default `low` is always sent.
- **A3 — scope:** the parameter rides BOTH `chat_stream()` (the chat turn) and `chat()` (the one-shot summaries / KB overview); the embeddings path is untouched.
- **A4 — free-form, no allowlist:** any non-empty text is accepted (like `llm_chat_model`); no validator, no value coercion.
- **A5 — no frontend code change:** the panel is generated from the catalogue; the row appears automatically as a free-form text input.
- **A6 — the per-turn line** gains `thinking=<trimmed value>` (`-` when empty) immediately before `thinking_chars`; no `query_log` schema change.
.agents/phases/complete/172_thinking_level/01_setting_and_env.md
+49
View File
@@ -0,0 +1,49 @@
# Task 01 — The `llm_thinking_level` setting + the catalogue row + `.env.example`
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
**Story:** n/a
## Objective
The new setting exists in `app/config.py` (default `low`, free-form like `llm_chat_model`), in the admin catalogue (so the build's completeness pin stays green — a field without a `SettingMeta` fails the build, and the per-task gate runs the full suite), and in `.env.example`, so `BOR_LLM_THINKING_LEVEL` is a first-class, documented knob before anything consumes it.
## Work
1. `app/config.py` — in the LLM section, **directly after** the `llm_summary_model` block (L103–107) and **before** `stream_thinking` (L108), add ONE field, house docstring style (env var named, default, wire mapping, off rule):
```python
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to
#: aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
#: parameter is not sent at all.
llm_thinking_level: str = "low"
```
No validator, no coercion — a plain `str` (ASSUMPTION A4, phase-locked). Verify by reading: `llm_chat_model` (L101) keeps its shape byte-identical.
2. `.env.example` — in the LLM block (L57–65), directly under `BOR_LLM_CHAT_MODEL=turbo` (L60), add:
```
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
```
(one-line comment, the house style of L61–63; `#`-prefixed optional vars stay commented — this one is set, like L60).
3. `app/core/settings_catalog.py` — in the `LLM (aipi)` catalogue group, **directly after** the `llm_summary_model` entry (L150–151) and **before** `llm_retries` (L156), add ONE entry:
```python
"llm_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level"
),
```
Nothing else — `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` are DERIVED from `CATALOG` (module docstring), so no other list changes. The row is plain: not `secret`, not `restart_required` (the LLM client reads the setting per request via `get_settings()` — live), overridable (the overlay is generic over `str` fields, `app/core/settings_overlay.py`).
4. `tests/unit/test_settings_catalog.py` — the completeness pin (`set(CATALOG) == set(Settings.model_fields)`) must stay green with the new pair; add the house-style assertion that `CATALOG["llm_thinking_level"]` sits in group `LLM (aipi)` with label `Thinking level` and is **not** in `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` (the derived sets).
5. `tests/unit/test_config.py` — add the house-style cases: (a) the DEFAULT is `"low"` (no env set); (b) the env override — `BOR_LLM_THINKING_LEVEL=high` resolves to `"high"`; (c) the empty env value (`BOR_LLM_THINKING_LEVEL=`) resolves to `""` (no coercion to a default, no allowlist — the off switch stays expressible).
- ASSUMPTION: A4 (phase-locked) — free-form text, no allowlist/validator; A2 (phase-locked) — empty = off switch, so the setting must NOT strip/normalize away the empty value at load (it is trimmed only at the wire, task 03); A5 (phase-locked) — the panel row falls out of the catalogue with zero `settings.js` code.
## Testing & Quality
- Unit: `tests/unit/test_config.py` + `tests/unit/test_settings_catalog.py` green (`uv run pytest tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov`).
- Coverage: **>90%** on `app/` (the gate must stay green).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` green: default `low`, env override `high`, empty stays `""`, the completeness pin holds with the new pair.
- [ ] `grep -n "BOR_LLM_THINKING_LEVEL" .env.example` shows the line under `BOR_LLM_CHAT_MODEL=turbo`.
- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, effective `low`) — the pin + a quick curl, or the task-02 integration test.
- [ ] `uv run pytest` green; ruff + pyright clean.
- [ ] no behavior change in completed work (no other `Settings` field touched; `settings.js` untouched).
.agents/phases/complete/172_thinking_level/02_admin_row_contract.md
+24
View File
@@ -0,0 +1,24 @@
# Task 02 — The admin row's API contract (set / reset / live-apply for `llm_thinking_level`)
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. … Update .env and admin settings."
**Story:** n/a
## Objective
Prove, at the API/overlay level, that the new key behaves exactly like `llm_chat_model`: it appears in the catalogue, `PUT` stores the raw string and the overlay resolves it live (no restart), `DELETE` resets to env, and the effective value the LLM layer will read follows the overlay.
## Work
1. `tests/integration/test_admin_settings_api.py` — extend the existing key-parameterized patterns (read how the phase-168 suite picks a `str` key, e.g. `llm_chat_model`) with the house-style cases for `llm_thinking_level`: (a) `GET /api/admin/settings` → the item is present, group `LLM (aipi)`, `type str`, effective value `low` (no row in `app_settings`), `env_var` `BOR_LLM_THINKING_LEVEL`; (b) `PUT` with `{"value": "high"}` → 200, the row is stored **raw** in `app_settings` (env-native string form, D1 of the overlay), the response reports the new effective value; (c) `PUT` with an empty string `""` → 200 (the off switch is storable — the API stores as-is, no allowlist); (d) `DELETE /api/admin/settings/llm_thinking_level` → the row is gone, effective value back to the env default `low`; (e) the non-admin 403 gate still applies (the phase-168 router dependency — assert once, mirroring the existing suite).
2. `tests/integration/test_admin_settings_live.py` — add the live-apply case in the house pattern: `PUT llm_thinking_level=high` → `get_settings().llm_thinking_level` is `"high"` **without** a restart (the `invalidate_settings()` path); `DELETE` → back to `"low"`. (This is the proof the LLM layer — which reads `get_settings()` per request, task 03 — picks the override up live.)
3. If the phase-168 suites parameterize over `CATALOG` keys generically (check `tests/integration/test_admin_settings_api.py` first — read it, do not assume): the new key may already be covered by the parameterization; in that case add only the empty-string (c) case and the live-apply (2) case, and note it in the test's comment.
- ASSUMPTION: A4 (phase-locked) — the empty string is storable (the off switch must be settable from the panel, not only via `.env`); the API "stores as-is" per the phase-168 module docstring — no value validation to add.
## Testing & Quality
- Integration: `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` green (`uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov`).
- Coverage: **>90%** on `app/` (unchanged `app/` code — the gate must stay green, not drop).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `GET /api/admin/settings` (admin) lists the row (`LLM (aipi)`, `str`, `low`); `PUT high` → live effective `high`; `PUT ""` → live effective `""`; `DELETE` → back to `low` (the integration cases green).
- [ ] `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` green; full suite green.
- [ ] ruff + pyright clean; no `app/` code changed by this task.
.agents/phases/complete/172_thinking_level/03_llm_extra_body_passthrough.md
+46
View File
@@ -0,0 +1,46 @@
# Task 03 — The thinking level rides every chat-completions request (`reasoning_effort` via `extra_body`)
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …"
**Story:** n/a
## Objective
`LLMClient` sends the operator's thinking level to aipi on every chat-completions request — `chat()` (the one-shot) and `chat_stream()` (the chat turn) — as the top-level `reasoning_effort` parameter via the SDK's `extra_body`; when the (trimmed) value is empty the request is byte-identical to today's (no `extra_body` at all).
## Work
1. `app/rag/llm.py` — ONE small helper (module-level, near the top, after the imports):
```python
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
```
Then in **BOTH** request sites, add the parameter only when present:
- `chat()` (L446, the `await self._client.chat.completions.create(...)` at L416–421): compute `extra = _thinking_extra_body(self.settings)` and pass the kwarg **only when present** — `**({"extra_body": extra} if extra is not None else {})` spread into the `create(...)` call, so `extra_body` is ABSENT (never `None`) when off.
- `chat_stream()` (L511, the kwargs dict L592–600): same pattern — after the dict is built (and after the `if tools is not None:` block, L601–602), `if (extra := _thinking_extra_body(self.settings)) is not None: kwargs["extra_body"] = extra`.
The value is the **trimmed** setting (`.strip()` in the helper) — the raw stored string is never sent with surrounding whitespace. The embeddings path (`_post_embeddings` L256, `embed` L346, `embed_one` L400) is **untouched** (A3) — verify by reading: no `extra_body` appears anywhere in the embedding code.
2. `app/rag/llm.py` — update the module docstring (L1–L30) with ONE sentence: the thinking level (phase 172, `BOR_LLM_THINKING_LEVEL`) rides chat completions as `reasoning_effort`; empty = absent. (The house style: wire facts live in the module docstring, cf. the `reasoning_content` note at L10–12.)
3. `tests/unit/test_llm_client.py` — extend the mocked-SDK-client family (read the existing `chat` / `chat_stream` cases first and mirror their mocking):
- `chat()` with `llm_thinking_level="low"` → `create` is awaited with `extra_body={"reasoning_effort": "low"}` (assert on the call kwargs).
- `chat()` with `llm_thinking_level=" "` (whitespace) → `extra_body` is **absent from the call kwargs** (assert `"extra_body" not in kwargs`).
- `chat_stream()` with `"high"` → same `extra_body` assertion on the `create` call.
- `chat_stream()` with `""` → `extra_body` absent (the off switch — the kwargs dict is byte-identical in shape to pre-phase-172).
- one `embed()` / `embed_one()` case with the level set → the HTTP POST body (the mocked transport) contains **no** `reasoning_effort` (the embeddings path is untouched, A3).
- the TRIM case: `llm_thinking_level="low "` (trailing spaces — reachable via a hand-edited `.env`) → the sent value is `"low"`.
- ASSUMPTION: A1 (phase-locked) — the wire key is the OpenAI-standard top-level `reasoning_effort` via `extra_body`; if aipi exposes a different key, ONLY this task's helper changes (the setting, the row, the log, the E2E are key-agnostic). A2 (phase-locked) — absence is the off switch (never `extra_body=None`, never `"off"`). A3 (phase-locked) — `chat()` + `chat_stream()` only; embeddings untouched.
## Testing & Quality
- Unit: `tests/unit/test_llm_client.py` green (`uv run pytest tests/unit/test_llm_client.py -v --no-cov`).
- Coverage: **>90%** on `app/` (the new helper is fully covered by the cases above).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] With the setting non-empty, a `chat()` / `chat_stream()` request carries `extra_body={"reasoning_effort": <trimmed value>}` (the unit cases green).
- [ ] With it empty/whitespace, `extra_body` is absent from the request kwargs (byte-identical shape to today); the embeddings POST body never contains `reasoning_effort`.
- [ ] `uv run pytest` green; ruff + pyright clean.
- [ ] no behavior change in completed work (temperature / max_tokens / tools / stream kwargs untouched; the `stream_thinking` kill-switch untouched).
.agents/phases/complete/172_thinking_level/04_per_turn_log_field.md
+24
View File
@@ -0,0 +1,24 @@
# Task 04 — The per-turn log line records the effective thinking level
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …"
**Story:** n/a
## Objective
The per-turn log line (PLAN §9, AGENTS rule 10) records the effective thinking level of the turn, so "which level did this turn run at" is one `grep` away, next to the `thinking_chars` it governs.
## Work
1. `app/api/chat.py` — the per-turn `logger.info(...)` (L1268–1291): insert `thinking=%s` **immediately before** the existing `thinking_chars=%d` in the format string, with the argument `settings.llm_thinking_level.strip() or "-"` (the effective overlay-resolved value — `settings` is already in scope, the line already logs `settings.relevance_threshold` L1280; trimmed, `-` when empty, per A6). No reordering of the existing fields; the `query_log` row (L1250–1260) is **untouched** (A6: the table is retrieval-scoped, the line is the home of turn knobs).
2. `app/api/chat.py` — the module docstring: the existing per-turn-line notes (L14, L46, L64, L78, L141, L161) set the house pattern — add ONE line where the log line's fields are described: the per-turn log line records `thinking=<level>` (the effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172).
3. The existing per-turn-line test (find it: `grep -rn "thinking_chars" tests/` — the house pattern pins the line's fields in one place; extend that test, do not invent a new file): assert the new `thinking=` field is present, carries the effective value in the non-empty case, and is `-` when the setting is empty.
- ASSUMPTION: A6 (phase-locked) — `thinking=<trimmed value>` or `-`, placed immediately before `thinking_chars`; no `query_log` schema change, no new `Settings` read (the per-turn `settings` object is already resolved through the overlay).
## Testing & Quality
- Unit/integration: the per-turn-line test (above) green; `uv run pytest` green (the SSE turn machinery is untouched — the change is one format field + one argument).
- Coverage: **>90%** on `app/` (the gate must stay green, not drop).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] A test turn's per-turn log line (the test's captured log) contains `thinking=low` (default) and `-` when the setting is empty; `thinking_chars` stays in its position after it.
- [ ] `grep -rn "thinking=" app/api/chat.py` shows the field exactly once in the format string.
- [ ] `uv run pytest` green; ruff + pyright clean; no `query_log` schema change.
.agents/phases/complete/172_thinking_level/05_e2e_admin_settings_row.md
+32
View File
@@ -0,0 +1,32 @@
# Task 05 — The phase's dedicated Playwright suite (AGENTS rule 4) + the full gates + the atomic commit
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
**Story:** n/a
## Objective
`tests/e2e/test_thinking_level.py` proves the owner's acceptance in a real browser — the Thinking level row exists in the admin Settings panel as a **free-form text** entry (like the chat model's), defaults to `low`, saves and persists, and resets to env — run **in isolation**; then the full gates, then ONE atomic commit.
## Work
1. `tests/e2e/test_thinking_level.py` — the phase's dedicated suite, run in isolation (DB must be up: `podman compose up -d db`; `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov`). **Copy the harness from `tests/e2e/test_admin_settings_panel.py`** (read it first, then mirror: the `_boot_app`/`app_server`/`app_url` fixtures with the mock LLM, the `_clean` autouse fixture (its `app_settings` truncation pattern — the new key's rows must be wiped between tests), the `_cookies`/`_admin_client` helpers, the settings-page nav + Save/Reset drives at L561ff). Three tests:
- `test_row_renders_free_form_with_default_low` — admin opens `/settings.html`; the `LLM (aipi)` group band is visible (the phase-171 band contract); a row labelled **Thinking level** (`BOR_LLM_THINKING_LEVEL` env-var display, the catalogue pattern) is present, its input is a **free-form text** input (`type="text"` — NOT `number`/`select`, the A4 contract), shows `low` with the `env` badge. Assert the `llm_chat_model` row's input is the same `type="text"` control (the "like chat model" reference, in one assertion pair).
- `test_saved_level_persists_without_restart` — type `high` into the row (free-form: also type a custom value `think-soft` once, to prove no allowlist — A4), Save (the phase-168 §7.4 Save lifecycle: dirty → save → `DB` badge), the effective value reads `think-soft`; reload `/settings.html` → still `think-soft` with the `DB` badge (no restart, the phase-168 contract).
- `test_reset_returns_to_env_default` — the row's Reset (per-row `DELETE`) → the value reads `low` again with the `env` badge (the overlay reset, task-02's contract, in the browser).
2. Run the **full gates** in order, all from the repo root:
- `uv run pytest -v` (unit + integration) and `uv run pytest --cov=app --cov-report=term-missing` → `app/` coverage **>90%**.
- `uv run ruff check . && uv run pyright`.
- `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` → green **in isolation**.
- Regression, in isolation: `uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov` (phase 168) and `uv run pytest tests/e2e/test_admin_settings_layout.py -v --no-cov` (phase 171) → both green.
3. The **atomic commit** (AGENTS rule 8 — one commit per phase, Conventional Commits, `--no-gpg-sign`): the whole phase — `app/config.py`, `app/core/settings_catalog.py`, `app/rag/llm.py`, `app/api/chat.py`, `.env.example`, the four test files, and `.agents/phases/todo/172_thinking_level/` → moved to `.agents/phases/complete/172_thinking_level/` in the SAME commit. Message: `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort` + the `LLM-Generated: true` trailer. The pre-existing dirty `TODO.md` (the now-converted entry, cleared in Phase 4 of this conversion) rides this commit ONLY if the conversion commit has not already committed it — check `git status` first; the conversion commit is the one that clears TODO.md (see the skill's Phase 4), so the phase commit should be the phase files + code only.
- ASSUMPTION: A5 (phase-locked) — no `settings.js` / `styles.css` change; the row renders from the catalogue. A6 (phase-locked) — the per-turn-line assertion lives in the unit/integration layer (task 04), not the E2E (the mock LLM turn does not exercise the wire; the E2E proves the settings surface only).
## Testing & Quality
- E2E: `tests/e2e/test_thinking_level.py` green **in isolation**; the phase-168 + phase-171 suites green in isolation (regression).
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green in isolation (the three contracts).
- [ ] `uv run pytest` green; `app/` coverage >90%; ruff + pyright clean.
- [ ] `test_admin_settings_panel.py` + `test_admin_settings_layout.py` green in isolation (no regression in the settings machinery).
- [ ] One atomic `--no-gpg-sign` Conventional Commit with the `LLM-Generated: true` trailer; the phase dir moved to `.agents/phases/complete/`.
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md
+9
View File
@@ -0,0 +1,9 @@
All verification complete. Final report:
**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.**
- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`.
- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands).
- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol.
- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__00_phase.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md
+22
View File
@@ -0,0 +1,22 @@
Task 01 complete — everything green.
**Implemented**
- `app/config.py`: `llm_thinking_level: str = "low"` with house docstring (env var, default, `reasoning_effort` wire mapping, empty = off switch), placed after `llm_summary_model` before `stream_thinking`; no validator/coercion (A4/A2)
- `.env.example`: `BOR_LLM_THINKING_LEVEL=low` with one-line comment, directly under `BOR_LLM_CHAT_MODEL=turbo` (L60→L61)
- `app/core/settings_catalog.py`: `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` after the `llm_summary_model` entry — plain (not secret, not restart-required, overridable)
- Tests: 3 new cases in `tests/unit/test_config.py` (default `low`, env override `high`, empty env stays `""`); 1 new case in `tests/unit/test_settings_catalog.py` (group/label + not in the derived SECRETS/RESTART_REQUIRED/NON_OVERRIDABLE sets)
**Results**
- `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` → 5 passed
- `uv run pytest` → 4073 passed (6 initial failures were only the DB being down; `podman compose up -d db` → all green)
- `uv run pytest --cov=app --cov-report=term-missing` → 99% on `app/` (>90% gate)
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
- `GET /api/admin/settings` verified via TestClient: row present in `LLM (aipi)`, `str`, effective `low` (source `default`), overridable, not secret/restart
**Docs**
- `README.md`: added `BOR_LLM_THINKING_LEVEL` row to the LLM env-var table; `AGENTS.md`: no doc impact (no env list there)
**Decisions**
- None beyond the phase design; `settings.js` untouched (A5 — the row falls out of the catalogue)
**Next pending task:** `02_admin_row_contract.md`
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__01_setting_and_env.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
......................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6393 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Report — phase 172, task 02 (admin row API contract for `llm_thinking_level`)**
Implemented (tests only, per task — no `app/` code touched):
- `tests/integration/test_admin_settings_api.py::test_llm_thinking_level_row_contract` — (a) GET: present, group `LLM (aipi)`, `str`, `env_var` `BOR_LLM_THINKING_LEVEL`, effective `low`, not secret/restart-required; (b) PUT `"high"` → 200, row stored **raw** in `app_settings`, live effective `high`; (c) PUT `""` → 200 storable (off switch, no allowlist); (d) DELETE → row gone, back to `low`; (e) non-admin 403 asserted once (full matrix already pinned). Comment notes the shape/coverage tests pin presence generically (suite is not key-parameterized).
- `tests/integration/test_admin_settings_live.py::test_llm_thinking_level_live_apply_no_restart` — house `_save` + `invalidate_settings()` pattern: `high` live without restart, empty resolves live (A2/A4), DELETE resets to `low`; module docstring updated.
- Deterministic base in both: `monkeypatch.delenv` + code default `low` (kill-switch test idiom, machine-independent).
Gates (all green):
- `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` → 20 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 4075 passed, app/ coverage 99% (>90%)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
Docs: no impact — test-only task; the env var is documented in `.env.example` (task 01), and README/AGENTS.md don't enumerate individual settings.
Next pending task: `03_llm_extra_body_passthrough.md`.
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__02_admin_row_contract.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6393 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md
+14
View File
@@ -0,0 +1,14 @@
Task 03 complete — all gates green.
**Report**
- **Implemented (task 03 — wire passthrough):**
- `_thinking_extra_body()` module-level helper in `app/rag/llm.py`: trimmed non-empty level → `{"reasoning_effort": <value>}`; empty/whitespace → `None` (off switch)
- `chat()` (via `_chat_once`): `extra_body` spread into `create()` **only when present** — key absent (never `None`) when off; pyright-safe typed `dict[str, Any]`
- `chat_stream()`: `kwargs["extra_body"]` set after the `tools` block, same absent-when-off pattern
- Module docstring: one sentence on the phase-172 wire contract (empty = absent, embeddings never carry it)
- 9 new unit tests in `tests/unit/test_llm_client.py`: `chat()` low/default/whitespace, `chat_stream()` high/empty/whitespace, TRIM (`"low "` → `"low"`), byte-identical kwargs-shape pins for the off switch, and the embeddings POST body never containing `reasoning_effort`
- **Results:** `uv run pytest --cov=app` → **4084 passed**, 99% on `app/` (>90% gate); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors; `uv run pytest tests/unit/test_llm_client.py -v --no-cov` → 81 passed
- **Docs:** no doc impact — README L1156 already documents `BOR_LLM_THINKING_LEVEL` incl. the wire mapping (task 01's doc work); AGENTS.md unaffected
- **Decisions:** task-spec inline `**` spread failed pyright (empty-dict inference, 33 errors) — split into an annotated `extra_kwargs: dict[str, Any]`, semantics identical (key absent when off)
- **Next pending task:** `04_per_turn_log_field.md`
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__03_llm_extra_body_passthrough.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md
+14
View File
@@ -0,0 +1,14 @@
All gates green. The change is a log-line-only observability field — `README.md` documents the `thinking` SSE events (untouched UI surface) but not the per-turn line's field list, and `AGENTS.md` rule 10 refers to the line generically, so no doc edits are needed.
**Report — task 04 (per-turn log field)**
- `app/api/chat.py`: per-turn log line gains `thinking=%s` immediately before `thinking_chars=%d`, argument `settings.llm_thinking_level.strip() or "-"` (effective overlay-resolved value; existing fields un-reordered, `query_log` row untouched)
- `app/api/chat.py`: one-line module-docstring note in the Thinking paragraph (house pattern) — `thinking=<level>` (effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172, A6)
- `tests/unit/test_chat_gate.py`: extended the existing per-turn-line field-order test — asserts `thinking=low` (default), `thinking=high` (effective non-default), `thinking=-` (empty and whitespace-only, pinning the trim), adjacency to `thinking_chars`, and `thinking=` inserted into the full field-order tuple
- `uv run pytest --cov=app` → 4084 passed, `app/` coverage 99% (>90% gate)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- `grep -rn "thinking=" app/api/chat.py` → exactly once in the format string (L1275); no `query_log` schema change
- Docs: no README/AGENTS.md impact (log-line field is server-observability only; both files reference the line generically)
- No deviations; A6 honored exactly
**Next pending task:** `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (dedicated Playwright suite), regression set, full gates, atomic commit.
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__04_per_turn_log_field.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.err
View File
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md
+12
View File
@@ -0,0 +1,12 @@
All gates green. TODO.md is already cleared by the conversion commit (unmodified), so the phase commit is phase files + code only, as the task anticipated. The commit itself and the `todo/ → complete/` move are the harness's to make at phase end (per protocol, I leave every change in the working tree).
## Report
- **Implemented:** `tests/e2e/test_thinking_level.py` — the phase's dedicated Playwright suite (3 tests), mirroring the phase-168 harness (module `_boot_app` with mock LLM + `BOR_SETTINGS_DB=1` opt-in, `BOR_LLM_THINKING_LEVEL=low` pinned to the code default, `app_settings`/`ui_settings` autouse wipe, `.env` mtime guard, `_admin_client`):
- `test_row_renders_free_form_with_default_low` — `LLM (aipi)` band visible; "Thinking level" row with `BOR_LLM_THINKING_LEVEL` env-var display; `type="text"` free-form control (not number/select); `low` + `env` badge; `llm_chat_model` control asserted as the same `type="text"` (one assertion pair)
- `test_saved_level_persists_without_restart` — dirty → save → `DB` badge; custom value `think-soft` saved raw (API effective + raw `app_settings` row); survives reload without restart
- `test_reset_returns_to_env_default` — per-row Reset (DELETE) after a save → status line, `low` + `env` badge, row gone
- **Gates (all green):** `uv run pytest -v` → 4084 passed; `--cov=app` → **99%** (>90%); `uv run ruff check . && uv run pyright` → clean; `tests/e2e/test_thinking_level.py --no-cov` → 3 passed in isolation; regression `test_admin_settings_panel.py` → 12 passed, `test_admin_settings_layout.py` → 5 passed (both in isolation)
- **Docs:** no doc changes needed — `README.md` already carries the `BOR_LLM_THINKING_LEVEL` row (tasks 01–04); AGENTS.md has no per-setting content
- **Decisions:** the module app pins `BOR_LLM_THINKING_LEVEL=low` (env set → `env` badge, house leak-guard) and `BOR_LLM_CHAT_MODEL` to its code default; no KB seeding (A6 — the settings surface only, per-turn line is task 04's integration contract); commit + phase-dir move deferred to the harness
- **Next pending task:** none — task 05 is the phase's final task; phase 172 is complete
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/172_thinking_level/172_thinking_level__05_e2e_admin_settings_row.a1.validate
+134
View File
@@ -0,0 +1,134 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 108 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 286 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 86 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 532 0 100%
--------------------------------------------------
TOTAL 6400 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.env.example
+1
View File
@@ -58,6 +58,7 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
BOR_LLM_CHAT_MODEL=turbo
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
# BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off
# BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120)
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls)
README.md
+1
View File
@@ -1153,6 +1153,7 @@ never the plaintext).
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` |
| `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model |
| `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries |
| `BOR_LLM_THINKING_LEVEL` | `low` | The model's thinking level (pi-style, free-form — no allowlist); rides every chat-completions request as `reasoning_effort`; empty = the parameter is not sent |
| `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) |
| `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off |
| `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries |
app/api/chat.py
+7 -3
View File
@@ -13,7 +13,10 @@ model's reasoning arrives ahead of the answer and is streamed as
``thinking`` events before the ``delta`` events of the same turn. Each
turn's thinking is counted in the per-turn log line
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
``thinking`` frames (the pieces are still counted).
``thinking`` frames (the pieces are still counted). The per-turn log
line records ``thinking=<level>`` (the effective
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6).
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
@@ -1269,8 +1272,8 @@ async def chat(
"question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d "
"suggested=%d tuning=%d "
"kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r "
"thinking_chars=%d tool_calls=%d rounds=%d cap_hit=%d total_ms=%d "
"retries=%d scaffold_stripped=%d",
"thinking=%s thinking_chars=%d tool_calls=%d rounds=%d "
"cap_hit=%d total_ms=%d retries=%d scaffold_stripped=%d",
request.message,
embed_ms,
plan.top_score,
@@ -1283,6 +1286,7 @@ async def chat(
settings.relevance_threshold,
plan.deflected,
source_paths,
settings.llm_thinking_level.strip() or "-", # phase 172 (A6)
thinking_chars,
holder.tool_calls,
holder.rounds,
app/config.py
+8
View File
@@ -105,6 +105,14 @@ class Settings(BaseSettings):
#: Served by the same OpenAI-compatible endpoint — no new model
#: management. Called via ``LLMClient.chat()``.
llm_summary_model: str = "lite"
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to
#: aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
#: parameter is not sent at all.
llm_thinking_level: str = "low"
#: Operator kill-switch for the ``thinking`` SSE events (phase 17,
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
#: pieces are still counted for the per-turn log line but never
app/core/settings_catalog.py
+3
View File
@@ -150,6 +150,9 @@ CATALOG: dict[str, SettingMeta] = {
"llm_summary_model": SettingMeta(
"llm_summary_model", "LLM (aipi)", "LLM summary model"
),
"llm_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level"
),
"stream_thinking": SettingMeta(
"stream_thinking", "LLM (aipi)", "Stream thinking"
),
app/rag/llm.py
+29 -1
View File
@@ -10,7 +10,13 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty).
(an answer can in principle be empty). The operator's thinking level
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
as the top-level ``reasoning_effort`` parameter (via the SDK's
``extra_body``); an empty value is the off switch — the parameter is
absent, the request is byte-identical to pre-phase-172, and the
embeddings path never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different
@@ -40,6 +46,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
class EmbeddingError(RuntimeError):
"""The embeddings endpoint failed (network, HTTP, or malformed reply)."""
@@ -412,6 +429,10 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else.
"""
extra = _thinking_extra_body(self.settings)
extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {}
)
try:
resp = await self._client.chat.completions.create(
model=model,
@@ -419,6 +440,9 @@ class LLMClient:
temperature=0.2,
max_tokens=2048,
stream=False,
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
**extra_kwargs,
)
except LLMError:
raise
@@ -598,6 +622,10 @@ class LLMClient:
}
if tools is not None:
kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None:
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None
try:
stream = cast(
tests/e2e/test_thinking_level.py
+405
View File
@@ -0,0 +1,405 @@
"""Phase 172 E2E (Playwright): the Thinking level row in the admin Settings
panel — the owner's acceptance for a pi-style, operator-tunable LLM
reasoning knob, proven in a real browser.
Source: ``TODO.md`` L3 — "Brain of Reese needs to support thinking levels
like pi does. should be a free-form text entry like chat model. Default
should be 'low'. Update .env and admin settings."
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_level.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_row_renders_free_form_with_default_low`` — the row: the
``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Thinking level** with the
``BOR_LLM_THINKING_LEVEL`` env-var display (the catalogue
pattern) is present, its input is a **free-form text** input
(``type="text"`` — NOT ``number``/``select``, the A4
contract), it shows ``low`` with the ``env`` badge, and the
``llm_chat_model`` row's input is the same ``type="text"``
control (the "like chat model" reference, in one assertion
pair).
2. ``test_saved_level_persists_without_restart`` — the save
(the phase-168 §7.4 Save lifecycle: dirty → save → ``DB``
badge): ``high`` is typed (the dirty hint appears), then the
free-form custom value ``think-soft`` is typed once (no
allowlist — A4) and saved; the effective value reads
``think-soft`` (the API catalogue AND the raw
``app_settings`` row), and a reload of ``/settings.html``
still shows ``think-soft`` with the ``DB`` badge (no
restart, the phase-168 contract).
3. ``test_reset_returns_to_env_default`` — the per-row Reset
(the per-row ``DELETE``): after a save, the Reset is live;
clicking it lands the status line and the value reads
``low`` again with the ``env`` badge (the overlay reset,
task 02's contract, in the browser).
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens); the module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_THINKING_LEVEL`` (and the
"like chat model" reference, ``BOR_LLM_CHAT_MODEL``) to their
CODE DEFAULTS — the env var is set (so the badge reads ``env``,
not ``default``) and an operator's local ``.env`` cannot leak a
different level into the suite's app (the house leak-guard
pattern). A module-scope ``.env`` mtime guard pins LOCKED A1:
no code path the suite drives opens the env file for writing.
ASSUMPTION A6 (phase-locked): the per-turn-line assertion
(``thinking=<value>``) lives in the unit/integration layer
(task 04) — the mock LLM turn does not exercise the wire, and
this suite proves the settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The save's custom free-form value (test 2) — NOT a known pi
#: level (proves the A4 contract: no allowlist, the endpoint is
#: the source of truth).
SAVED_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-172 pins,
with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the row's save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 172 (task 05): pin the thinking level to the CODE
# DEFAULT — the env var is SET (so the badge reads ``env``, not
# ``default``) and an operator's local .env cannot leak a
# different level into the suite's app.
env["BOR_LLM_THINKING_LEVEL"] = Settings.model_fields[
"llm_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: the thinking level
pinned to the code default ``low`` and ``BOR_SETTINGS_DB=1``
(the module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the new key's rows
are wiped between tests) and the single-row ``ui_settings``
(the middleware reads it for EVERY page), and delete this
suite's issued tokens (label-scoped on ``e2e-`` — never a
TRUNCATE: the shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
_clean_settings_state()
yield
_clean_settings_state()
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
# ---------------------------------------------------------------------------
# 1. The row renders free-form with default low (the "like chat
# model" reference, in one assertion pair)
# ---------------------------------------------------------------------------
def test_row_renders_free_form_with_default_low(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
# The LLM (aipi) group band is visible (the phase-171 band
# contract — the row sits in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# …and the row is present with its label…
row = page.locator('tr[data-key="llm_thinking_level"]')
expect(row).to_be_visible()
expect(row.locator(".cell-clip")).to_have_text("Thinking level")
# …and the env-var display (the catalogue pattern).
expect(row.locator(".settings-env")).to_have_text("BOR_LLM_THINKING_LEVEL")
# The input is a FREE-FORM TEXT control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth).
assert row.locator(".settings-control").get_attribute("type") == "text"
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair).
thinking_type = row.locator(".settings-control").get_attribute("type")
chat_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert thinking_type == chat_type == "text"
# The default: the env-pinned ``low`` with the env badge (the
# env var is set — no app_settings row, no code-default badge).
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# Nothing is saved yet: the row's Reset is disabled (no saved
# override — the phase-168 render contract).
expect(row.locator(".settings-reset")).to_be_disabled()
# ---------------------------------------------------------------------------
# 2. The save: dirty → save → DB badge, the effective value is the
# free-form custom, and it survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_level_persists_without_restart(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type ``high`` — the dirty hint
# appears (one unsaved change on the row)…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
# …then type the free-form custom value once (A4 — NOT a known
# pi level: no allowlist, the endpoint is the source of truth).
row.locator(".settings-control").fill(SAVED_LEVEL)
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then the badge flips to DB (the
# re-GET settled the row).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
# The effective value reads ``think-soft`` — the API catalogue
# (what the LLM layer reads per request via get_settings)…
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "db" and item["value"] == SAVED_LEVEL
# …and the raw app_settings row is stored RAW (the overlay's
# D1 env-native string form).
with SessionLocal() as db:
raw = db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar_one()
assert raw == SAVED_LEVEL, f"the row must be stored raw (got {raw!r})"
# No restart: the DB value survives a reload of /settings.html
# (the panel re-reads the settled server state) with the DB
# badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-control')
).to_have_value(SAVED_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Reset: control goes back to the environment (the per-row
# DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_default(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish a saved override first (the Reset is disabled with
# nothing to reset — the phase-168 render contract): save
# ``high`` so the row's Reset goes live…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
expect(row.locator(".settings-reset")).to_be_enabled()
# The per-row Reset (the per-row DELETE): the status line…
row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_THINKING_LEVEL is back to the environment value.", timeout=30_000
)
# …the value reads ``low`` again with the env badge (the env
# var is set — not the code-default badge)…
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# …and the server agrees: the override is gone and the
# effective value is the env value again.
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "env" and item["value"] == "low"
with SessionLocal() as db:
assert (
db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar()
== 0
), "the DELETE must remove the row"
tests/integration/test_admin_settings_api.py
+86
View File
@@ -516,3 +516,89 @@ def test_catalog_covers_every_settings_field(admin_client) -> None:
assert r.status_code == 200
assert {item["key"] for item in r.json()["items"]} == set(Settings.model_fields)
assert set(CATALOG) == set(Settings.model_fields)
def test_llm_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the new ``llm_thinking_level`` row behaves
exactly like ``llm_chat_model`` — the machinery is generic over
the catalogue (D4 / A5), so the per-key contract is: (a) present in
the GET body (group ``LLM (aipi)``, ``str``, effective ``low``
with no row in ``app_settings``, ``env_var`` named); (b) PUT stores
the RAW string in ``app_settings`` (the overlay's env-native
string form) and the response reports the new effective value;
(c) the EMPTY value is storable (the off switch, A2/A4 — the API
stores as-is, no allowlist); (d) DELETE resets to the env value;
(e) the non-admin 403 gate applies (asserted once here; the full
four-route matrix is ``test_auth_matrix_all_four_routes``).
``test_catalogue_shape_and_sources_on_empty_table`` +
``test_catalog_covers_every_settings_field`` already pin the row's
PRESENCE generically — these cases pin the per-key behavior.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
# (a) the row: group, type, env var, effective ``low`` with NO row
# in app_settings, not secret, not restart-required (the LLM layer
# reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == "low"
assert item["source"] in ("default", "env") # no row in app_settings
assert item["has_override"] is False
assert item["db_value"] is None
assert item["default"] == "low"
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable (the off switch, A2/A4 — the
# API stores as-is; no allowlist, no value validation): the row
# holds the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``low`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == "low"
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_thinking_level == "low"
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
tests/integration/test_admin_settings_live.py
+45 -1
View File
@@ -9,7 +9,10 @@ per-use read, ``app/api/chat.py``'s ``plan_turn`` gate), the shared
snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
the ``scripts/``-consumer surface (``import_extension_set``).
the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_thinking_level`` — the value the LLM
layer reads per request via ``get_settings()``; the off switch is
live-settable, A2).
The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` —
@@ -160,6 +163,47 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
assert llm.settings.llm_chat_model == "gpt-second"
# ---------------------------------------------------------------------------
# the thinking level: save/reset resolve the live value (phase 172, task 02)
# ---------------------------------------------------------------------------
def test_llm_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the thinking level follows the overlay
live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE
call after the write) resolves
``get_settings().llm_thinking_level`` WITHOUT a restart (the proof
the LLM layer — task 03 reads the setting per request via
``get_settings()`` — picks the override up live), the EMPTY value
resolves live too (the off switch is live-settable, A2/A4), and a
reset (the DELETE) returns the env value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
_save(db, "llm_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_thinking_level", "")
invalidate_settings()
assert get_settings().llm_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
# ---------------------------------------------------------------------------
# the chat concurrency semaphore follows the saved cap
# ---------------------------------------------------------------------------
tests/unit/test_chat_gate.py
+44 -4
View File
@@ -1547,22 +1547,33 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
``suggested=N`` immediately AFTER ``summary_hits=N`` — the field
order of every existing field is untouched (the phase-114
``retries=N scaffold_stripped=N`` tail stays last). The value is the
seeded suggestion tier's size (``len(plan.suggested_docs)``)."""
seeded suggestion tier's size (``len(plan.suggested_docs)``).
Phase 172 (A6): the line gains ``thinking=<level>`` immediately
BEFORE ``thinking_chars=N`` — the effective (overlay-resolved)
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off
switch)."""
_session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)]
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever(chunks))
def _last_log_line() -> str:
caplog.clear()
with caplog.at_level("INFO", logger="app.chat"):
_ask(client, "How is my Kubernetes cluster set up?")
log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert log_lines
line = log_lines[-1]
return log_lines[-1]
# The default (the fixture's settings carry the field default):
# thinking=low, immediately before thinking_chars.
line = _last_log_line()
assert "thinking=low thinking_chars=" in line
# The new slot: suggested=N right after summary_hits=N (one suggested
# doc — the single fixture doc — and one summary hit on it).
assert "summary_hits=1 suggested=1" in line
# The full field order (the phase-114 tail stays last).
# The full field order (the phase-114 tail stays last; phase 172
# inserts thinking= between sources= and thinking_chars=).
order = (
"question=",
"embed_ms=",
@@ -1576,6 +1587,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
"threshold=",
"deflected=",
"sources=",
"thinking=",
"thinking_chars=",
"tool_calls=",
"rounds=",
@@ -1589,3 +1601,31 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
pos = line.find(field)
assert pos > idx, f"{field} out of order in the per-turn log line"
idx = pos
# The effective value: a non-default setting rides the line as-is.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=high thinking_chars=" in line
# The off switch: an empty setting logs ``-`` ...
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
# ... and so does a whitespace-only one (A6: the TRIMMED value
# decides, the log field mirrors the wire's trim).
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue]
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
tests/unit/test_config.py
+28
View File
@@ -105,6 +105,34 @@ def test_llm_summary_model_env_override(monkeypatch) -> None:
assert s.llm_summary_model == "mini"
def test_llm_thinking_level_default_is_low(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 172: the model's thinking level defaults to ``low``
(``BOR_LLM_THINKING_LEVEL`` unset)."""
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_thinking_level == "low"
def test_llm_thinking_level_env_override(monkeypatch) -> None:
"""Phase 172 (A4): free-form like ``llm_chat_model`` —
``BOR_LLM_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_thinking_level == "high"
def test_llm_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 172 (A2): the EMPTY value is the off switch — it must not
be coerced back to the default (trimming happens only at the wire,
task 03; no normalization at load)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "")
s = _settings()
assert s.llm_thinking_level == ""
def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None:
"""Phase 30: document content sent to the ``lite`` model is capped at
``BOR_SUMMARY_MAX_CHARS`` (default 12 000 chars per call)."""
tests/unit/test_llm_client.py
+118
View File
@@ -1324,6 +1324,124 @@ def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch)
assert sleeps == [1.25, 1.25, 1.25]
# ---------- thinking level passthrough (phase 172, task 03) ----------
def test_chat_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): a non-empty level rides the one-shot request as
the top-level ``reasoning_effort`` parameter via the SDK's
``extra_body``."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low"
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_default_level_low_is_always_sent() -> None:
"""The default (``low``) is always sent (A2): a client built without
naming the setting still carries the parameter."""
llm, completions = _make_chat_client(_FakeCompletion("ok"))
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_trims_the_level_before_sending() -> None:
"""TRIM: a hand-edited ``.env`` value with trailing whitespace
(``"low "``) is sent trimmed — the raw stored string never reaches
the wire with surrounding whitespace."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"}
def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): a whitespace value is the off switch —
``extra_body`` is ABSENT (never None, never ``"off"``) and the
request kwargs keep the pre-phase-172 shape."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level=" "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_stream_sends_reasoning_effort_extra_body_when_level_set() -> None:
"""Phase 172 (A1): the chat turn rides the parameter too."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level="high"
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_trims_the_level_before_sending() -> None:
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" high "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): the stream's off switch — ``extra_body`` absent
(never None) and the kwargs dict keeps the pre-phase-172 shape."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=""
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs
assert set(completions.kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_stream_whitespace_level_is_the_off_switch() -> None:
"""Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" "
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs
def test_embeddings_body_never_carries_reasoning_effort() -> None:
"""Phase 172 (A3): the embeddings path is untouched — with the level
set, every POST body stays the minimal ``{model, input}`` payload;
``reasoning_effort`` never appears in it."""
service = _FakeEmbeddingsService()
llm, http = _make_client(service, llm_thinking_level="high")
asyncio.run(llm.embed(["a", "b"]))
vec = asyncio.run(llm.embed_one("c"))
assert len(vec) == 768
assert http.bodies # the requests actually happened
for body in http.bodies:
assert set(body) == {"model", "input"}
assert "reasoning_effort" not in body
assert "extra_body" not in body
# ---------- chat_stream_retried (phase 67, task 01) ----------
_RETRY_MSGS: list[dict[str, str]] = [{"role": "user", "content": "q"}]
tests/unit/test_settings_catalog.py
+15
View File
@@ -93,6 +93,21 @@ def test_env_var_is_bare_key_upper_prefixed() -> None:
assert env_var(key).startswith("BOR_")
def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None:
"""Phase 172 (task 01): the new ``llm_thinking_level`` pair sits in
``LLM (aipi)`` with the ``Thinking level`` label and is plain —
not ``secret`` (a free-form text value, like ``llm_chat_model``),
not ``restart_required`` (the LLM client reads the setting per
request via ``get_settings()`` — live), overridable (the overlay is
generic over ``str`` fields)."""
meta = CATALOG["llm_thinking_level"]
assert meta.group == "LLM (aipi)"
assert meta.label == "Thinking level"
assert "llm_thinking_level" not in SECRETS
assert "llm_thinking_level" not in RESTART_REQUIRED
assert "llm_thinking_level" not in NON_OVERRIDABLE
def test_labels_nonempty_and_unique() -> None:
labels = [meta.label for meta in CATALOG.values()]
assert all(label.strip() for label in labels), "empty label in the catalogue"