Compare commits

..
2 Commits
Author SHA1 Message Date
ducoterra 149e753952 phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Build and Push Containers / build-and-push-db (push) Successful in 14s
## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
2026-10-11 23:38:18 -04:00
ducoterra c6223c93dc chore(agent): phase roadmap from TODO.md (178 split thinking levels)
LLM-Generated: true
2026-10-11 20:56:16 -04:00
45 changed files with 2189 additions and 544 deletions
@@ -0,0 +1,49 @@
# Phase 178 — Split thinking levels: a chat level and a summary level, separately tunable
**Source:** `TODO.md` L3 — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-11)
**Context:** `app/config.py` L108–115 — `llm_thinking_level: str = "low"` (the phase-172 shared knob, directly after `llm_summary_model` L103–107, before `stream_thinking` L116), `app/core/settings_catalog.py` L154–155 — the single `llm_thinking_level` `SettingMeta` row (label "Thinking level", group `LLM (aipi)`; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes every `Settings` field a build failure until it has a `SettingMeta`), `.env.example` L61 — `BOR_LLM_THINKING_LEVEL=low`, `app/rag/llm.py` — `_thinking_extra_body(settings)` (L49–57, reads `settings.llm_thinking_level`), `LLMClient.chat()` (L470, the one-shot `chat.completions.create` L430–452 — `extra = _thinking_extra_body(self.settings)` L432, `**extra_kwargs` L444; callers: `app/rag/summarizer.py` L242 + L359 document summaries, `app/rag/folder_summaries.py` L347, `app/rag/overview.py` L254 KB overview, `check_models` L805 the pre-sync probe), `LLMClient.chat_stream()` (L511, the kwargs dict L615–622 + `kwargs["extra_body"] = extra` L625–628), `app/api/chat.py` — the per-turn log line (L1272–1291, `thinking=%s` L1275, `settings.llm_thinking_level.strip() or "-"` L1289), the module docstring L14–19, `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: `str` catalogue entries render as free-form text rows automatically), `tests/unit/test_config.py` L108–133 (default / env override / empty-stays-empty), `tests/unit/test_settings_catalog.py` L99–111, `tests/unit/test_llm_client.py` L1330–1440 (the mocked-client `extra_body` family), `tests/unit/test_chat_gate.py` L1550–1630 (the per-turn log-line family), `tests/integration/test_admin_settings_api.py` L521–607 + `test_admin_settings_live.py` L171–204, `tests/e2e/test_thinking_level.py` (the phase-172 Playwright suite — the pattern to extend).
## Objective
The operator tunes the chat turns' thinking level and the one-shot summaries' thinking level **independently**: `llm_chat_thinking_level` (renamed from the phase-172 `llm_thinking_level`, default `low` — chat stays at low, TODO L3) rides `chat_stream()`; `llm_summary_thinking_level` (default empty = **off** — the TODO's "none", the phase-172 locked A2 off-switch: the parameter is ABSENT from the request) rides the one-shot `chat()` (document summaries, folder summaries, KB overview, the `check_models` probe). Both are free-form text, both live in `.env` (`BOR_LLM_CHAT_THINKING_LEVEL` / `BOR_LLM_SUMMARY_THINKING_LEVEL`) and the admin Settings panel, both live-apply without a restart.
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the two fields (task 01).** `app/config.py` LLM section: `llm_thinking_level` is **renamed** to `llm_chat_thinking_level` (default stays `"low"`) and `llm_summary_thinking_level: str = ""` is added next to it (both before `stream_thinking`). Plain `str`, no allowlist, no validator (phase-172 A4, kept). House docstrings name the env var, the default, the wire mapping and the off rule. `.env.example` L61: `BOR_LLM_THINKING_LEVEL=low` is **replaced** by `BOR_LLM_CHAT_THINKING_LEVEL=low` + `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty value, one-line comment each).
- **D2 — the consumers rename with the field (task 01, behavior-identical).** Every consumer of the old field is mechanically renamed to `llm_chat_thinking_level` in task 01 — `app/rag/llm.py` (module docstring, `_thinking_extra_body`), `app/api/chat.py` (module docstring, per-turn log line L1289) — and every test pin is renamed in place (unit `test_config.py` / `test_settings_catalog.py` / `test_llm_client.py` / `test_chat_gate.py`, integration `test_admin_settings_api.py` / `test_admin_settings_live.py`, e2e `test_thinking_level.py`). After task 01 the suite is green and behavior is **byte-identical** to pre-phase: both `chat()` and `chat_stream()` ride the (renamed) chat level, default `low`; the new summary field exists but is inert until task 03.
- **D3 — the wire split (task 03).** `_thinking_extra_body` takes the **level string** (pure function, no `Settings` read): `chat_stream()` passes `self.settings.llm_chat_thinking_level`; `chat()` passes `self.settings.llm_summary_thinking_level`. Trimmed non-empty → `{"reasoning_effort": <value>}`; empty/whitespace → `None` (the parameter is ABSENT — never a literal `"none"` or `"off"`, phase-172 A2). The `check_models` probe (L805) goes through `chat()` → summary level → off by default; no special-casing (it is a reachability ping). Embeddings path untouched.
- **D4 — the admin rows (task 02).** The single `SettingMeta("llm_thinking_level", …, "Thinking level")` becomes TWO rows: `llm_chat_thinking_level` → "Chat thinking level", `llm_summary_thinking_level` → "Summary thinking level" (both group `LLM (aipi)`, placed where the old row sat — after `llm_summary_model`, before `stream_thinking`). The panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes and the DB-over-env overlay are generic over the catalogue — the two free-form text rows, their save/reset and live-apply fall out with zero extra code. The completeness pin is the enforcement.
- **D5 — the per-turn line (task 04).** The per-turn log line's `thinking=<value>` (or `-` when empty) reads the **chat** level — the level the turn actually ran at; the summary level is a one-shot concern and gains no log field (the summary log lines have no knob field today). No `query_log` schema change (phase-172 A6, kept).
- **NOT touched:** `frontend/assets/settings.js`, `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events), every other setting.
## Dependencies
— (none: the app is fully functional at the head of `complete/`; this phase refines the committed phase-172 thinking-level machinery)
## Tasks
1. `01_settings_catalog_env.md` — the rename + the new field in `app/config.py` + the two `CATALOG` rows + `.env.example` + the mechanical consumer/test rename (suite green, behavior byte-identical).
2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for BOTH keys (stored raw, live without restart, empty storable as the summary off-switch).
3. `03_llm_extra_body_split.md` — `chat_stream()` → chat level, `chat()` → summary level; the mocked-client unit tests per method.
4. `04_per_turn_log_field.md` — `thinking=<chat level>` (or `-`) pinned: the line follows the chat knob, never the summary knob.
5. `05_e2e_admin_settings_rows.md` — `tests/e2e/test_split_thinking_levels.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_config.py` — chat default `low` / env override / empty-stays-empty; summary default `""` / env override / empty-stays-empty (no coercion, no allowlist — D3/A2). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (both fields have their `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat_stream()` sends `extra_body={"reasoning_effort": <chat level>}` when the chat level is trimmed-non-empty and no `extra_body` when it is empty; `chat()` sends it from the **summary** level with the same absent-when-empty rule; an empty summary level (the default) keeps one-shot requests byte-identical to pre-phase-172; the embeddings path is unaffected. `tests/unit/test_chat_gate.py` — the per-turn line shows the chat level (with the summary level deliberately different — it must not leak into the line).
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — BOTH rows appear in `GET /api/admin/settings` (`str`, correct labels, effective defaults `low` / ``); `PUT` stores the raw string on each, the empty value is storable (the summary off-switch), the overlay resolves live without a restart; `DELETE` resets each to env.
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_split_thinking_levels.py`, run **in isolation** — both rows visible in the `LLM (aipi)` group (labels "Chat thinking level" / "Summary thinking level"), chat default `low`, summary default empty, both free-form text inputs (type, save, survives reload), both reset-to-env restore the defaults, the env-var display pins `BOR_LLM_CHAT_THINKING_LEVEL` / `BOR_LLM_SUMMARY_THINKING_LEVEL`; the `test_admin_settings_panel.py` + `test_admin_settings_layout.py` regression set stays green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest -k "config and thinking" -v` shows chat default `low` + summary default `""` (env overrides for both).
- [ ] `GET /api/admin/settings` (admin) lists `llm_chat_thinking_level` and `llm_summary_thinking_level` in the `LLM (aipi)` group (`str`, defaults `low` / empty); `PUT` / `DELETE` round-trip each per the overlay contract; the old `llm_thinking_level` key no longer appears.
- [ ] With the levels set, `chat_stream()` carries `reasoning_effort` from the chat level and `chat()` from the summary level (unit-asserted via the mocked client); with a level empty, that method's request kwargs contain no `extra_body` (byte-identical to pre-phase-172 for the default one-shot path).
- [ ] The per-turn log line carries `thinking=<chat level>` (`-` when empty) — never the summary level.
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
- [ ] `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation.
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): split the thinking level into a chat level (default low) and a summary level (default off)`); the phase dir moves to `.agents/phases/complete/`.
## Locked decisions (owner-confirmed 2026-10-11, roadmap confirmation)
- **A1 — rename, not add-on:** `llm_thinking_level` is **renamed** to `llm_chat_thinking_level` (`BOR_LLM_THINKING_LEVEL` → `BOR_LLM_CHAT_THINKING_LEVEL`). Any DB-stored override under the old key becomes inert (it is no longer a catalogue row) and chat falls back to the env default `low` — chat behavior is preserved exactly as the TODO wants; no value migration.
- **A2 — the TODO's "none" = the phase-172 off-switch:** `llm_summary_thinking_level` defaults to empty (`""`) → `reasoning_effort` is ABSENT from one-shot `chat()` requests. Per phase-172's locked A2 ("nothing is sent as `off` or `null` — absence is the off"), no literal `"none"` string is ever sent.
- **A3 — scope:** `chat_stream()` (the chat turn) → chat level; every one-shot `chat()` (document summaries, folder summaries, KB overview, the `check_models` probe) → summary level; the embeddings path is untouched.
- **A4 — no frontend code change:** the panel is generated from the catalogue; the two free-form text rows appear automatically (phase-172 A5 pattern).
- **A5 — the per-turn line** keeps its single `thinking=<chat level>` field (`-` when empty), reads the chat level, and gains no summary field; no `query_log` schema change.
@@ -0,0 +1,69 @@
# Task 01 — The two settings + the two catalogue rows + `.env.example` + the mechanical rename
**Phase:** `178_split_thinking_levels` · **Source:** `TODO.md:3` — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a
## Objective
The two settings exist in `app/config.py` (chat default `low`, summary default empty = off, both free-form), in the admin catalogue (the completeness pin stays green), and in `.env.example` — and every existing consumer/test of the old single `llm_thinking_level` is renamed so the suite is green and behavior is **byte-identical** to pre-phase (both LLM paths still ride the chat level, default `low`).
## Work
1. `app/config.py` — in the LLM section, where the phase-172 field sits (L108–115, directly after the `llm_summary_model` block, before `stream_thinking` L116): **rename** `llm_thinking_level` → `llm_chat_thinking_level` (default stays `"low"`) and add the summary field next to it. House docstring style (env var named, default, wire mapping, off rule):
```python
#: The chat turn's thinking level (phase 178,
#: ``BOR_LLM_CHAT_THINKING_LEVEL``; the phase-172 knob, renamed) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides the streaming chat-completions
#: requests to aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py`` ``chat_stream``); the EMPTY value is the off
#: switch — the parameter is not sent at all.
llm_chat_thinking_level: str = "low"
#: The one-shot summary thinking level (phase 178,
#: ``BOR_LLM_SUMMARY_THINKING_LEVEL``) — rides the one-shot
#: ``chat()`` calls (document summaries, folder summaries, KB
#: overview, the ``check_models`` probe) as ``reasoning_effort``.
#: Default EMPTY = the off switch (phase-172 A2): the parameter is
#: not sent at all; the TODO's "none" means absence, never a
#: literal "none" string.
llm_summary_thinking_level: str = ""
```
No validator, no coercion — plain `str` (phase-172 A4, kept). Verify by reading: `llm_summary_model` (L103–107) and `stream_thinking` (L116–120) keep their shape byte-identical.
2. `app/core/settings_catalog.py` — replace the single row (L154–155) with TWO rows, placed exactly where the old row sat (after `llm_summary_model` L151–152, before `stream_thinking` L157):
```python
"llm_chat_thinking_level": SettingMeta(
"llm_chat_thinking_level", "LLM (aipi)", "Chat thinking level"
),
"llm_summary_thinking_level": SettingMeta(
"llm_summary_thinking_level", "LLM (aipi)", "Summary thinking level"
),
```
Nothing else: the panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes and the overlay are generic over the catalogue (D4).
3. `.env.example` — in the LLM block (L57–65), replace `BOR_LLM_THINKING_LEVEL=low` (L61) with:
```
BOR_LLM_CHAT_THINKING_LEVEL=low # the chat turn's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
BOR_LLM_SUMMARY_THINKING_LEVEL= # the one-shot summary thinking level (document/folder summaries, KB overview); empty (default) = the parameter is not sent
```
directly under `BOR_LLM_CHAT_MODEL=turbo` (L60), house comment style.
4. `app/rag/llm.py` — **mechanical rename, behavior-identical** (D2): the module docstring (L14–19 — rename `BOR_LLM_THINKING_LEVEL` / `default low` to `BOR_LLM_CHAT_THINKING_LEVEL`; the "rides every chat-completions request (one-shot `chat` and streaming `chat_stream`)" sentence stays TRUE after this task because both paths still ride the chat level; task 03 finalizes the wording) and `_thinking_extra_body` (L49–57, `settings.llm_thinking_level` → `settings.llm_chat_thinking_level`; docstring names the chat level). `chat()` (L432) and `chat_stream()` (L625) keep calling the same helper with the chat level — no split yet (that is task 03). The embeddings path is untouched.
5. `app/api/chat.py` — **mechanical rename, behavior-identical** (D2): the module docstring (L14–19, `BOR_LLM_THINKING_LEVEL` → `BOR_LLM_CHAT_THINKING_LEVEL`, "the effective chat level") and the per-turn log line argument (L1289, `settings.llm_thinking_level` → `settings.llm_chat_thinking_level`; the format string L1275 keeps `thinking=%s`).
6. `tests/unit/test_config.py` — L108–133: the three phase-172 tests become SIX: `llm_chat_thinking_level` (default `low` with `BOR_LLM_CHAT_THINKING_LEVEL` unset; env override `high`; empty env value stays empty) and `llm_summary_thinking_level` (default `""` with `BOR_LLM_SUMMARY_THINKING_LEVEL` unset; env override `high`; empty env value stays empty). Same monkeypatch pattern, no coercion asserted.
7. `tests/unit/test_settings_catalog.py` — L99–111: the single-row test becomes a BOTH-rows test (each `SettingMeta` in the `LLM (aipi)` group with the correct label, `str`-ish flags — not in `SECRETS`, not in `RESTART_REQUIRED`, not in `NON_OVERRIDABLE`); the completeness pin (`set(CATALOG) == set(Settings.model_fields)`) must stay green.
8. `tests/unit/test_llm_client.py` — L1330–1440: rename the kwarg `llm_thinking_level=` → `llm_chat_thinking_level=` in the existing family (behavior assertions unchanged — the helper still reads the chat level); the `chat()`-path tests keep passing as-is.
9. `tests/unit/test_chat_gate.py` — L1550–1630: rename the `Settings(..., llm_thinking_level=…)` pins → `llm_chat_thinking_level=…`; the log-line assertions (L1553, `BOR_LLM_THINKING_LEVEL` docstring) name `BOR_LLM_CHAT_THINKING_LEVEL` / the chat level.
10. `tests/integration/test_admin_settings_api.py` — L521–607: the `llm_thinking_level` row contract is renamed to `llm_chat_thinking_level` (env var `BOR_LLM_CHAT_THINKING_LEVEL`, default `low`; PUT/DELETE/anonymous-401 pins unchanged in shape). Task 02 extends it for the summary row.
11. `tests/integration/test_admin_settings_live.py` — L171–204: the live-apply test is renamed to `llm_chat_thinking_level` (same save/empty/delete pattern against the new key).
12. `tests/e2e/test_thinking_level.py` — rename the pins in place: `tr[data-key="llm_thinking_level"]` → `tr[data-key="llm_chat_thinking_level"]`, `BOR_LLM_THINKING_LEVEL` → `BOR_LLM_CHAT_THINKING_LEVEL`, the `Settings.model_fields["llm_thinking_level"]` env lookup → `llm_chat_thinking_level`. E2E runs only in isolation (the per-task gate covers unit + integration), but the file must not rot: task 05 turns it into `test_split_thinking_levels.py`.
- ASSUMPTION: A1 (phase-locked) — the rename orphans any stored `app_settings` row under `llm_thinking_level` (it becomes inert; chat falls back to env default `low`); no value migration is performed.
- ASSUMPTION: A2 (phase-locked) — the summary default is the empty string (the phase-172 off-switch: `reasoning_effort` ABSENT), never a literal `"none"`.
## Testing & Quality
- Unit: `tests/unit/test_config.py` (six thinking-level tests), `tests/unit/test_settings_catalog.py` (the completeness pin + both rows), `tests/unit/test_llm_client.py` (renamed family, unchanged assertions), `tests/unit/test_chat_gate.py` (renamed pins).
- Integration: `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (renamed row contract, green).
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
## Completion Criteria
- [ ] `grep -rn "llm_thinking_level\b\|BOR_LLM_THINKING_LEVEL\b" app/ tests/ .env.example` returns **nothing** (the old name is gone everywhere; `llm_chat_thinking_level` / `BOR_LLM_CHAT_THINKING_LEVEL` in its place).
- [ ] `uv run pytest -k "config and thinking" -v` — chat default `low` + summary default `""` + env overrides for both.
- [ ] `uv run pytest` green (unit + integration), `app/` coverage >90%, `uv run ruff check . && uv run pyright` clean.
- [ ] Behavior check (read, not run): `chat()` and `chat_stream()` both still ride the chat level; with both defaults (`low` / empty) and the old `.env` value, streaming requests are byte-identical to pre-phase-172.
- [ ] No behavior change in completed work.
@@ -0,0 +1,28 @@
# Task 02 — The admin row contract: both keys, set / reset / live-apply
**Phase:** `178_split_thinking_levels` · **Source:** `TODO.md:3` — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a
## Objective
The integration contract proves both rows behave per the overlay: they appear in the catalogue with the right group/label/default, `PUT` stores the raw string (empty included — the summary off-switch), the value resolves live without a restart, and `DELETE` resets to env — for the chat row AND the summary row, independently.
## Work
1. `tests/integration/test_admin_settings_api.py` — the phase-172 row contract (now `test_llm_chat_thinking_level_row_contract`, from task 01) gains a **summary twin** `test_llm_summary_thinking_level_row_contract` following the same shape, with the summary specifics:
- `GET /api/admin/settings` lists `llm_summary_thinking_level` in the `LLM (aipi)` group, `str`, label "Summary thinking level", `env_var` `BOR_LLM_SUMMARY_THINKING_LEVEL`, effective default `""` (with `BOR_LLM_SUMMARY_THINKING_LEVEL` deleted from the env for the test — the house `monkeypatch.delenv(…, raising=False)` pattern).
- `PUT {API}/llm_summary_thinking_level` with `{"value": "high"}` → 200, the raw string stored (`SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'` → `"high"`), `get_settings().llm_summary_thinking_level == "high"` **without a restart**.
- `PUT` with `{"value": ""}` → 200 (the off-switch is storable — an empty stored value must not be rejected as "no value"), `get_settings().llm_summary_thinking_level == ""`.
- `DELETE` → 200, the row gone from `app_settings`, `get_settings().llm_summary_thinking_level == ""` (the env default).
- An anonymous (non-admin) `PUT` → the 401/403 pin, same as the chat twin.
- Independence pin: after `PUT`-ing the summary row `"high"`, the chat row's effective value is still `"low"` (the two overlays resolve independently — the whole point of the split).
2. `tests/integration/test_admin_settings_live.py` — the chat live-apply test (from task 01) gains a **summary twin** `test_llm_summary_thinking_level_live_apply_no_restart`: `delenv` `BOR_LLM_SUMMARY_THINKING_LEVEL` → default `""`; `_save(db, "llm_summary_thinking_level", "high")` → `get_settings().llm_summary_thinking_level == "high"` **NO restart**; `_save(…, "")` → `""`; `DELETE FROM app_settings WHERE key = 'llm_summary_thinking_level'` → back to `""`.
3. Verify by running: `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v` — both families green, and the old single-row test is gone (no `llm_thinking_level` key referenced anywhere: `grep -rn "llm_thinking_level\b" tests/` empty).
## Testing & Quality
- Integration: the two extended families above (row contract + live-apply, chat AND summary).
- Coverage: **>90%** on `app/` (this task is test-only — `app/` code is untouched, so the task-01 coverage must simply stay green).
## Completion Criteria
- [ ] `uv run pytest tests/integration/test_admin_settings_api.py -v -k "thinking" -v` — the chat + summary row contracts green (group/label/env-var/default, PUT raw storage incl. empty, DELETE reset, anonymous rejected, independence).
- [ ] `uv run pytest tests/integration/test_admin_settings_live.py -v -k "thinking"` — both live-apply twins green (no restart, empty storable, delete resets).
- [ ] `uv run pytest` green, `app/` coverage >90%.
- [ ] No behavior change in completed work.
@@ -0,0 +1,41 @@
# Task 03 — The wire split: `chat_stream()` → chat level, `chat()` → summary level
**Phase:** `178_split_thinking_levels` · **Source:** `TODO.md:3` — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a
## Objective
The one-shot `chat()` path (document summaries, folder summaries, KB overview, the `check_models` probe) sends `reasoning_effort` from the **summary** level — absent by default (A2) — while `chat_stream()` keeps sending it from the **chat** level (default `low`), so the two surfaces are independently tunable on the wire.
## Work
1. `app/rag/llm.py` — `_thinking_extra_body` (L49–57) becomes a pure function of the level string (drop the `Settings` read):
```python
def _thinking_extra_body(level: str) -> dict[str, str] | None:
"""The thinking level as the chat-completions extra body (phase 178).
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, phase-172 A1); empty /
whitespace → ``None`` (the off switch — the parameter is ABSENT
from the request, phase-172 A2; never a literal ``"none"`` or
``"off"``).
"""
trimmed = level.strip()
return {"reasoning_effort": trimmed} if trimmed else None
```
2. `app/rag/llm.py` — `chat()`'s single-attempt body (L432): `extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)`; the comment (L443–444) becomes "Phase 178 (A3/A2): the **summary** level rides as `reasoning_effort` (document summaries, folder summaries, KB overview, the `check_models` probe); the key is absent (never None) when off — the default (empty) keeps one-shot requests byte-identical to pre-phase-172."
3. `app/rag/llm.py` — `chat_stream()` (L625–628): `if (extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)) is not None: …` (the comment L626–627 becomes "Phase 178 (A3/A2): the **chat** level rides as `reasoning_effort`; the key is absent (never None) when off.").
4. `app/rag/llm.py` — finalize the module docstring (L1–25, rewritten in task 01): replace the single-level paragraph with the split wording — the chat turn's level (`BOR_LLM_CHAT_THINKING_LEVEL`, default `low`) rides streaming `chat_stream`; the one-shot level (`BOR_LLM_SUMMARY_THINKING_LEVEL`, default empty = off) rides `chat` (document summaries, folder summaries, KB overview, the `check_models` probe); an empty value is the off switch (parameter absent, byte-identical to pre-phase-172); the embeddings path never carries it. Note the probe's consequence: `check_models` (L805) goes through `chat()` → off by default — it is a reachability ping, no special-casing.
5. `tests/unit/test_llm_client.py` — the mocked-client family (L1330–1440, renamed in task 01):
- The `chat()`-path tests (currently `llm_chat_thinking_level="low"` / `"low "` / `" "` at L1335–1368) are re-pointed at the **summary** kwarg: `llm_summary_thinking_level="high"` → the `chat.completions.create` call carries `extra_body={"reasoning_effort": "high"}`; `" high "` → trimmed `"high"`; `""` / `" "` → the call kwargs contain **no** `extra_body` key (never `None`, never a literal "none").
- The `chat_stream()`-path tests (L1385–1422) keep their shape: `llm_chat_thinking_level="high"` / `" high "` → `extra_body={"reasoning_effort": "high"}`; `""` / `" "` → no `extra_body` key.
- Add the cross-pins (the whole point of the split): client with `llm_chat_thinking_level="high"` + `llm_summary_thinking_level=""` (the defaults inverted) → `chat_stream()` carries `reasoning_effort="high"` while `chat()` carries no `extra_body`; the default client (chat `"low"` + summary `""`) → `chat()` is byte-identical to pre-phase-172 (no `extra_body`) while `chat_stream()` carries `reasoning_effort="low"`.
- The embeddings-path test (no `extra_body` ever) stays green untouched.
## Testing & Quality
- Unit: `tests/unit/test_llm_client.py` (the re-pointed + extended family above; the mocked-SDK-client pattern).
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_llm_client.py -v -k "thinking" -v` — per-method `extra_body` behavior green (chat level → `chat_stream` only; summary level → `chat` only; absent-when-empty; cross-pins).
- [ ] Default-config read check: with chat `low` + summary `""`, a `chat()` request carries no `extra_body` and a `chat_stream()` request carries `reasoning_effort="low"` (the test pins above).
- [ ] `uv run pytest` green, `app/` coverage >90%, `uv run ruff check . && uv run pyright` clean.
- [ ] No behavior change in completed work (the streaming path is byte-identical at the defaults; the one-shot path is off by default — the TODO's "summary should be none").
@@ -0,0 +1,25 @@
# Task 04 — The per-turn line: `thinking=<chat level>`, never the summary knob
**Phase:** `178_split_thinking_levels` · **Source:** `TODO.md:3` — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a
## Objective
The per-turn log line (PLAN §9, AGENTS rule 10) is pinned to the **chat** level — the level the turn actually ran at: `thinking=<trimmed chat level>` (`-` when empty) immediately before `thinking_chars=N`, with a cross-pin proving the summary level (set to a different value) never leaks into the line. No `query_log` schema change (phase-172 A6 / phase-178 A5, kept).
## Work
1. `app/api/chat.py` — the per-turn log line (L1272–1291) already reads the chat level after task 01's mechanical rename (`settings.llm_chat_thinking_level.strip() or "-"` L1289, format `thinking=%s` L1275): verify by reading, no functional change. Finalize the inline comment (L1289) to "phase 178 (A5): the effective **chat** thinking level, trimmed, `-` when empty — the level this turn ran at; the summary level is a one-shot concern and gains no field here."
2. `app/api/chat.py` — the module docstring (L14–19, renamed in task 01): finalize the wording — "The per-turn log line records `thinking=<level>` (the effective `BOR_LLM_CHAT_THINKING_LEVEL`, trimmed, `-` when empty) before `thinking_chars` (phase 172, A6; phase 178, A5 — the **chat** level; the one-shot `BOR_LLM_SUMMARY_THINKING_LEVEL` never appears on the line)."
3. `tests/unit/test_chat_gate.py` — the log-line family (L1545–1630, pins renamed in task 01: the default `thinking=low`, the `"high"` override, the `""` → `-` off-switch, the `" "` → `-` trim pin, the full field-order pin):
- Update the test docstring (L1550–1557) to name `BOR_LLM_CHAT_THINKING_LEVEL` / the chat level (phase 178, A5).
- Add the **cross-pin** (the split's proof on the log line): `Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level="low", llm_summary_thinking_level="high")` → the line is `thinking=low thinking_chars=…` — the summary's `high` does NOT appear as the line's `thinking=` value.
- Add the inverse off-pin: `llm_chat_thinking_level=""` + `llm_summary_thinking_level="high"` → `thinking=- thinking_chars=…` (the chat off-switch logs `-` even though the summary knob is set).
## Testing & Quality
- Unit: `tests/unit/test_chat_gate.py` (the extended log-line family above).
- Coverage: **>90%** on `app/` (this task touches only the docstring/comment + tests — coverage must stay green).
## Completion Criteria
- [ ] `uv run pytest tests/unit/test_chat_gate.py -v -k "log" -v` — the full log-line family green: default `thinking=low`, override `high`, off-switch `-`, trim `-`, field order, AND the two cross-pins (summary level never leaks into `thinking=`).
- [ ] `grep -n "BOR_LLM_SUMMARY_THINKING_LEVEL" app/api/chat.py` returns nothing (the one-shot knob has no presence on the chat turn's log line, A5).
- [ ] `uv run pytest` green, `app/` coverage >90%, `uv run ruff check . && uv run pyright` clean.
- [ ] No behavior change in completed work.
@@ -0,0 +1,30 @@
# Task 05 — The E2E suite: `test_split_thinking_levels.py` + regression + gates + commit
**Phase:** `178_split_thinking_levels` · **Source:** `TODO.md:3` — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a
## Objective
The phase's dedicated Playwright suite (AGENTS rule 4 — one test file for the phase, run in isolation) proves the operator's acceptance in a real browser: TWO free-form thinking-level rows in the admin Settings panel (chat default `low`, summary default empty = the TODO's "none"), independently saveable/resettable with live-apply, plus the full phase gates and the atomic commit.
## Work
1. `tests/e2e/test_thinking_level.py` → `tests/e2e/test_split_thinking_levels.py` — `git mv` the phase-172 suite (the module scaffolding is the house pattern, kept: `_boot_app` / `_stop_app` / `_clean_settings_state` / the autouse `_clean` (truncates `app_settings` + `ui_settings`, deletes `e2e-`-labeled tokens, module teardown pins `app_settings` empty) / the `.env` mtime guard / the `_cookies` / `_admin_client` / `_catalog_item` helpers) and rewrite the module docstring (Source → `TODO.md` L3 this item; the test → contract mapping below; the env pins: the module app opts BACK IN over the conftest's global `BOR_SETTINGS_DB=0` pin (`env["BOR_SETTINGS_DB"] = "1"`), and pins `BOR_LLM_CHAT_THINKING_LEVEL=low` AND `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty string SET — so both badges read `env`, not `default`, and an operator's local `.env` cannot leak a different level into the suite; the house leak-guard pattern) and `BOR_LLM_CHAT_MODEL` (the "like chat model" reference)).
2. The three tests (renamed/extended from the phase-172 trio):
- `test_both_rows_render_free_form_with_defaults` — the `LLM (aipi)` group band is visible (the phase-171 band contract); a row labelled **Chat thinking level** with the `BOR_LLM_CHAT_THINKING_LEVEL` env-var display shows `low` with the `env` badge; a row labelled **Summary thinking level** with the `BOR_LLM_SUMMARY_THINKING_LEVEL` display shows an **empty** input with the `env` badge (the TODO's "none" = the off switch, A2); BOTH inputs are `type="text"` (free-form — NOT `number`/`select`, the A4 contract); the `llm_chat_model` row's input is the same control (the "like chat model" reference, one assertion pair); `tr[data-key="llm_thinking_level"]` is **absent** (the old key is gone, A1).
- `test_saved_levels_persist_without_restart` — the phase-168 §7.4 Save lifecycle per row: type `think-soft` (no allowlist — A4) into the summary row and `high` into the chat row (dirty hints appear), save both; the API catalogue (`_catalog_item`) AND the raw `app_settings` rows carry the saved values for EACH key independently; a reload of `/settings.html` still shows `think-soft` / `high` with the `DB` badges (no restart — the phase-168 contract); the independence pin: the two rows' controls are distinct and the saved values do not cross.
- `test_reset_returns_to_env_defaults` — the per-row Reset (the per-row `DELETE`): after the saves, both Resets are live; clicking the summary Reset lands the value at **empty** with the `env` badge, clicking the chat Reset lands `low` with the `env` badge (the overlay reset, task 02's contract, in the browser); the status lines show the house "… is back to the environment value." toast for each.
3. Regression set (in isolation, per the house E2E protocol): `uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov` (phase 168) and `uv run pytest tests/e2e/test_admin_settings_layout.py -v --no-cov` (phase 171) — both green (they do not pin the thinking rows — verified: no `thinking` references — but the two new rows must not break the panel/layout contracts).
4. The full gates: `uv run pytest` (unit + integration) green; `uv run pytest --cov=app --cov-report=term-missing` — `app/` coverage **>90%**; `uv run ruff check . && uv run pyright` clean; `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` green **in isolation** (DB up: `podman compose up -d db`).
5. Commit (AGENTS rule 8): ONE atomic Conventional Commit, `--no-gpg-sign` — e.g. `feat(llm): split the thinking level into a chat level (default low) and a summary level (default off)` — body: the TODO L3 source, the rename (A1), the wire split (A3), the log line (A5), the two admin rows; then move the phase dir: `git mv .agents/phases/todo/178_split_thinking_levels .agents/phases/complete/178_split_thinking_levels` and include it in the commit (the `.agents/` tree is tracked — only `.agents/phase-sessions/` + `.agents/pipeline.log` are gitignored).
6. `TODO.md` — ALREADY CLEARED by the conversion (the file holds only the `# TODO` title; verify by reading — do not re-add the item).
## Testing & Quality
- E2E (AGENTS rule 4): `tests/e2e/test_split_thinking_levels.py` run **in isolation** (the three tests above).
- Coverage: **>90%** on `app/`.
- Lint/types: ruff + pyright clean.
## Completion Criteria
- [ ] `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` green **in isolation** (both rows, defaults `low`/empty, free-form, save + reload persistence per key, reset to env defaults, old key absent).
- [ ] `uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov` and `uv run pytest tests/e2e/test_admin_settings_layout.py -v --no-cov` green in isolation (regression).
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
- [ ] One atomic `--no-gpg-sign` Conventional Commit including the `.agents/` phase move; `TODO.md` holds only the `# TODO` title.
- [ ] No behavior change in completed work.
@@ -0,0 +1,20 @@
## Phase 178 — final verification pass: 100% green
**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation
**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.
**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.
**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).
**Next pending phase:** none — 178 is the last phase in `todo/`.
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,22 @@
All complete and verified. `image-1.png` is a pre-existing untracked file (created Oct 11 09:59, before this task — not touched by me).
**Report**
**Implemented (phase 178, task 01):**
- `app/config.py`: `llm_thinking_level` → `llm_chat_thinking_level` (default `"low"`), new `llm_summary_thinking_level` (default `""`), house docstrings, no validator
- `app/core/settings_catalog.py`: one row → two rows ("Chat thinking level" / "Summary thinking level", `LLM (aipi)`)
- `.env.example`: `BOR_LLM_CHAT_THINKING_LEVEL=low` + `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty)
- `app/rag/llm.py` / `app/api/chat.py`: mechanical rename (docstrings, `_thinking_extra_body`, per-turn log arg) — both LLM paths still ride the chat level, byte-identical behavior
- Tests renamed/extended in place: `test_config.py` (3→6 tests), `test_settings_catalog.py` (both-rows test, completeness pin intact), `test_llm_client.py`, `test_chat_gate.py`, `test_admin_settings_api.py`, `test_admin_settings_live.py`, e2e `test_thinking_level.py` (incl. the now-correct "Chat thinking level" label pin)
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (chat `low`/summary `""` defaults + env overrides)
- `uv run pytest --cov=app --cov-report=term-missing` → 4164 passed, 99% coverage (>90%)
- `uv run ruff check . && uv run pyright` → clean (0 errors, 0 warnings)
- `grep -rn "llm_thinking_level\b|BOR_LLM_THINKING_LEVEL\b" app/ tests/ .env.example README.md` → nothing (old name gone everywhere)
**Docs:** README env table updated (one `BOR_LLM_THINKING_LEVEL` row → the two renamed/split rows); no AGENTS.md impact.
**Decisions:** A1/A2 honored — rename orphans stored old-key overrides (no migration); summary default is empty = absent parameter, never literal "none". Wire split deferred to task 03 as designed.
**Next pending task:** `02_admin_row_contract.md`
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
............................................................ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,19 @@
**Task 02 complete — report**
**Implemented (test-only, `app/` untouched):**
- `tests/integration/test_admin_settings_api.py`: added `test_llm_summary_thinking_level_row_contract` — summary row in `LLM (aipi)` (label "Summary thinking level", `str`, `BOR_LLM_SUMMARY_THINKING_LEVEL`, effective `""` with `delenv`), PUT raw storage (`"high"`), empty storable as the off-switch, DELETE reset to `""`, anonymous 403 pin, and the independence pin (summary `"high"` → chat still `"low"`, verified via `get_settings()` and the GET item).
- `tests/integration/test_admin_settings_live.py`: added `test_llm_summary_thinking_level_live_apply_no_restart` — `delenv` → `""`, `_save "high"` → live `high` (no restart), `_save ""` → `""`, DELETE → `""`; also updated the module docstring to name the summary twin.
- Removed a stale `__pycache__/test_thinking_level.*.pyc` artifact so the `llm_thinking_level\b` grep pin is clean.
**Results:**
- `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v` → 24 passed
- `… -k "thinking"` on each file → 2 passed each (4 total: both row contracts + both live-apply twins)
- `uv run pytest --cov=app --cov-report=term-missing` → 4166 passed, 0 failed, coverage 99% (>90%)
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
- `grep -rn "llm_thinking_level\b" tests/ app/ .env.example` → empty
**Docs:** no doc impact (README already current from task 01; no user-visible changes in a test-only task).
**Notable:** the independence pin `delenv`s both env vars for a machine-independent base (`.env` sets no thinking vars, so `source == "default"` is deterministic).
**Next pending task:** `.agents/phases/todo/178_split_thinking_levels/03_llm_extra_body_split.md`
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.............................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,21 @@
All complete and green. The README's env-var table (L1164–1165, written in task 01) already describes the split accurately — no doc changes needed.
## Report
**Implemented (task 03 — the wire split):**
- `_thinking_extra_body` is now a pure function of the level string (no `Settings` read); trimmed non-empty → `{"reasoning_effort": <value>}`, empty/whitespace → `None` (parameter absent, never a literal "none")
- One-shot `chat()` path (via `_chat_once`) passes `llm_summary_thinking_level` — document summaries, folder summaries, KB overview, `check_models` probe (off by default, no special-casing)
- `chat_stream()` passes `llm_chat_thinking_level` (default `low` unchanged); embeddings path untouched
- Module docstring finalized with the split wording incl. the probe consequence
- `tests/unit/test_llm_client.py`: `chat()` family re-pointed at the summary kwarg (`high` / trim / `""` / whitespace-absent, full pre-172 kwargs shape), `chat_stream()` family kept, two cross-pins (defaults-inverted: chat `high` + summary `""` → only the stream carries it; defaults: stream `low`, one-shot no `extra_body`); embeddings test untouched
**Gates (all 100% green):**
- `uv run pytest tests/unit/test_llm_client.py -k "thinking" -v` → 15 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 4169 passed, 0 failed; `app/` coverage **99%** (>90%)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
**Docs:** no impact — README env table (task 01) already matches the new behavior; AGENTS.md has no thinking-level lines.
**Decisions:** one client-per-cross-pin helper (`_make_both_client`) since the fake `completions` serves both `create()` shapes; test names carry `thinking` so the task's `-k "thinking"` gate selects the whole family.
**Next pending task:** `04_per_turn_log_field.md`
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,20 @@
All gates pass. Done.
## Report — Task 04: per-turn log line pinned to the chat level
**Implemented:**
- `app/api/chat.py` docstring finalized: per-turn line records `thinking=<level>` (effective `BOR_LLM_CHAT_THINKING_LEVEL`, trimmed, `-` when empty) before `thinking_chars` (phase 172, A6; phase 178, A5 — the **chat** level; the one-shot summary level never appears on the line).
- Inline comment at the log site finalized: "phase 178 (A5): the effective chat thinking level, trimmed, `-` when empty — the level this turn ran at; the summary level is a one-shot concern and gains no field here." No functional change to the line.
- `tests/unit/test_chat_gate.py`: docstring updated to name `BOR_LLM_CHAT_THINKING_LEVEL` / chat level (phase 178, A5); added the **cross-pin** (chat `low` + summary `high` → `thinking=low`, `thinking=high` absent) and the **inverse off-pin** (chat `""` + summary `high` → `thinking=-`, `thinking=high` absent).
**Test / lint / coverage (all green):**
- `uv run pytest tests/unit/test_chat_gate.py -k "log" -v` → 1 passed (full log-line family: default `low`, `high` override, `""`→`-`, trim→`-`, field order, both cross-pins)
- `grep -n "BOR_LLM_SUMMARY_THINKING_LEVEL" app/api/chat.py` → no output (exit 1)
- `uv run pytest -q` → exit 0 (100% green); `uv run pytest --cov=app` → `app/` TOTAL **99%** (>90%)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
**Docs:** no doc impact (docstring/comment + tests only; no new env var, command, feature, or API).
**Deviation:** the task's work item 2 specified docstring wording containing `BOR_LLM_SUMMARY_THINKING_LEVEL`, but the completion criterion requires that grep to return nothing in `app/api/chat.py`. The verifiable grep gate was prioritized — the docstring reads "the one-shot **summary level** never appears on the line" (identical meaning, name omitted).
**Next pending task:** `05_e2e_admin_settings_rows.md`
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,8 @@
**Task 05 complete — report**
- **Implemented:** `tests/e2e/test_thinking_level.py` → `test_split_thinking_levels.py` (phase 178 Playwright suite, 3 tests): both rows render free-form (`type="text"`, `LLM (aipi)` band, labels/env-var displays, chat `low`/summary empty with `env` badges, old `llm_thinking_level` row absent, "like chat model" reference pair); per-key independent save (`think-soft`/`high`) with catalogue + raw `app_settings` pins and reload persistence (DB badges, no restart); per-row reset to env defaults (summary empty / chat `low`, env badges, house toasts, no raw rows). Env pins: `BOR_SETTINGS_DB=1`, `BOR_LLM_CHAT_THINKING_LEVEL=low`, `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty SET), `BOR_LLM_CHAT_MODEL`.
- **Notable fix:** the suite's direct-SQL `TRUNCATE app_settings` does not invalidate the app's `get_settings()` lru-cache (the admin API is the only invalidating mutation surface) — test 3 typed the same values test 2 saved, saw the stale value as "served", registered no dirty state. Fixed in-suite: `_clean` now follows each truncate with the app's own `DELETE /api/admin/settings` (zero rows, cache rebuilds env-only). No app-code change. Also pinned the reset-rows settle (DB badge before `to_be_enabled`) to kill the save-click race.
- **Gates (all green):** `uv run pytest` → 4169 passed; `uv run pytest --cov=app --cov-report=term-missing` → 4169 passed, TOTAL **99%** (>90%); `uv run ruff check . && uv run pyright` → 0 errors; `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation (run ×3, stable); regression `test_admin_settings_panel.py` → 12 passed, `test_admin_settings_layout.py` → 5 passed.
- **Docs:** README already updated by tasks 01–04 (both env vars documented, no stale `BOR_LLM_THINKING_LEVEL` refs); no AGENTS.md impact.
- **Deviations (harness-owned):** no commit / no phase-dir move made — left in the working tree per my rules. `TODO.md` still holds one *external* owner item added 22:40 today ("long git clone freezes the server?") — predates/outsides this phase, L3 is cleared and not re-added; left the item untouched. Stray `image-1.png` (repo root, 09:59, pre-phase) left alone.
- **Next pending task:** none — task 05 was the last of phase 178.
@@ -0,0 +1,136 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
................................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 280 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 234 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 289 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 111 0 100%
app/core/theming.py 40 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 141 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 841 4 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 533 0 100%
--------------------------------------------------
TOTAL 6468 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
+2 -1
View File
@@ -58,7 +58,8 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1 BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed" BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
BOR_LLM_CHAT_MODEL=turbo BOR_LLM_CHAT_MODEL=turbo
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent BOR_LLM_CHAT_THINKING_LEVEL=low # the chat turn's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
BOR_LLM_SUMMARY_THINKING_LEVEL= # the one-shot summary thinking level (document/folder summaries, KB overview); empty (default) = the parameter is not sent
# BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off # BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off
# BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120) # BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120)
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls) # BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls)
+2 -1
View File
@@ -1161,7 +1161,8 @@ startup (the DB rows shadow the env, `.env` is never rewritten).
| `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` | | `BOR_LLM_CHAT_MODEL` | `turbo` | Chat model — must support vision when `BOR_IMAGES=1` |
| `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model | | `BOR_LLM_EMBED_MODEL` | `embed` | Embedding model |
| `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries | | `BOR_LLM_SUMMARY_MODEL` | `lite` | One-shot completions: document summaries at import, KB overview, folder summaries |
| `BOR_LLM_THINKING_LEVEL` | `low` | The model's thinking level (pi-style, free-form — no allowlist); rides every chat-completions request as `reasoning_effort`; empty = the parameter is not sent | | `BOR_LLM_CHAT_THINKING_LEVEL` | `low` | The chat turn's thinking level (pi-style, free-form — no allowlist); rides the streaming chat-completions requests as `reasoning_effort`; empty = the parameter is not sent |
| `BOR_LLM_SUMMARY_THINKING_LEVEL` | *(empty)* | The one-shot summary thinking level (document/folder summaries, KB overview, the model probe); rides the one-shot `chat()` calls as `reasoning_effort`; empty (default) = the parameter is not sent |
| `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) | | `BOR_EMBEDDING_DIM` | `768` | Vector dimension (fixed at table creation) |
| `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off | | `BOR_LLM_RETRIES` | `3` | Retries: chat-turn stream before the first token + empty one-shot summary replies; `0` = off |
| `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries | | `BOR_LLM_RETRY_DELAY` | `5` | Seconds between LLM retries |
+2
View File
@@ -1 +1,3 @@
# TODO # TODO
- [ ] A long git clone seems to freeze the server? Investigate to find out if this is true.
+8 -3
View File
@@ -15,8 +15,9 @@ turn's thinking is counted in the per-turn log line
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the (``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
``thinking`` frames (the pieces are still counted). The per-turn log ``thinking`` frames (the pieces are still counted). The per-turn log
line records ``thinking=<level>`` (the effective line records ``thinking=<level>`` (the effective
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before ``BOR_LLM_CHAT_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6). ``thinking_chars`` (phase 172, A6; phase 178, A5 — the **chat**
level; the one-shot summary level never appears on the line).
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0`` ``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
@@ -1286,7 +1287,11 @@ async def chat(
settings.relevance_threshold, settings.relevance_threshold,
plan.deflected, plan.deflected,
source_paths, source_paths,
settings.llm_thinking_level.strip() or "-", # phase 172 (A6) # phase 178 (A5): the effective chat thinking
# level, trimmed, `-` when empty — the level this
# turn ran at; the summary level is a one-shot
# concern and gains no field here.
settings.llm_chat_thinking_level.strip() or "-",
thinking_chars, thinking_chars,
holder.tool_calls, holder.tool_calls,
holder.rounds, holder.rounds,
+15 -6
View File
@@ -105,14 +105,23 @@ class Settings(BaseSettings):
#: Served by the same OpenAI-compatible endpoint — no new model #: Served by the same OpenAI-compatible endpoint — no new model
#: management. Called via ``LLMClient.chat()``. #: management. Called via ``LLMClient.chat()``.
llm_summary_model: str = "lite" llm_summary_model: str = "lite"
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) — #: The chat turn's thinking level (phase 178,
#: ``BOR_LLM_CHAT_THINKING_LEVEL``; the phase-172 knob, renamed) —
#: pi-style reasoning effort, free-form text exactly like #: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of #: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to #: truth). Default ``low``. Rides the streaming chat-completions
#: aipi as the top-level ``reasoning_effort`` parameter #: requests to aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the #: (``app/rag/llm.py`` ``chat_stream``); the EMPTY value is the off
#: parameter is not sent at all. #: switch — the parameter is not sent at all.
llm_thinking_level: str = "low" llm_chat_thinking_level: str = "low"
#: The one-shot summary thinking level (phase 178,
#: ``BOR_LLM_SUMMARY_THINKING_LEVEL``) — rides the one-shot
#: ``chat()`` calls (document summaries, folder summaries, KB
#: overview, the ``check_models`` probe) as ``reasoning_effort``.
#: Default EMPTY = the off switch (phase-172 A2): the parameter is
#: not sent at all; the TODO's "none" means absence, never a
#: literal "none" string.
llm_summary_thinking_level: str = ""
#: Operator kill-switch for the ``thinking`` SSE events (phase 17, #: Operator kill-switch for the ``thinking`` SSE events (phase 17,
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking #: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
#: pieces are still counted for the per-turn log line but never #: pieces are still counted for the per-turn log line but never
+5 -2
View File
@@ -151,8 +151,11 @@ CATALOG: dict[str, SettingMeta] = {
"llm_summary_model": SettingMeta( "llm_summary_model": SettingMeta(
"llm_summary_model", "LLM (aipi)", "LLM summary model" "llm_summary_model", "LLM (aipi)", "LLM summary model"
), ),
"llm_thinking_level": SettingMeta( "llm_chat_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level" "llm_chat_thinking_level", "LLM (aipi)", "Chat thinking level"
),
"llm_summary_thinking_level": SettingMeta(
"llm_summary_thinking_level", "LLM (aipi)", "Summary thinking level"
), ),
"stream_thinking": SettingMeta( "stream_thinking": SettingMeta(
"stream_thinking", "LLM (aipi)", "Stream thinking" "stream_thinking", "LLM (aipi)", "Stream thinking"
+30 -18
View File
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens`` ``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty). The operator's thinking level (an answer can in principle be empty). The operator's thinking levels
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every (phase 178) ride chat-completions requests as the top-level
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``) ``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
as the top-level ``reasoning_effort`` parameter (via the SDK's per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
``extra_body``); an empty value is the off switch — the parameter is default ``low``; the phase-172 knob, renamed) rides streaming
absent, the request is byte-identical to pre-phase-172, and the ``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
embeddings path never carries it. default EMPTY = off) rides ``chat`` — document summaries, folder
summaries, the KB overview, and the ``check_models`` probe (a
reachability ping, off by default, no special-casing). An empty value is
the off switch — the parameter is absent (never a literal ``"none"``),
the request is byte-identical to pre-phase-172, and the embeddings path
never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768 Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different dimensions when the table is created, so a model that returns a different
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm") logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None: def _thinking_extra_body(level: str) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body. """The thinking level as the chat-completions extra body (phase 178).
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None`` OpenAI-standard top-level parameter, phase-172 A1); empty /
(the off switch — the parameter is ABSENT from the request, A2). whitespace → ``None`` (the off switch — the parameter is ABSENT
from the request, phase-172 A2; never a literal ``"none"`` or
``"off"``).
""" """
level = settings.llm_thinking_level.strip() trimmed = level.strip()
return {"reasoning_effort": level} if level else None return {"reasoning_effort": trimmed} if trimmed else None
class EmbeddingError(RuntimeError): class EmbeddingError(RuntimeError):
@@ -429,7 +436,7 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class ``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else. :meth:`chat` retries — and :class:`LLMError` for everything else.
""" """
extra = _thinking_extra_body(self.settings) extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
extra_kwargs: dict[str, Any] = ( extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {} {"extra_body": extra} if extra is not None else {}
) )
@@ -440,8 +447,11 @@ class LLMClient:
temperature=0.2, temperature=0.2,
max_tokens=2048, max_tokens=2048,
stream=False, stream=False,
# Phase 172 (A1/A2): the thinking level rides as # Phase 178 (A3/A2): the **summary** level rides as
# reasoning_effort; the key is absent (never None) when off. # reasoning_effort (document summaries, folder summaries,
# KB overview, the ``check_models`` probe); the key is
# absent (never None) when off — the default (empty)
# keeps one-shot requests byte-identical to pre-phase-172.
**extra_kwargs, **extra_kwargs,
) )
except LLMError: except LLMError:
@@ -622,8 +632,10 @@ class LLMClient:
} }
if tools is not None: if tools is not None:
kwargs["tools"] = tools kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None: if (
# Phase 172 (A1/A2): the thinking level rides as extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
) is not None:
# Phase 178 (A3/A2): the **chat** level rides as
# reasoning_effort; the key is absent (never None) when off. # reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None stream: AsyncStream[ChatCompletionChunk] | None = None
+532
View File
@@ -0,0 +1,532 @@
"""Phase 178 E2E (Playwright): the SPLIT thinking levels — a Chat
thinking level row and a Summary thinking level row in the admin
Settings panel — the owner's acceptance for TWO independently
tunable, pi-style LLM reasoning knobs, proven in a real browser.
Source: ``TODO.md`` L3 — "Need a separate summary thinking level and
chat thinking level. Default thinking level for summary should be
none. Chat should stay at low"
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_both_rows_render_free_form_with_defaults`` — the rows:
the ``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Chat thinking level** with the
``BOR_LLM_CHAT_THINKING_LEVEL`` env-var display shows ``low``
with the ``env`` badge (chat stays at low, TODO L3), a row
labelled **Summary thinking level** with the
``BOR_LLM_SUMMARY_THINKING_LEVEL`` display shows an **empty**
input with the ``env`` badge (the TODO's "none" = the phase-172
off switch, LOCKED A2 — the parameter is ABSENT, never a
literal "none"), BOTH inputs are ``type="text"`` (free-form —
NOT ``number``/``select``, the A4 contract: no allowlist, the
endpoint is the source of truth), the ``llm_chat_model`` row's
input is the same control (the "like chat model" reference, one
assertion pair), and ``tr[data-key="llm_thinking_level"]`` is
**absent** (the old phase-172 key is gone, LOCKED A1).
2. ``test_saved_levels_persist_without_restart`` — the save (the
phase-168 §7.4 Save lifecycle per row: dirty → save → ``DB``
badge): ``think-soft`` (a free-form custom — NOT a known pi
level, A4) is typed into the summary row and ``high`` into the
chat row (the dirty hints appear), then BOTH are saved; the
API catalogue (``_catalog_item``) AND the raw ``app_settings``
rows carry the saved values for EACH key independently (the
independence pin — the two rows' controls are distinct and the
saved values do not cross), and a reload of ``/settings.html``
still shows ``think-soft`` / ``high`` with the ``DB`` badges
(no restart, the phase-168 contract).
3. ``test_reset_returns_to_env_defaults`` — the per-row Reset (the
per-row ``DELETE``): after the saves, both Resets are live;
clicking the summary Reset lands the value at **empty** with
the ``env`` badge and clicking the chat Reset lands ``low``
with the ``env`` badge (the overlay reset, task 02's contract,
in the browser); the status lines show the house "… is back to
the environment value." toast for each.
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens), and — because the
direct SQL truncate does NOT invalidate the app's settings cache
(the admin API is the only mutation surface that does) — follows
it with the app's own clear-all (``DELETE /api/admin/settings``;
without it, a test that types the SAME value its predecessor
saved sees the stale value as the served one, registers no dirty
state, and the Save button stays disabled). The module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_CHAT_THINKING_LEVEL=low`` AND
``BOR_LLM_SUMMARY_THINKING_LEVEL=`` (the empty string SET — so
both badges read ``env``, not ``default``, and an operator's
local ``.env`` cannot leak a different level into the suite;
the house leak-guard pattern) together with the "like chat
model" reference (``BOR_LLM_CHAT_MODEL``). A module-scope ``.env``
mtime guard pins LOCKED A1: no code path the suite drives opens
the env file for writing.
ASSUMPTION A5 (phase-locked): the per-turn-line assertion
(``thinking=<chat level>``) and the wire split (``chat_stream()``
→ chat level, ``chat()`` → summary level) live in the
unit/integration layer (tasks 03/04) — this suite proves the
settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The chat row's saved value (test 2) — a known pi level, the
#: operator's "bump the chat turn" case.
SAVED_CHAT_LEVEL = "high"
#: The summary row's saved value (test 2) — NOT a known pi level
#: (proves the A4 contract: no allowlist, the endpoint is the
#: source of truth).
SAVED_SUMMARY_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-178
pins, with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the rows' save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 178 (task 05): pin BOTH thinking levels to their CODE
# DEFAULTS — each env var is SET (so the badge reads ``env``,
# not ``default``) and an operator's local .env cannot leak a
# different level into the suite's app. The summary level is
# set to EMPTY (the TODO's "none" = the off switch, A2).
env["BOR_LLM_CHAT_THINKING_LEVEL"] = Settings.model_fields[
"llm_chat_thinking_level"
].default
env["BOR_LLM_SUMMARY_THINKING_LEVEL"] = Settings.model_fields[
"llm_summary_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: BOTH thinking
levels pinned to their code defaults (chat ``low``, summary
empty = the A2 off switch) and ``BOR_SETTINGS_DB=1`` (the
module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the rows are wiped
between tests) and the single-row ``ui_settings`` (the
middleware reads it for EVERY page), and delete this suite's
issued tokens (label-scoped on ``e2e-`` — never a TRUNCATE: the
shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
def _invalidate_app_cache(app_url: str) -> None:
"""The direct SQL truncate does NOT invalidate the app's
``get_settings()`` lru-cache (the admin API is the only
mutation surface that invalidates — the phase-168 contract)
— so after the truncate, hit the app's own clear-all
(``DELETE /api/admin/settings``): zero rows to delete, but the
invalidate call rebuilds the cache env-only. Without it, a test
that types the SAME value its predecessor saved sees the STALE
value as the served one (``markDirty`` finds no diff, the Save
button stays disabled)."""
client = _admin_client(app_url)
r = client.delete(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, f"clear-all → {r.status_code}: {r.text[:200]}"
@pytest.fixture(autouse=True)
def _clean(app_server: str, db_ready: None) -> Iterator[None]:
_clean_settings_state()
_invalidate_app_cache(app_server)
yield
_clean_settings_state()
_invalidate_app_cache(app_server)
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
def _raw_value(key: str) -> str | None:
"""The raw ``app_settings`` row for ``key`` (the overlay's D1
env-native string form), or None when no override is stored."""
with SessionLocal() as db:
return db.execute(
text("SELECT value FROM app_settings WHERE key = :k"), {"k": key}
).scalar_one_or_none()
def _raw_count(key: str) -> int:
with SessionLocal() as db:
return db.execute(
text("SELECT count(*) FROM app_settings WHERE key = :k"), {"k": key}
).scalar_one()
# ---------------------------------------------------------------------------
# 1. Both rows render free-form with defaults (chat ``low``,
# summary empty = the A2 off switch; the old key is gone)
# ---------------------------------------------------------------------------
def test_both_rows_render_free_form_with_defaults(
page: Page, app_url: str, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
client = _admin_client(app_url)
# The LLM (aipi) group band is visible (the phase-171 band
# contract — both rows sit in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# The CHAT row: the label, the env-var display (the catalogue
# pattern), the free-form text control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth)…
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
expect(chat_row).to_be_visible()
expect(chat_row.locator(".cell-clip")).to_have_text("Chat thinking level")
expect(chat_row.locator(".settings-env")).to_have_text("BOR_LLM_CHAT_THINKING_LEVEL")
assert chat_row.locator(".settings-control").get_attribute("type") == "text"
# …chat stays at low (TODO L3): the env-pinned ``low`` with the
# env badge (the env var is set — no app_settings row, no
# code-default badge).
expect(chat_row.locator(".settings-control")).to_have_value("low")
expect(chat_row.locator(".settings-src")).to_have_text("env")
expect(chat_row.locator(".settings-reset")).to_be_disabled()
# The SUMMARY row: the label, the env-var display, the free-form
# text control…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
expect(summary_row).to_be_visible()
expect(summary_row.locator(".cell-clip")).to_have_text("Summary thinking level")
expect(summary_row.locator(".settings-env")).to_have_text(
"BOR_LLM_SUMMARY_THINKING_LEVEL"
)
assert summary_row.locator(".settings-control").get_attribute("type") == "text"
# …and the TODO's "none" = the off switch (A2): an EMPTY input
# with the env badge (the env var is SET to the empty string —
# the badge reads env, not default).
expect(summary_row.locator(".settings-control")).to_have_value("")
expect(summary_row.locator(".settings-src")).to_have_text("env")
expect(summary_row.locator(".settings-reset")).to_be_disabled()
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair
# covering both thinking rows).
chat_model_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert chat_model_type == "text"
assert (
chat_row.locator(".settings-control").get_attribute("type")
== summary_row.locator(".settings-control").get_attribute("type")
== chat_model_type
)
# The old phase-172 key is GONE (LOCKED A1 — the rename, not
# add-on): no row for llm_thinking_level anywhere in the panel.
assert page.locator('tr[data-key="llm_thinking_level"]').count() == 0
# The server agrees with the panel: both defaults, both sourced
# from the env (the module pins), stored raw on no row.
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "env" and chat_item["value"] == "low"
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "env" and summary_item["value"] == ""
assert _raw_count("llm_chat_thinking_level") == 0
assert _raw_count("llm_summary_thinking_level") == 0
# ---------------------------------------------------------------------------
# 2. The saves: dirty → save → DB badge, EACH key's value persists
# independently and survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_levels_persist_without_restart(
page: Page, app_url: str, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type the summary's free-form custom
# value (A4 — NOT a known pi level: no allowlist, the endpoint
# is the source of truth) and the chat's bump — two dirty
# rows, the hints appear…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
summary_row.locator(".settings-control").fill(SAVED_SUMMARY_LEVEL)
chat_row.locator(".settings-control").fill(SAVED_CHAT_LEVEL)
expect(page.locator("#settings-dirty")).to_have_text("2 unsaved changes")
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then BOTH badges flip to DB (the
# re-GET settled the rows).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 2 settings.", timeout=30_000
)
expect(summary_row.locator(".settings-src")).to_have_text("DB")
expect(chat_row.locator(".settings-src")).to_have_text("DB")
# The effective values are the saved ones for EACH key
# independently (the independence pin — the two rows' controls
# are distinct and the saved values do not cross): the API
# catalogue (what the LLM layer reads per request via
# get_settings)…
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "db" and summary_item["value"] == SAVED_SUMMARY_LEVEL
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "db" and chat_item["value"] == SAVED_CHAT_LEVEL
# …and the raw app_settings rows are stored RAW (the overlay's
# D1 env-native string form), one per key.
assert _raw_value("llm_summary_thinking_level") == SAVED_SUMMARY_LEVEL
assert _raw_value("llm_chat_thinking_level") == SAVED_CHAT_LEVEL
# No restart: the DB values survive a reload of /settings.html
# (the panel re-reads the settled server state), the summary
# row's custom value and the chat row's bump each still on
# their OWN row with the DB badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_summary_thinking_level"] .settings-control')
).to_have_value(SAVED_SUMMARY_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_summary_thinking_level"] .settings-src')
).to_have_text("DB")
expect(
page.locator('tr[data-key="llm_chat_thinking_level"] .settings-control')
).to_have_value(SAVED_CHAT_LEVEL)
expect(
page.locator('tr[data-key="llm_chat_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Resets: each row goes back to ITS environment default (the
# per-row DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_defaults(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish saved overrides first (the Resets are disabled with
# nothing to reset — the phase-168 render contract): save a
# value on each row so both Resets go live…
summary_row = page.locator('tr[data-key="llm_summary_thinking_level"]')
chat_row = page.locator('tr[data-key="llm_chat_thinking_level"]')
summary_row.locator(".settings-control").fill(SAVED_SUMMARY_LEVEL)
chat_row.locator(".settings-control").fill(SAVED_CHAT_LEVEL)
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 2 settings.", timeout=30_000
)
# The re-GET settles the rows (DB badges) before the Resets go
# live (the phase-168 render contract — the button tracks the
# SETTLED source, not the click).
expect(summary_row.locator(".settings-src")).to_have_text("DB")
expect(chat_row.locator(".settings-src")).to_have_text("DB")
expect(summary_row.locator(".settings-reset")).to_be_enabled()
expect(chat_row.locator(".settings-reset")).to_be_enabled()
# The summary Reset (the per-row DELETE): the status line…
summary_row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_SUMMARY_THINKING_LEVEL is back to the environment value.",
timeout=30_000,
)
# …the value lands at EMPTY (the env var is set to the empty
# string — the env badge, not the code-default badge) and the
# row's Reset goes live-free again.
expect(summary_row.locator(".settings-control")).to_have_value("")
expect(summary_row.locator(".settings-src")).to_have_text("env")
expect(summary_row.locator(".settings-reset")).to_be_disabled()
# The chat Reset (the per-row DELETE): the status line…
chat_row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_CHAT_THINKING_LEVEL is back to the environment value.",
timeout=30_000,
)
# …the value reads ``low`` again (chat stays at low, TODO L3)
# with the env badge, and the row's Reset is disabled.
expect(chat_row.locator(".settings-control")).to_have_value("low")
expect(chat_row.locator(".settings-src")).to_have_text("env")
expect(chat_row.locator(".settings-reset")).to_be_disabled()
# The server agrees: both overrides are gone, the effective
# values are the env defaults, and no raw row survives.
summary_item = _catalog_item(client, app_url, "llm_summary_thinking_level")
assert summary_item["source"] == "env" and summary_item["value"] == ""
chat_item = _catalog_item(client, app_url, "llm_chat_thinking_level")
assert chat_item["source"] == "env" and chat_item["value"] == "low"
assert _raw_count("llm_summary_thinking_level") == 0
assert _raw_count("llm_chat_thinking_level") == 0
-405
View File
@@ -1,405 +0,0 @@
"""Phase 172 E2E (Playwright): the Thinking level row in the admin Settings
panel — the owner's acceptance for a pi-style, operator-tunable LLM
reasoning knob, proven in a real browser.
Source: ``TODO.md`` L3 — "Brain of Reese needs to support thinking levels
like pi does. should be a free-form text entry like chat model. Default
should be 'low'. Update .env and admin settings."
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_level.py -v --no-cov
Test → contract mapping (one story, one phase, one isolated file):
1. ``test_row_renders_free_form_with_default_low`` — the row: the
``LLM (aipi)`` group band is visible (the phase-171 band
contract), a row labelled **Thinking level** with the
``BOR_LLM_THINKING_LEVEL`` env-var display (the catalogue
pattern) is present, its input is a **free-form text** input
(``type="text"`` — NOT ``number``/``select``, the A4
contract), it shows ``low`` with the ``env`` badge, and the
``llm_chat_model`` row's input is the same ``type="text"``
control (the "like chat model" reference, in one assertion
pair).
2. ``test_saved_level_persists_without_restart`` — the save
(the phase-168 §7.4 Save lifecycle: dirty → save → ``DB``
badge): ``high`` is typed (the dirty hint appears), then the
free-form custom value ``think-soft`` is typed once (no
allowlist — A4) and saved; the effective value reads
``think-soft`` (the API catalogue AND the raw
``app_settings`` row), and a reload of ``/settings.html``
still shows ``think-soft`` with the ``DB`` badge (no
restart, the phase-168 contract).
3. ``test_reset_returns_to_env_default`` — the per-row Reset
(the per-row ``DELETE``): after a save, the Reset is live;
clicking it lands the status line and the value reads
``low`` again with the ``env`` badge (the overlay reset,
task 02's contract, in the browser).
DB isolation: the shared e2e Postgres keeps ``app_settings``
(the overlay rows — a leftover row would retune every later
suite's app through the conftest pin's failure mode),
``ui_settings`` (the single row the caching middleware reads
for EVERY served page) and ``api_tokens`` rows across suites.
An autouse fixture truncates ``app_settings`` and
``ui_settings`` and deletes the ``e2e-``-labeled tokens before
AND after every test (never a TRUNCATE on ``api_tokens`` — the
shared DB may hold the owner's real tokens); the module teardown
pins ``app_settings`` empty. The module app opts BACK IN over
the conftest's global ``BOR_SETTINGS_DB=0`` pin (``env["BOR_
SETTINGS_DB"] = "1"`` — the ``test_pictorial_pdf.py`` opt-in
pattern) and pins ``BOR_LLM_THINKING_LEVEL`` (and the
"like chat model" reference, ``BOR_LLM_CHAT_MODEL``) to their
CODE DEFAULTS — the env var is set (so the badge reads ``env``,
not ``default``) and an operator's local ``.env`` cannot leak a
different level into the suite's app (the house leak-guard
pattern). A module-scope ``.env`` mtime guard pins LOCKED A1:
no code path the suite drives opens the env file for writing.
ASSUMPTION A6 (phase-locked): the per-turn-line assertion
(``thinking=<value>``) lives in the unit/integration layer
(task 04) — the mock LLM turn does not exercise the wire, and
this suite proves the settings surface only.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
APP_PORT,
REPO,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The save's custom free-form value (test 2) — NOT a known pi
#: level (proves the A4 contract: no allowlist, the endpoint is
#: the source of truth).
SAVED_LEVEL = "think-soft"
# ---------------------------------------------------------------------------
# Per-module app env (the phase-168 tuning pattern)
# ---------------------------------------------------------------------------
def _boot_app() -> subprocess.Popen[bytes]:
"""The one module app: the shared env block + the phase-172 pins,
with the overlay opted BACK IN (``BOR_SETTINGS_DB=1``)."""
global _MOCK_PORT
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{_MOCK_PORT}/v1"
)
# The E2E-calibrated gate pins (the conftest pattern — the mock's
# token-overlap distribution, explicit so an operator's local
# .env cannot leak a different gate into the suite's app).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
# Leak guards: the branding base and the "like chat model"
# reference row stay the CODE defaults (derived from the class
# fields, never drift from app/config.py).
env["BOR_APP_NAME"] = Settings.model_fields["app_name"].default
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
env["BOR_GIT_SOURCES"] = ""
env["BOR_LLM_CHAT_MODEL"] = Settings.model_fields["llm_chat_model"].default
# Phase 168 (the panel suite's pattern): the module opts BACK IN
# over the conftest's global BOR_SETTINGS_DB=0 pin (the
# test_pictorial_pdf.py pattern) — the row's save/reset
# contract needs the overlay on.
env["BOR_SETTINGS_DB"] = "1"
# Phase 172 (task 05): pin the thinking level to the CODE
# DEFAULT — the env var is SET (so the badge reads ``env``, not
# ``default``) and an operator's local .env cannot leak a
# different level into the suite's app.
env["BOR_LLM_THINKING_LEVEL"] = Settings.model_fields[
"llm_thinking_level"
].default
return subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
def _stop_app(proc: subprocess.Popen[bytes]) -> None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(mock_llm: int) -> Iterator[str]:
"""The real app under test — per-module env: the thinking level
pinned to the code default ``low`` and ``BOR_SETTINGS_DB=1``
(the module opts back in over the conftest pin)."""
global _MOCK_PORT
_MOCK_PORT = mock_llm
proc = _boot_app()
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
_stop_app(proc)
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# DB isolation + helpers
# ---------------------------------------------------------------------------
def _clean_settings_state() -> None:
"""Fresh settings + theme + token state per test: truncate
``app_settings`` (a leftover row is the cross-suite poison this
phase's overlay contract exists to prevent — the new key's rows
are wiped between tests) and the single-row ``ui_settings``
(the middleware reads it for EVERY page), and delete this
suite's issued tokens (label-scoped on ``e2e-`` — never a
TRUNCATE: the shared DB may hold the owner's real tokens)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE app_settings"))
db.execute(text("TRUNCATE ui_settings"))
db.execute(text("DELETE FROM api_tokens WHERE label LIKE 'e2e-%'"))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
_clean_settings_state()
yield
_clean_settings_state()
# The module's final state: app_settings EMPTY (the cross-suite
# poison guard — the next suite's app must boot with no
# overrides).
with SessionLocal() as db:
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
@pytest.fixture(scope="module", autouse=True)
def _env_never_written() -> Iterator[None]:
"""LOCKED A1, mtime-pinned: no code path the suite drives opens
``.env`` for writing (the file is gitignored — a ``git status``
check cannot see it, so ``stat``'s mtime is the assertion)."""
env_file = REPO / ".env"
before = env_file.stat().st_mtime_ns if env_file.exists() else None
yield
after = env_file.stat().st_mtime_ns if env_file.exists() else None
assert before == after, "the app under test must never write .env"
def _cookies(page: Page) -> dict[str, str]:
"""The session cookies the browser context holds (the test's API
side sees exactly what that browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _admin_client(app_url: str) -> httpx.Client:
"""The API side of the form login (the 204 + the signed session
cookie in the jar)."""
client = httpx.Client(timeout=60.0)
r = client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login → {r.status_code}: {r.text[:120]}"
return client
def _catalog_item(client: httpx.Client, app_url: str, key: str) -> dict[str, Any]:
r = client.get(f"{app_url}/api/admin/settings", timeout=30.0)
assert r.status_code == 200, r.text[:200]
for item in r.json()["items"]:
if item["key"] == key:
return item
raise AssertionError(f"{key} missing from the catalogue")
# ---------------------------------------------------------------------------
# 1. The row renders free-form with default low (the "like chat
# model" reference, in one assertion pair)
# ---------------------------------------------------------------------------
def test_row_renders_free_form_with_default_low(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(page.locator("#settings-disabled")).to_be_hidden() # the overlay is on
# The LLM (aipi) group band is visible (the phase-171 band
# contract — the row sits in it)…
expect(page.locator("tr.settings-group-row", has_text="LLM (aipi)")).to_be_visible()
# …and the row is present with its label…
row = page.locator('tr[data-key="llm_thinking_level"]')
expect(row).to_be_visible()
expect(row.locator(".cell-clip")).to_have_text("Thinking level")
# …and the env-var display (the catalogue pattern).
expect(row.locator(".settings-env")).to_have_text("BOR_LLM_THINKING_LEVEL")
# The input is a FREE-FORM TEXT control (A4: type="text" — NOT
# number/select; no allowlist, the endpoint is the source of
# truth).
assert row.locator(".settings-control").get_attribute("type") == "text"
# The "like chat model" reference: the llm_chat_model row's
# input is the SAME type="text" control (one assertion pair).
thinking_type = row.locator(".settings-control").get_attribute("type")
chat_type = page.locator(
'tr[data-key="llm_chat_model"] .settings-control'
).get_attribute("type")
assert thinking_type == chat_type == "text"
# The default: the env-pinned ``low`` with the env badge (the
# env var is set — no app_settings row, no code-default badge).
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# Nothing is saved yet: the row's Reset is disabled (no saved
# override — the phase-168 render contract).
expect(row.locator(".settings-reset")).to_be_disabled()
# ---------------------------------------------------------------------------
# 2. The save: dirty → save → DB badge, the effective value is the
# free-form custom, and it survives a reload WITHOUT a restart
# ---------------------------------------------------------------------------
def test_saved_level_persists_without_restart(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# The §7.4 Save lifecycle: type ``high`` — the dirty hint
# appears (one unsaved change on the row)…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
expect(page.locator("#settings-dirty")).to_have_text("1 unsaved changes")
# …then type the free-form custom value once (A4 — NOT a known
# pi level: no allowlist, the endpoint is the source of truth).
row.locator(".settings-control").fill(SAVED_LEVEL)
expect(page.locator("#settings-save")).to_be_enabled()
# Save: the status line, then the badge flips to DB (the
# re-GET settled the row).
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
# The effective value reads ``think-soft`` — the API catalogue
# (what the LLM layer reads per request via get_settings)…
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "db" and item["value"] == SAVED_LEVEL
# …and the raw app_settings row is stored RAW (the overlay's
# D1 env-native string form).
with SessionLocal() as db:
raw = db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar_one()
assert raw == SAVED_LEVEL, f"the row must be stored raw (got {raw!r})"
# No restart: the DB value survives a reload of /settings.html
# (the panel re-reads the settled server state) with the DB
# badge still on.
page.goto(app_url + "/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-control')
).to_have_value(SAVED_LEVEL, timeout=15_000)
expect(
page.locator('tr[data-key="llm_thinking_level"] .settings-src')
).to_have_text("DB")
# ---------------------------------------------------------------------------
# 3. The Reset: control goes back to the environment (the per-row
# DELETE — task 02's contract, in the browser)
# ---------------------------------------------------------------------------
def test_reset_returns_to_env_default(page: Page, app_url: str, db_ready: None) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/settings.html")
expect(page.locator("#settings-content")).to_be_visible(timeout=15_000)
client = _admin_client(app_url)
# Establish a saved override first (the Reset is disabled with
# nothing to reset — the phase-168 render contract): save
# ``high`` so the row's Reset goes live…
row = page.locator('tr[data-key="llm_thinking_level"]')
row.locator(".settings-control").fill("high")
page.click("#settings-save")
expect(page.locator("#settings-status")).to_contain_text(
"Saved 1 settings.", timeout=30_000
)
expect(row.locator(".settings-src")).to_have_text("DB")
expect(row.locator(".settings-reset")).to_be_enabled()
# The per-row Reset (the per-row DELETE): the status line…
row.locator(".settings-reset").click()
expect(page.locator("#settings-status")).to_contain_text(
"BOR_LLM_THINKING_LEVEL is back to the environment value.", timeout=30_000
)
# …the value reads ``low`` again with the env badge (the env
# var is set — not the code-default badge)…
expect(row.locator(".settings-control")).to_have_value("low")
expect(row.locator(".settings-src")).to_have_text("env")
# …and the server agrees: the override is gone and the
# effective value is the env value again.
item = _catalog_item(client, app_url, "llm_thinking_level")
assert item["source"] == "env" and item["value"] == "low"
with SessionLocal() as db:
assert (
db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
).scalar()
== 0
), "the DELETE must remove the row"
+122 -18
View File
@@ -518,11 +518,13 @@ def test_catalog_covers_every_settings_field(admin_client) -> None:
assert set(CATALOG) == set(Settings.model_fields) assert set(CATALOG) == set(Settings.model_fields)
def test_llm_thinking_level_row_contract( def test_llm_chat_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None: ) -> None:
"""Phase 172 (task 02): the new ``llm_thinking_level`` row behaves """Phase 178 (task 01; task 02 extends for the summary row): the
exactly like ``llm_chat_model`` — the machinery is generic over renamed ``llm_chat_thinking_level`` row (the phase-172 knob, the
rename per locked A1) behaves exactly like ``llm_chat_model`` —
the machinery is generic over
the catalogue (D4 / A5), so the per-key contract is: (a) present in the catalogue (D4 / A5), so the per-key contract is: (a) present in
the GET body (group ``LLM (aipi)``, ``str``, effective ``low`` the GET body (group ``LLM (aipi)``, ``str``, effective ``low``
with no row in ``app_settings``, ``env_var`` named); (b) PUT stores with no row in ``app_settings``, ``env_var`` named); (b) PUT stores
@@ -538,20 +540,20 @@ def test_llm_thinking_level_row_contract(
""" """
# A deterministic base (machine-independent, the kill-switch test's # A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table. # idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False) monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings() invalidate_settings()
assert get_settings().llm_thinking_level == "low" assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, type, env var, effective ``low`` with NO row # (a) the row: group, type, env var, effective ``low`` with NO row
# in app_settings, not secret, not restart-required (the LLM layer # in app_settings, not secret, not restart-required (the LLM layer
# reads the setting per request — live). # reads the setting per request — live).
r = admin_client.get(API) r = admin_client.get(API)
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
item = _item(r.json(), "llm_thinking_level") item = _item(r.json(), "llm_chat_thinking_level")
assert item["group"] == "LLM (aipi)" assert item["group"] == "LLM (aipi)"
assert item["label"] == "Thinking level" assert item["label"] == "Chat thinking level"
assert item["type"] == "str" assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_THINKING_LEVEL" assert item["env_var"] == "BOR_LLM_CHAT_THINKING_LEVEL"
assert item["secret"] is False assert item["secret"] is False
assert item["restart_required"] is False assert item["restart_required"] is False
assert item["value"] == "low" assert item["value"] == "low"
@@ -562,48 +564,150 @@ def test_llm_thinking_level_row_contract(
# (b) PUT stores the RAW string (the overlay's D1 — env-native # (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value. # string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"}) r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
assert r.json()["value"] == "high" assert r.json()["value"] == "high"
assert r.json()["source"] == "db" assert r.json()["source"] == "db"
assert r.json()["has_override"] is True assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high" assert r.json()["db_value"] == "high"
assert db.execute( assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'") text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent ).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_thinking_level == "high" # live, no restart assert get_settings().llm_chat_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable (the off switch, A2/A4 — the # (c) the EMPTY value is storable (the off switch, A2/A4 — the
# API stores as-is; no allowlist, no value validation): the row # API stores as-is; no allowlist, no value validation): the row
# holds the empty string and the effective value is ``""``. # holds the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""}) r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
assert r.json()["value"] == "" assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty assert r.json()["has_value"] is False # the effective value is empty
assert db.execute( assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'") text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "" # stored as-is (the off switch) ).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_thinking_level == "" assert get_settings().llm_chat_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to # (d) DELETE: the row is gone and the effective value is back to
# the env value (``low`` — the base pinned above). # the env value (``low`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_thinking_level") r = admin_client.delete(f"{API}/llm_chat_thinking_level")
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
assert r.json()["value"] == "low" assert r.json()["value"] == "low"
assert r.json()["has_override"] is False assert r.json()["has_override"] is False
assert db.execute( assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'") text("SELECT count(*) FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == 0 # the row is gone ).scalar() == 0 # the row is gone
assert get_settings().llm_thinking_level == "low" assert get_settings().llm_chat_thinking_level == "low"
# (e) the non-admin 403 gate still applies (asserted once for this # (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in # key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``). # ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app) anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"}) r = anon.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"} assert r.status_code == 403 and r.json() == {"detail": "admin only"}
def test_llm_summary_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the ``llm_summary_thinking_level`` row (the
new one-shot knob) behaves like the chat row — the machinery is
generic over the catalogue (D4 / A4) — with the summary specifics:
(a) present in the GET body (group ``LLM (aipi)``, ``str``, label
``"Summary thinking level"``, ``env_var``
``BOR_LLM_SUMMARY_THINKING_LEVEL``, effective ``""`` with no row in
``app_settings``); (b) PUT stores the RAW string in
``app_settings`` and the value resolves live WITHOUT a restart;
(c) the EMPTY value is storable (the summary off switch, A2 — the
API stores as-is, no allowlist, an empty stored value is never
rejected as "no value"); (d) DELETE resets to the env value
(``""`` — the code default); (e) the non-admin 403 gate applies;
(f) INDEPENDENCE — the two overlays resolve independently (the
whole point of the split): a saved summary value never changes the
chat row's effective value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): BOTH knobs at their code defaults, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, label, type, env var, effective ``""`` with NO
# row in app_settings, not secret, not restart-required (the LLM
# layer reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_summary_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Summary thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_SUMMARY_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == ""
assert item["source"] == "default" # the env var is unset, no row
assert item["has_override"] is False
assert item["db_value"] is None
assert item["default"] == ""
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_summary_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable — the summary off switch (A2; the
# API stores as-is; no allowlist, no value validation): an empty
# stored value must not be rejected as "no value"; the row holds
# the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_summary_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``""`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_summary_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_summary_thinking_level == ""
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
# (f) INDEPENDENCE — the whole point of the split: save the summary
# row "high" again; the chat row's effective value is STILL "low"
# (the two overlays resolve independently — a saved summary value
# never touches the chat level).
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert get_settings().llm_summary_thinking_level == "high"
assert get_settings().llm_chat_thinking_level == "low" # untouched
chat_item = _item(admin_client.get(API).json(), "llm_chat_thinking_level")
assert chat_item["value"] == "low" # the panel agrees, too
admin_client.delete(f"{API}/llm_summary_thinking_level") # no row survives
def test_docs_token_is_write_only_masked_end_to_end( def test_docs_token_is_write_only_masked_end_to_end(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None: ) -> None:
+61 -16
View File
@@ -10,9 +10,10 @@ snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and (``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
the ``scripts/``-consumer surface (``import_extension_set``), and the the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_thinking_level`` — the value the LLM phase-172 thinking level (``llm_chat_thinking_level`` — the value the
layer reads per request via ``get_settings()``; the off switch is LLM layer reads per request via ``get_settings()``; the off switch is
live-settable, A2). live-settable, A2) and its phase-178 summary twin (task 02:
``llm_summary_thinking_level``, default ``""`` = off, A2).
The rows are inserted directly through the ``db`` session (the admin The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` — API is task 04); the save itself is ``invalidate_settings()`` —
@@ -164,44 +165,88 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# the thinking level: save/reset resolve the live value (phase 172, task 02) # the chat thinking level: save/reset resolve the live value
# (phase 178, task 01 — the phase-172 test, renamed)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def test_llm_thinking_level_live_apply_no_restart( def test_llm_chat_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch db: Session, monkeypatch: pytest.MonkeyPatch
) -> None: ) -> None:
"""Phase 172 (task 02): the thinking level follows the overlay """Phase 178 (task 01): the chat thinking level follows the overlay
live — a save (the PUT's write, minus the API: ``_save`` + live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE ``invalidate_settings()`` is exactly what task 04's PUT/DELETE
call after the write) resolves call after the write) resolves
``get_settings().llm_thinking_level`` WITHOUT a restart (the proof ``get_settings().llm_chat_thinking_level`` WITHOUT a restart (the
the LLM layer — task 03 reads the setting per request via proof the LLM layer — task 03 reads the setting per request via
``get_settings()`` — picks the override up live), the EMPTY value ``get_settings()`` — picks the override up live), the EMPTY value
resolves live too (the off switch is live-settable, A2/A4), and a resolves live too (the off switch is live-settable, A2/A4), and a
reset (the DELETE) returns the env value. reset (the DELETE) returns the env value.
""" """
# A deterministic base (machine-independent, the kill-switch test's # A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table. # idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False) monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings() invalidate_settings()
assert get_settings().llm_thinking_level == "low" assert get_settings().llm_chat_thinking_level == "low"
_save(db, "llm_thinking_level", "high") _save(db, "llm_chat_thinking_level", "high")
invalidate_settings() invalidate_settings()
assert get_settings().llm_thinking_level == "high" # NO restart assert get_settings().llm_chat_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value # The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced. # resolves through the overlay, not stripped or coerced.
_save(db, "llm_thinking_level", "") _save(db, "llm_chat_thinking_level", "")
invalidate_settings() invalidate_settings()
assert get_settings().llm_thinking_level == "" assert get_settings().llm_chat_thinking_level == ""
# The reset returns the env value (no process restart in between — # The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03). # the LLM layer re-reads per request, task 03).
db.execute( db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'") text("DELETE FROM app_settings WHERE key = 'llm_chat_thinking_level'")
) )
db.commit() db.commit()
invalidate_settings() invalidate_settings()
assert get_settings().llm_thinking_level == "low" assert get_settings().llm_chat_thinking_level == "low"
# ---------------------------------------------------------------------------
# the summary thinking level: save/reset resolve the live value
# (phase 178, task 02 — the chat twin's summary counterpart)
# ---------------------------------------------------------------------------
def test_llm_summary_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the summary thinking level follows the
overlay live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE call
after the write) resolves ``get_settings().llm_summary_thinking_level``
WITHOUT a restart (the proof the LLM layer — task 03 reads the
setting per request via ``get_settings()`` — picks the override up
live), the EMPTY value resolves live too (the off switch is
live-settable, A2/A4), and a reset (the DELETE) returns the env
value (``""`` — the code default; the chat twin's reset returns
``"low"``, the two levels are independent).
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
_save(db, "llm_summary_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_summary_thinking_level", "")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_summary_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+45 -5
View File
@@ -1550,8 +1550,13 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
seeded suggestion tier's size (``len(plan.suggested_docs)``). seeded suggestion tier's size (``len(plan.suggested_docs)``).
Phase 172 (A6): the line gains ``thinking=<level>`` immediately Phase 172 (A6): the line gains ``thinking=<level>`` immediately
BEFORE ``thinking_chars=N`` — the effective (overlay-resolved) BEFORE ``thinking_chars=N`` — the effective (overlay-resolved)
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off ``BOR_LLM_CHAT_THINKING_LEVEL`` (the chat level), trimmed,
switch).""" ``-`` when empty (the off switch).
Phase 178 (A5): the field stays pinned to the CHAT level — the
level the turn actually ran at; the one-shot summary level
(``llm_summary_thinking_level``, a different knob) is never read
by the line and must not leak into the line's ``thinking=``
value (cross-pin + inverse off-pin below)."""
_session, _llm = gate_env _session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT") doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)] chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)]
@@ -1606,7 +1611,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr( monkeypatch.setattr(
chat_api, chat_api,
"get_settings", "get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue] lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level="high"), # pyright: ignore[reportCallIssue]
) )
line = _last_log_line() line = _last_log_line()
assert "thinking=high thinking_chars=" in line assert "thinking=high thinking_chars=" in line
@@ -1615,7 +1620,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr( monkeypatch.setattr(
chat_api, chat_api,
"get_settings", "get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue] lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level=""), # pyright: ignore[reportCallIssue]
) )
line = _last_log_line() line = _last_log_line()
assert "thinking=- thinking_chars=" in line assert "thinking=- thinking_chars=" in line
@@ -1625,7 +1630,42 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
monkeypatch.setattr( monkeypatch.setattr(
chat_api, chat_api,
"get_settings", "get_settings",
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue] lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_chat_thinking_level=" "), # pyright: ignore[reportCallIssue]
) )
line = _last_log_line() line = _last_log_line()
assert "thinking=- thinking_chars=" in line assert "thinking=- thinking_chars=" in line
# Phase 178 (A5) cross-pin: the line is pinned to the CHAT level
# — with the one-shot summary level set to a DIFFERENT value,
# the line still shows the chat level; the summary's "high"
# does not appear as the line's ``thinking=`` value.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
relevance_threshold=0.30,
llm_chat_thinking_level="low",
llm_summary_thinking_level="high",
),
)
line = _last_log_line()
assert "thinking=low thinking_chars=" in line
assert "thinking=high" not in line
# The inverse off-pin: the chat off-switch logs ``-`` even
# though the one-shot summary knob is set — only the chat level
# (the level this turn ran at) rides the line.
monkeypatch.setattr(
chat_api,
"get_settings",
lambda: Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
relevance_threshold=0.30,
llm_chat_thinking_level="",
llm_summary_thinking_level="high",
),
)
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
assert "thinking=high" not in line
+56 -23
View File
@@ -105,32 +105,65 @@ def test_llm_summary_model_env_override(monkeypatch) -> None:
assert s.llm_summary_model == "mini" assert s.llm_summary_model == "mini"
def test_llm_thinking_level_default_is_low(monkeypatch: pytest.MonkeyPatch) -> None: def test_llm_chat_thinking_level_default_is_low(
"""Phase 172: the model's thinking level defaults to ``low``
(``BOR_LLM_THINKING_LEVEL`` unset)."""
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_thinking_level == "low"
def test_llm_thinking_level_env_override(monkeypatch) -> None:
"""Phase 172 (A4): free-form like ``llm_chat_model`` —
``BOR_LLM_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_thinking_level == "high"
def test_llm_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
"""Phase 172 (A2): the EMPTY value is the off switch — it must not """Phase 178 (the phase-172 knob, renamed): the chat turn's
be coerced back to the default (trimming happens only at the wire, thinking level defaults to ``low`` (``BOR_LLM_CHAT_THINKING_LEVEL``
task 03; no normalization at load).""" unset)."""
monkeypatch.setenv("BOR_LLM_THINKING_LEVEL", "") monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
s = _settings() s = _settings()
assert s.llm_thinking_level == "" assert s.llm_chat_thinking_level == "low"
def test_llm_chat_thinking_level_env_override(monkeypatch) -> None:
"""Phase 178 (A4, kept): free-form like ``llm_chat_model`` —
``BOR_LLM_CHAT_THINKING_LEVEL=high`` resolves to ``"high"`` (no
allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_CHAT_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_chat_thinking_level == "high"
def test_llm_chat_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2, kept): the EMPTY value is the off switch — it
must not be coerced back to the default (trimming happens only at
the wire; no normalization at load)."""
monkeypatch.setenv("BOR_LLM_CHAT_THINKING_LEVEL", "")
s = _settings()
assert s.llm_chat_thinking_level == ""
def test_llm_summary_thinking_level_default_is_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2): the one-shot summary thinking level defaults to
EMPTY (``BOR_LLM_SUMMARY_THINKING_LEVEL`` unset) — the TODO's
"none" means absence, never a literal "none" string."""
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
s = _settings()
assert s.llm_summary_thinking_level == ""
def test_llm_summary_thinking_level_env_override(monkeypatch) -> None:
"""Phase 178: free-form like the chat level —
``BOR_LLM_SUMMARY_THINKING_LEVEL=high`` resolves to ``"high"``
(no allowlist, no validator)."""
monkeypatch.setenv("BOR_LLM_SUMMARY_THINKING_LEVEL", "high")
s = _settings()
assert s.llm_summary_thinking_level == "high"
def test_llm_summary_thinking_level_empty_env_value_stays_empty(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 178 (A2, kept): the EMPTY value is the off switch — it
must not be coerced to anything at load (no normalization)."""
monkeypatch.setenv("BOR_LLM_SUMMARY_THINKING_LEVEL", "")
s = _settings()
assert s.llm_summary_thinking_level == ""
def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None: def test_summary_max_chars_default_and_env_override(monkeypatch: pytest.MonkeyPatch) -> None:
+116 -33
View File
@@ -1324,48 +1324,62 @@ def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch)
assert sleeps == [1.25, 1.25, 1.25] assert sleeps == [1.25, 1.25, 1.25]
# ---------- thinking level passthrough (phase 172, task 03) ---------- # ---------- thinking level passthrough (phase 172; the split, 178 A3) ----------
# chat() (one-shot) rides the SUMMARY level; chat_stream() rides the CHAT
# level; an empty (trimmed) level is the off switch (A2 — the parameter
# is absent, never a literal "none").
def test_chat_sends_reasoning_effort_extra_body_when_level_set() -> None: # --- the one-shot path: the summary level (phase 178, A3) ---
"""Phase 172 (A1): a non-empty level rides the one-shot request as
the top-level ``reasoning_effort`` parameter via the SDK's
def test_chat_sends_thinking_level_extra_body_when_summary_level_set() -> None:
"""Phase 178 (A3): a non-empty summary level rides the one-shot
request as the top-level ``reasoning_effort`` parameter via the SDK's
``extra_body``.""" ``extra_body``."""
llm, completions = _make_chat_client( llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low" _FakeCompletion("ok"), llm_summary_thinking_level="high"
) )
asyncio.run(llm.chat([{"role": "user", "content": "q"}])) asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_default_level_low_is_always_sent() -> None: def test_chat_default_summary_thinking_off_is_pre_phase172_byte_identical() -> None:
"""The default (``low``) is always sent (A2): a client built without """The default summary level is EMPTY (the TODO's "none", A2): a
naming the setting still carries the parameter.""" client built without naming either setting sends NO ``extra_body`` —
one-shot requests are byte-identical to pre-phase-172."""
llm, completions = _make_chat_client(_FakeCompletion("ok")) llm, completions = _make_chat_client(_FakeCompletion("ok"))
asyncio.run(llm.chat([{"role": "user", "content": "q"}])) asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_chat_trims_the_level_before_sending() -> None: def test_chat_trims_the_summary_thinking_level_before_sending() -> None:
"""TRIM: a hand-edited ``.env`` value with trailing whitespace """TRIM: a hand-edited ``.env`` value with surrounding whitespace
(``"low "``) is sent trimmed — the raw stored string never reaches (``" high "``) is sent trimmed — the raw stored string never reaches
the wire with surrounding whitespace.""" the wire with whitespace."""
llm, completions = _make_chat_client( llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level="low " _FakeCompletion("ok"), llm_summary_thinking_level=" high "
) )
asyncio.run(llm.chat([{"role": "user", "content": "q"}])) asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None assert completions.chat_kwargs is not None
assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "low"} assert completions.chat_kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None: def test_chat_empty_summary_thinking_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): a whitespace value is the off switch — """Phase 172 (A2): an empty value is the off switch —
``extra_body`` is ABSENT (never None, never ``"off"``) and the ``extra_body`` is ABSENT (never None, never a literal ``"none"``) and
request kwargs keep the pre-phase-172 shape.""" the request kwargs keep the pre-phase-172 shape."""
llm, completions = _make_chat_client( llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_thinking_level=" " _FakeCompletion("ok"), llm_summary_thinking_level=""
) )
asyncio.run(llm.chat([{"role": "user", "content": "q"}])) asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None assert completions.chat_kwargs is not None
@@ -1379,30 +1393,42 @@ def test_chat_empty_level_absent_from_kwargs_byte_identical() -> None:
} }
def test_chat_stream_sends_reasoning_effort_extra_body_when_level_set() -> None: def test_chat_whitespace_summary_thinking_level_is_the_off_switch() -> None:
"""Phase 172 (A1): the chat turn rides the parameter too.""" """Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_chat_client(
_FakeCompletion("ok"), llm_summary_thinking_level=" "
)
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
def test_chat_stream_sends_thinking_level_extra_body_when_level_set() -> None:
"""Phase 178 (A3): the chat turn rides the parameter from the
**chat** level."""
llm, completions = _make_stream_client( llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level="high" [_chunk("x")], llm_chat_thinking_level="high"
) )
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"} assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_trims_the_level_before_sending() -> None: def test_chat_stream_trims_the_thinking_level_before_sending() -> None:
llm, completions = _make_stream_client( llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" high " [_chunk("x")], llm_chat_thinking_level=" high "
) )
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"} assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None: def test_chat_stream_empty_thinking_level_absent_from_kwargs_byte_identical() -> None:
"""Phase 172 (A2): the stream's off switch — ``extra_body`` absent """Phase 178 (A2): the stream's off switch — ``extra_body`` absent
(never None) and the kwargs dict keeps the pre-phase-172 shape.""" (never None, never a literal ``"none"``) and the kwargs dict keeps
the pre-phase-172 shape."""
llm, completions = _make_stream_client( llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level="" [_chunk("x")], llm_chat_thinking_level=""
) )
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None assert completions.kwargs is not None
@@ -1416,22 +1442,79 @@ def test_chat_stream_empty_level_absent_from_kwargs_byte_identical() -> None:
} }
def test_chat_stream_whitespace_level_is_the_off_switch() -> None: def test_chat_stream_whitespace_thinking_level_is_the_off_switch() -> None:
"""Whitespace-only (``" "``) is empty for wire purposes (A2).""" """Whitespace-only (``" "``) is empty for wire purposes (A2)."""
llm, completions = _make_stream_client( llm, completions = _make_stream_client(
[_chunk("x")], llm_thinking_level=" " [_chunk("x")], llm_chat_thinking_level=" "
) )
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}])) asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None assert completions.kwargs is not None
assert "extra_body" not in completions.kwargs assert "extra_body" not in completions.kwargs
# --- the cross-pins (phase 178, A3 — the whole point of the split) ---
def _make_both_client(
chunks: list,
completion: _FakeCompletion,
**settings_kwargs: Any,
) -> tuple[LLMClient, _FakeCompletions]:
"""One client for BOTH paths (a shared fake ``completions`` serves the
streaming and the non-streaming ``create()`` alike) so the two levels
can be observed on one client."""
completions = _FakeCompletions(chunks, None, completion)
fake_openai = SimpleNamespace(chat=SimpleNamespace(completions=completions))
llm = LLMClient(_settings(**settings_kwargs))
llm._client = fake_openai # pyright: ignore[reportAttributeAccessIssue]
return llm, completions
def test_split_thinking_levels_inverted_defaults_chat_high_summary_off() -> None:
"""Phase 178 (A3): the defaults INVERTED — the chat level ``high``
rides ``chat_stream`` while the (empty) summary level keeps ``chat``
free of ``extra_body``. The two knobs are independent on the wire."""
llm, completions = _make_both_client(
[_chunk("x")],
_FakeCompletion("ok"),
llm_chat_thinking_level="high",
llm_summary_thinking_level="",
)
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "high"}
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
def test_split_thinking_levels_default_client_stream_low_summary_off() -> None:
"""The DEFAULTS (chat ``low`` + summary ``""``) — the stream carries
``reasoning_effort="low"`` while the one-shot request is
byte-identical to pre-phase-172 (no ``extra_body``; the TODO's
"summary should be none")."""
llm, completions = _make_both_client([_chunk("x")], _FakeCompletion("ok"))
asyncio.run(_collect(llm, [{"role": "user", "content": "q"}]))
assert completions.kwargs is not None
assert completions.kwargs["extra_body"] == {"reasoning_effort": "low"}
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
assert completions.chat_kwargs is not None
assert "extra_body" not in completions.chat_kwargs
assert set(completions.chat_kwargs) == {
"model",
"messages",
"temperature",
"max_tokens",
"stream",
}
def test_embeddings_body_never_carries_reasoning_effort() -> None: def test_embeddings_body_never_carries_reasoning_effort() -> None:
"""Phase 172 (A3): the embeddings path is untouched — with the level """Phase 172 (A3): the embeddings path is untouched — with the level
set, every POST body stays the minimal ``{model, input}`` payload; set, every POST body stays the minimal ``{model, input}`` payload;
``reasoning_effort`` never appears in it.""" ``reasoning_effort`` never appears in it."""
service = _FakeEmbeddingsService() service = _FakeEmbeddingsService()
llm, http = _make_client(service, llm_thinking_level="high") llm, http = _make_client(service, llm_chat_thinking_level="high")
asyncio.run(llm.embed(["a", "b"])) asyncio.run(llm.embed(["a", "b"]))
vec = asyncio.run(llm.embed_one("c")) vec = asyncio.run(llm.embed_one("c"))
assert len(vec) == 768 assert len(vec) == 768
+19 -13
View File
@@ -96,19 +96,25 @@ def test_env_var_is_bare_key_upper_prefixed() -> None:
assert env_var(key).startswith("BOR_") assert env_var(key).startswith("BOR_")
def test_llm_thinking_level_is_a_plain_live_overridable_row() -> None: def test_llm_thinking_levels_are_plain_live_overridable_rows() -> None:
"""Phase 172 (task 01): the new ``llm_thinking_level`` pair sits in """Phase 178 (task 01): the two thinking-level rows (the phase-172
``LLM (aipi)`` with the ``Thinking level`` label and is plain — knob, renamed + the new summary level) sit in ``LLM (aipi)`` with
not ``secret`` (a free-form text value, like ``llm_chat_model``), the ``Chat thinking level`` / ``Summary thinking level`` labels and
not ``restart_required`` (the LLM client reads the setting per are plain — not ``secret`` (free-form text values, like
request via ``get_settings()`` — live), overridable (the overlay is ``llm_chat_model``), not ``restart_required`` (the LLM client
generic over ``str`` fields).""" reads the settings per request via ``get_settings()`` — live),
meta = CATALOG["llm_thinking_level"] overridable (the overlay is generic over ``str`` fields)."""
assert meta.group == "LLM (aipi)" for key, label in (
assert meta.label == "Thinking level" ("llm_chat_thinking_level", "Chat thinking level"),
assert "llm_thinking_level" not in SECRETS ("llm_summary_thinking_level", "Summary thinking level"),
assert "llm_thinking_level" not in RESTART_REQUIRED ):
assert "llm_thinking_level" not in NON_OVERRIDABLE meta = CATALOG[key]
assert meta.group == "LLM (aipi)"
assert meta.label == label
assert meta.secret is False
assert key not in SECRETS
assert key not in RESTART_REQUIRED
assert key not in NON_OVERRIDABLE
def test_docs_user_is_a_plain_live_overridable_row() -> None: def test_docs_user_is_a_plain_live_overridable_row() -> None: