Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Details
Build and Push Containers / build-and-push-db (push) Successful in 14s
Details

Browse Source 
## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
ducoterra
2026-10-11 23:38:18 -04:00
parent c6223c93dc
commit 149e753952
45 changed files with 1947 additions and 544 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/complete/178_split_thinking_levels/00_phase.md
+49
View File
@@ -0,0 +1,49 @@
# Phase 178 — Split thinking levels: a chat level and a summary level, separately tunable
**Source:** `TODO.md` L3 — "Need a separate summary thinking level and chat thinking level. Default thinking level for summary should be none. Chat should stay at low"
**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-11)
**Context:** `app/config.py` L108–115 — `llm_thinking_level: str = "low"` (the phase-172 shared knob, directly after `llm_summary_model` L103–107, before `stream_thinking` L116), `app/core/settings_catalog.py` L154–155 — the single `llm_thinking_level` `SettingMeta` row (label "Thinking level", group `LLM (aipi)`; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes every `Settings` field a build failure until it has a `SettingMeta`), `.env.example` L61 — `BOR_LLM_THINKING_LEVEL=low`, `app/rag/llm.py` — `_thinking_extra_body(settings)` (L49–57, reads `settings.llm_thinking_level`), `LLMClient.chat()` (L470, the one-shot `chat.completions.create` L430–452 — `extra = _thinking_extra_body(self.settings)` L432, `**extra_kwargs` L444; callers: `app/rag/summarizer.py` L242 + L359 document summaries, `app/rag/folder_summaries.py` L347, `app/rag/overview.py` L254 KB overview, `check_models` L805 the pre-sync probe), `LLMClient.chat_stream()` (L511, the kwargs dict L615–622 + `kwargs["extra_body"] = extra` L625–628), `app/api/chat.py` — the per-turn log line (L1272–1291, `thinking=%s` L1275, `settings.llm_thinking_level.strip() or "-"` L1289), the module docstring L14–19, `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: `str` catalogue entries render as free-form text rows automatically), `tests/unit/test_config.py` L108–133 (default / env override / empty-stays-empty), `tests/unit/test_settings_catalog.py` L99–111, `tests/unit/test_llm_client.py` L1330–1440 (the mocked-client `extra_body` family), `tests/unit/test_chat_gate.py` L1550–1630 (the per-turn log-line family), `tests/integration/test_admin_settings_api.py` L521–607 + `test_admin_settings_live.py` L171–204, `tests/e2e/test_thinking_level.py` (the phase-172 Playwright suite — the pattern to extend).
## Objective
The operator tunes the chat turns' thinking level and the one-shot summaries' thinking level **independently**: `llm_chat_thinking_level` (renamed from the phase-172 `llm_thinking_level`, default `low` — chat stays at low, TODO L3) rides `chat_stream()`; `llm_summary_thinking_level` (default empty = **off** — the TODO's "none", the phase-172 locked A2 off-switch: the parameter is ABSENT from the request) rides the one-shot `chat()` (document summaries, folder summaries, KB overview, the `check_models` probe). Both are free-form text, both live in `.env` (`BOR_LLM_CHAT_THINKING_LEVEL` / `BOR_LLM_SUMMARY_THINKING_LEVEL`) and the admin Settings panel, both live-apply without a restart.
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the two fields (task 01).** `app/config.py` LLM section: `llm_thinking_level` is **renamed** to `llm_chat_thinking_level` (default stays `"low"`) and `llm_summary_thinking_level: str = ""` is added next to it (both before `stream_thinking`). Plain `str`, no allowlist, no validator (phase-172 A4, kept). House docstrings name the env var, the default, the wire mapping and the off rule. `.env.example` L61: `BOR_LLM_THINKING_LEVEL=low` is **replaced** by `BOR_LLM_CHAT_THINKING_LEVEL=low` + `BOR_LLM_SUMMARY_THINKING_LEVEL=` (empty value, one-line comment each).
- **D2 — the consumers rename with the field (task 01, behavior-identical).** Every consumer of the old field is mechanically renamed to `llm_chat_thinking_level` in task 01 — `app/rag/llm.py` (module docstring, `_thinking_extra_body`), `app/api/chat.py` (module docstring, per-turn log line L1289) — and every test pin is renamed in place (unit `test_config.py` / `test_settings_catalog.py` / `test_llm_client.py` / `test_chat_gate.py`, integration `test_admin_settings_api.py` / `test_admin_settings_live.py`, e2e `test_thinking_level.py`). After task 01 the suite is green and behavior is **byte-identical** to pre-phase: both `chat()` and `chat_stream()` ride the (renamed) chat level, default `low`; the new summary field exists but is inert until task 03.
- **D3 — the wire split (task 03).** `_thinking_extra_body` takes the **level string** (pure function, no `Settings` read): `chat_stream()` passes `self.settings.llm_chat_thinking_level`; `chat()` passes `self.settings.llm_summary_thinking_level`. Trimmed non-empty → `{"reasoning_effort": <value>}`; empty/whitespace → `None` (the parameter is ABSENT — never a literal `"none"` or `"off"`, phase-172 A2). The `check_models` probe (L805) goes through `chat()` → summary level → off by default; no special-casing (it is a reachability ping). Embeddings path untouched.
- **D4 — the admin rows (task 02).** The single `SettingMeta("llm_thinking_level", …, "Thinking level")` becomes TWO rows: `llm_chat_thinking_level` → "Chat thinking level", `llm_summary_thinking_level` → "Summary thinking level" (both group `LLM (aipi)`, placed where the old row sat — after `llm_summary_model`, before `stream_thinking`). The panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes and the DB-over-env overlay are generic over the catalogue — the two free-form text rows, their save/reset and live-apply fall out with zero extra code. The completeness pin is the enforcement.
- **D5 — the per-turn line (task 04).** The per-turn log line's `thinking=<value>` (or `-` when empty) reads the **chat** level — the level the turn actually ran at; the summary level is a one-shot concern and gains no log field (the summary log lines have no knob field today). No `query_log` schema change (phase-172 A6, kept).
- **NOT touched:** `frontend/assets/settings.js`, `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events), every other setting.
## Dependencies
— (none: the app is fully functional at the head of `complete/`; this phase refines the committed phase-172 thinking-level machinery)
## Tasks
1. `01_settings_catalog_env.md` — the rename + the new field in `app/config.py` + the two `CATALOG` rows + `.env.example` + the mechanical consumer/test rename (suite green, behavior byte-identical).
2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for BOTH keys (stored raw, live without restart, empty storable as the summary off-switch).
3. `03_llm_extra_body_split.md` — `chat_stream()` → chat level, `chat()` → summary level; the mocked-client unit tests per method.
4. `04_per_turn_log_field.md` — `thinking=<chat level>` (or `-`) pinned: the line follows the chat knob, never the summary knob.
5. `05_e2e_admin_settings_rows.md` — `tests/e2e/test_split_thinking_levels.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_config.py` — chat default `low` / env override / empty-stays-empty; summary default `""` / env override / empty-stays-empty (no coercion, no allowlist — D3/A2). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (both fields have their `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat_stream()` sends `extra_body={"reasoning_effort": <chat level>}` when the chat level is trimmed-non-empty and no `extra_body` when it is empty; `chat()` sends it from the **summary** level with the same absent-when-empty rule; an empty summary level (the default) keeps one-shot requests byte-identical to pre-phase-172; the embeddings path is unaffected. `tests/unit/test_chat_gate.py` — the per-turn line shows the chat level (with the summary level deliberately different — it must not leak into the line).
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — BOTH rows appear in `GET /api/admin/settings` (`str`, correct labels, effective defaults `low` / ``); `PUT` stores the raw string on each, the empty value is storable (the summary off-switch), the overlay resolves live without a restart; `DELETE` resets each to env.
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_split_thinking_levels.py`, run **in isolation** — both rows visible in the `LLM (aipi)` group (labels "Chat thinking level" / "Summary thinking level"), chat default `low`, summary default empty, both free-form text inputs (type, save, survives reload), both reset-to-env restore the defaults, the env-var display pins `BOR_LLM_CHAT_THINKING_LEVEL` / `BOR_LLM_SUMMARY_THINKING_LEVEL`; the `test_admin_settings_panel.py` + `test_admin_settings_layout.py` regression set stays green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest -k "config and thinking" -v` shows chat default `low` + summary default `""` (env overrides for both).
- [ ] `GET /api/admin/settings` (admin) lists `llm_chat_thinking_level` and `llm_summary_thinking_level` in the `LLM (aipi)` group (`str`, defaults `low` / empty); `PUT` / `DELETE` round-trip each per the overlay contract; the old `llm_thinking_level` key no longer appears.
- [ ] With the levels set, `chat_stream()` carries `reasoning_effort` from the chat level and `chat()` from the summary level (unit-asserted via the mocked client); with a level empty, that method's request kwargs contain no `extra_body` (byte-identical to pre-phase-172 for the default one-shot path).
- [ ] The per-turn log line carries `thinking=<chat level>` (`-` when empty) — never the summary level.
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
- [ ] `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation.
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): split the thinking level into a chat level (default low) and a summary level (default off)`); the phase dir moves to `.agents/phases/complete/`.
## Locked decisions (owner-confirmed 2026-10-11, roadmap confirmation)
- **A1 — rename, not add-on:** `llm_thinking_level` is **renamed** to `llm_chat_thinking_level` (`BOR_LLM_THINKING_LEVEL` → `BOR_LLM_CHAT_THINKING_LEVEL`). Any DB-stored override under the old key becomes inert (it is no longer a catalogue row) and chat falls back to the env default `low` — chat behavior is preserved exactly as the TODO wants; no value migration.
- **A2 — the TODO's "none" = the phase-172 off-switch:** `llm_summary_thinking_level` defaults to empty (`""`) → `reasoning_effort` is ABSENT from one-shot `chat()` requests. Per phase-172's locked A2 ("nothing is sent as `off` or `null` — absence is the off"), no literal `"none"` string is ever sent.
- **A3 — scope:** `chat_stream()` (the chat turn) → chat level; every one-shot `chat()` (document summaries, folder summaries, KB overview, the `check_models` probe) → summary level; the embeddings path is untouched.
- **A4 — no frontend code change:** the panel is generated from the catalogue; the two free-form text rows appear automatically (phase-172 A5 pattern).
- **A5 — the per-turn line** keeps its single `thinking=<chat level>` field (`-` when empty), reads the chat level, and gains no summary field; no `query_log` schema change.