Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 172_thinking_level
Build and Push Containers / build-and-push-app (push) Successful in 1m43s
Details
Build and Push Containers / build-and-push-db (push) Successful in 12s
Details

Browse Source 
All verification complete. Final report:

**Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.**

- **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`.
- **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands).
- **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol.
- **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
This commit is contained in:
ducoterra
2026-10-10 13:19:10 -04:00
parent b21b05eaee
commit f27106a638
43 changed files with 1689 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/complete/172_thinking_level/00_phase.md
+50
View File
@@ -0,0 +1,50 @@
# Phase 172 — Thinking level: a pi-style, operator-tunable LLM reasoning knob
**Source:** `TODO.md` L3 — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-10)
**Context:** `app/config.py` — the LLM section (L97–125: `llm_base_url`, `llm_api_key`, `llm_chat_model` L101, `llm_embed_model`, `llm_summary_model` L107, `stream_thinking` L112, `llm_retries` / `llm_retry_delay` / `llm_timeout` L122 — the house pattern this setting joins), `.env.example` L57–65 (the LLM block — `BOR_LLM_CHAT_MODEL=turbo` L60), `app/core/settings_catalog.py` L137–161 (the `LLM (aipi)` catalogue group; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes the new field a build failure until it has a `SettingMeta`), `app/core/settings_overlay.py` (the generic DB-over-env overlay — a plain `str` field rides it with zero extra code: `PUT` stores the raw string, `invalidate_settings()` re-reads), `app/rag/llm.py` — `LLMClient.chat()` (L446, the one-shot `chat.completions.create` L416, `temperature=0.2`, 2048 tokens) and `LLMClient.chat_stream()` (L511, the kwargs dict L592–600: `model` / `messages` / `temperature=0.4` / `max_tokens` / `stream`), `app/api/chat.py` — the per-turn log line (L1268–1291: `…thinking_chars=%d tool_calls=%d…`), `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: a new `str` catalogue entry renders as a free-form text row automatically, exactly like `llm_chat_model`), `tests/unit/test_config.py`, `tests/unit/test_llm_client.py` (the mocked-SDK-client family), `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (the set/reset/live-apply contract), `tests/e2e/test_admin_settings_panel.py` (the phase-168 suite — the pattern to copy).
## Objective
The operator can pick the model's thinking level — pi-style, free-form text, default `low` — from `.env` (`BOR_LLM_THINKING_LEVEL`) or the admin Settings panel, and the value rides every chat-completions request to aipi as the OpenAI-standard `reasoning_effort` parameter, so the same knob works as the model's reasoning support evolves. The empty value is the off switch (the parameter is not sent at all).
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — one `Settings` field (task 01).** `llm_thinking_level: str = "low"` in the LLM section of `app/config.py` (placed directly after the `llm_summary_model` block, before `stream_thinking` — model-identity settings stay together). Free-form, exactly like `llm_chat_model` (A4: **no allowlist, no validator** — any non-empty text is accepted; the endpoint is the source of truth). The docstring names the env var (`BOR_LLM_THINKING_LEVEL`), the default, the wire mapping (D2) and the empty-off rule (D3). `.env.example` gains `BOR_LLM_THINKING_LEVEL=low` directly under `BOR_LLM_CHAT_MODEL=turbo` (L60) with a one-line comment (the house style of L61–63).
- **D2 — the wire (task 03).** The value is sent as a **top-level chat-completions parameter** `reasoning_effort` via the OpenAI SDK's `extra_body` (`extra_body={"reasoning_effort": "<value>"}` — the SDK merges `extra_body` into the request JSON body). Applied in BOTH `LLMClient.chat()` (the one-shot: document summaries, KB overview) and `LLMClient.chat_stream()` (the chat turn) — A3. The embeddings path (`_post_embeddings` / `embed` / `embed_one`) is **untouched**. `ASSUMPTION: A1 — aipi accepts the OpenAI-standard `reasoning_effort` top-level parameter (pi's `reasoningEffort`); if aipi exposes a different key (e.g. `chat_template_kwargs`), only task 03's key changes — the rest of the phase is key-agnostic by design.**
- **D3 — the off switch (tasks 03/04).** The **trimmed** value decides: non-empty → the parameter is sent with the trimmed value; empty/whitespace → `extra_body` is absent from the request (byte-identical to today's payload). Nothing is sent as `"off"` or `null` — absence is the off.
- **D4 — the admin row (task 02).** One `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` entry in `CATALOG` (placed after `llm_summary_model` L150–151, before `llm_retries` L156). Nothing else: the panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes, and the DB-over-env overlay are all **generic over the catalogue** — the row, its free-form text input, its save/reset and its live-apply fall out with zero extra code (A5). The completeness pin is the enforcement: a missing entry fails the build.
- **D5 — the per-turn line (task 04).** The per-turn log line (PLAN §9, `app/api/chat.py` L1268) gains `thinking=<value>` immediately before the existing `thinking_chars=%d` — the effective (overlay-resolved) value, trimmed, or `-` when empty, so "which level did this turn run at" stays one `grep` away (AGENTS rule 10). No `query_log` schema change (the table is retrieval-scoped; the line is the home of turn knobs).
- **NOT touched:** `frontend/assets/settings.js` (generated from the catalogue), `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events — different concern, stays as-is), every other setting.
## Dependencies
— (none: the app is fully functional at the head of `complete/`; this phase builds on the committed phase-168/171 settings machinery)
## Tasks
1. `01_setting_and_env.md` — `Settings.llm_thinking_level: str = "low"` + the `CATALOG` row (the completeness pin keeps the build green) + `BOR_LLM_THINKING_LEVEL=low` in `.env.example`; the config/catalog unit tests.
2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for the new key (stored raw, live without restart, empty storable as the off switch).
3. `03_llm_extra_body_passthrough.md` — `extra_body={"reasoning_effort": …}` on `chat()` + `chat_stream()` when trimmed-non-empty; absent when empty; the mocked-client unit tests.
4. `04_per_turn_log_field.md` — `thinking=<value>` (or `-`) on the per-turn log line; the log-line test.
5. `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_config.py` — the default is `low`, the env override works, an empty env value yields an empty string (no coercion, no allowlist — D3/A4). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (the new field has its `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat()` and `chat_stream()` send `extra_body={"reasoning_effort": <trimmed value>}` when the setting is non-empty, send **no** `extra_body` when it is empty/whitespace, and the embeddings path is unaffected (the mocked-SDK-client family).
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — the row appears in `GET /api/admin/settings` (group, `str`, effective `low`); `PUT` stores the raw string, the empty value is storable (the off switch), the overlay resolves live without a restart; `DELETE` resets to env.
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_thinking_level.py`, run **in isolation** — the row is visible in the `LLM (aipi)` group with default `low`, is a free-form text input (type `high`, save, survives reload), reset-to-env restores `low`, and the `test_admin_settings_panel.py` regression set stays green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] `uv run pytest -k "config and thinking" -v` shows the default `low` + the env override; `.env.example` contains `BOR_LLM_THINKING_LEVEL=low` under `BOR_LLM_CHAT_MODEL`.
- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, default `low`); `PUT` / `DELETE` round-trip per the overlay contract.
- [ ] With the setting non-empty, a `chat_stream()` / `chat()` request carries `reasoning_effort` (unit-asserted via the mocked client); with it empty, the request kwargs contain no `extra_body` (byte-identical to today).
- [ ] The per-turn log line carries `thinking=<value>` (`-` when empty) next to `thinking_chars`.
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation.
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort`); the phase dir moves to `.agents/phases/complete/`.
## Locked decisions (owner-confirmed 2026-10-10, roadmap confirmation)
- **A1 — the wire key is `reasoning_effort`** (the OpenAI-standard top-level chat-completions parameter, pi's `reasoningEffort`), sent via the SDK's `extra_body`. If aipi exposes a different key, only task 03 changes.
- **A2 — the off switch is absence:** empty/whitespace value → the parameter is not sent at all; the default `low` is always sent.
- **A3 — scope:** the parameter rides BOTH `chat_stream()` (the chat turn) and `chat()` (the one-shot summaries / KB overview); the embeddings path is untouched.
- **A4 — free-form, no allowlist:** any non-empty text is accepted (like `llm_chat_model`); no validator, no value coercion.
- **A5 — no frontend code change:** the panel is generated from the catalogue; the row appears automatically as a free-form text input.
- **A6 — the per-turn line** gains `thinking=<trimmed value>` (`-` when empty) immediately before `thinking_chars`; no `query_log` schema change.