chore(agent): phase roadmap from TODO.md, phase 172 (thinking level)
LLM-Generated: true
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
# Phase 172 — Thinking level: a pi-style, operator-tunable LLM reasoning knob
|
||||
|
||||
**Source:** `TODO.md` L3 — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
|
||||
**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-10)
|
||||
**Context:** `app/config.py` — the LLM section (L97–125: `llm_base_url`, `llm_api_key`, `llm_chat_model` L101, `llm_embed_model`, `llm_summary_model` L107, `stream_thinking` L112, `llm_retries` / `llm_retry_delay` / `llm_timeout` L122 — the house pattern this setting joins), `.env.example` L57–65 (the LLM block — `BOR_LLM_CHAT_MODEL=turbo` L60), `app/core/settings_catalog.py` L137–161 (the `LLM (aipi)` catalogue group; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes the new field a build failure until it has a `SettingMeta`), `app/core/settings_overlay.py` (the generic DB-over-env overlay — a plain `str` field rides it with zero extra code: `PUT` stores the raw string, `invalidate_settings()` re-reads), `app/rag/llm.py` — `LLMClient.chat()` (L446, the one-shot `chat.completions.create` L416, `temperature=0.2`, 2048 tokens) and `LLMClient.chat_stream()` (L511, the kwargs dict L592–600: `model` / `messages` / `temperature=0.4` / `max_tokens` / `stream`), `app/api/chat.py` — the per-turn log line (L1268–1291: `…thinking_chars=%d tool_calls=%d…`), `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: a new `str` catalogue entry renders as a free-form text row automatically, exactly like `llm_chat_model`), `tests/unit/test_config.py`, `tests/unit/test_llm_client.py` (the mocked-SDK-client family), `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (the set/reset/live-apply contract), `tests/e2e/test_admin_settings_panel.py` (the phase-168 suite — the pattern to copy).
|
||||
|
||||
## Objective
|
||||
The operator can pick the model's thinking level — pi-style, free-form text, default `low` — from `.env` (`BOR_LLM_THINKING_LEVEL`) or the admin Settings panel, and the value rides every chat-completions request to aipi as the OpenAI-standard `reasoning_effort` parameter, so the same knob works as the model's reasoning support evolves. The empty value is the off switch (the parameter is not sent at all).
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **D1 — one `Settings` field (task 01).** `llm_thinking_level: str = "low"` in the LLM section of `app/config.py` (placed directly after the `llm_summary_model` block, before `stream_thinking` — model-identity settings stay together). Free-form, exactly like `llm_chat_model` (A4: **no allowlist, no validator** — any non-empty text is accepted; the endpoint is the source of truth). The docstring names the env var (`BOR_LLM_THINKING_LEVEL`), the default, the wire mapping (D2) and the empty-off rule (D3). `.env.example` gains `BOR_LLM_THINKING_LEVEL=low` directly under `BOR_LLM_CHAT_MODEL=turbo` (L60) with a one-line comment (the house style of L61–63).
|
||||
- **D2 — the wire (task 03).** The value is sent as a **top-level chat-completions parameter** `reasoning_effort` via the OpenAI SDK's `extra_body` (`extra_body={"reasoning_effort": "<value>"}` — the SDK merges `extra_body` into the request JSON body). Applied in BOTH `LLMClient.chat()` (the one-shot: document summaries, KB overview) and `LLMClient.chat_stream()` (the chat turn) — A3. The embeddings path (`_post_embeddings` / `embed` / `embed_one`) is **untouched**. `ASSUMPTION: A1 — aipi accepts the OpenAI-standard `reasoning_effort` top-level parameter (pi's `reasoningEffort`); if aipi exposes a different key (e.g. `chat_template_kwargs`), only task 03's key changes — the rest of the phase is key-agnostic by design.**
|
||||
- **D3 — the off switch (tasks 03/04).** The **trimmed** value decides: non-empty → the parameter is sent with the trimmed value; empty/whitespace → `extra_body` is absent from the request (byte-identical to today's payload). Nothing is sent as `"off"` or `null` — absence is the off.
|
||||
- **D4 — the admin row (task 02).** One `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` entry in `CATALOG` (placed after `llm_summary_model` L150–151, before `llm_retries` L156). Nothing else: the panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes, and the DB-over-env overlay are all **generic over the catalogue** — the row, its free-form text input, its save/reset and its live-apply fall out with zero extra code (A5). The completeness pin is the enforcement: a missing entry fails the build.
|
||||
- **D5 — the per-turn line (task 04).** The per-turn log line (PLAN §9, `app/api/chat.py` L1268) gains `thinking=<value>` immediately before the existing `thinking_chars=%d` — the effective (overlay-resolved) value, trimmed, or `-` when empty, so "which level did this turn run at" stays one `grep` away (AGENTS rule 10). No `query_log` schema change (the table is retrieval-scoped; the line is the home of turn knobs).
|
||||
- **NOT touched:** `frontend/assets/settings.js` (generated from the catalogue), `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events — different concern, stays as-is), every other setting.
|
||||
|
||||
## Dependencies
|
||||
— (none: the app is fully functional at the head of `complete/`; this phase builds on the committed phase-168/171 settings machinery)
|
||||
|
||||
## Tasks
|
||||
1. `01_setting_and_env.md` — `Settings.llm_thinking_level: str = "low"` + the `CATALOG` row (the completeness pin keeps the build green) + `BOR_LLM_THINKING_LEVEL=low` in `.env.example`; the config/catalog unit tests.
|
||||
2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for the new key (stored raw, live without restart, empty storable as the off switch).
|
||||
3. `03_llm_extra_body_passthrough.md` — `extra_body={"reasoning_effort": …}` on `chat()` + `chat_stream()` when trimmed-non-empty; absent when empty; the mocked-client unit tests.
|
||||
4. `04_per_turn_log_field.md` — `thinking=<value>` (or `-`) on the per-turn log line; the log-line test.
|
||||
5. `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_config.py` — the default is `low`, the env override works, an empty env value yields an empty string (no coercion, no allowlist — D3/A4). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (the new field has its `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat()` and `chat_stream()` send `extra_body={"reasoning_effort": <trimmed value>}` when the setting is non-empty, send **no** `extra_body` when it is empty/whitespace, and the embeddings path is unaffected (the mocked-SDK-client family).
|
||||
- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — the row appears in `GET /api/admin/settings` (group, `str`, effective `low`); `PUT` stores the raw string, the empty value is storable (the off switch), the overlay resolves live without a restart; `DELETE` resets to env.
|
||||
- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_thinking_level.py`, run **in isolation** — the row is visible in the `LLM (aipi)` group with default `low`, is a free-form text input (type `high`, save, survives reload), reset-to-env restores `low`, and the `test_admin_settings_panel.py` regression set stays green.
|
||||
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest -k "config and thinking" -v` shows the default `low` + the env override; `.env.example` contains `BOR_LLM_THINKING_LEVEL=low` under `BOR_LLM_CHAT_MODEL`.
|
||||
- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, default `low`); `PUT` / `DELETE` round-trip per the overlay contract.
|
||||
- [ ] With the setting non-empty, a `chat_stream()` / `chat()` request carries `reasoning_effort` (unit-asserted via the mocked client); with it empty, the request kwargs contain no `extra_body` (byte-identical to today).
|
||||
- [ ] The per-turn log line carries `thinking=<value>` (`-` when empty) next to `thinking_chars`.
|
||||
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort`); the phase dir moves to `.agents/phases/complete/`.
|
||||
|
||||
## Locked decisions (owner-confirmed 2026-10-10, roadmap confirmation)
|
||||
- **A1 — the wire key is `reasoning_effort`** (the OpenAI-standard top-level chat-completions parameter, pi's `reasoningEffort`), sent via the SDK's `extra_body`. If aipi exposes a different key, only task 03 changes.
|
||||
- **A2 — the off switch is absence:** empty/whitespace value → the parameter is not sent at all; the default `low` is always sent.
|
||||
- **A3 — scope:** the parameter rides BOTH `chat_stream()` (the chat turn) and `chat()` (the one-shot summaries / KB overview); the embeddings path is untouched.
|
||||
- **A4 — free-form, no allowlist:** any non-empty text is accepted (like `llm_chat_model`); no validator, no value coercion.
|
||||
- **A5 — no frontend code change:** the panel is generated from the catalogue; the row appears automatically as a free-form text input.
|
||||
- **A6 — the per-turn line** gains `thinking=<trimmed value>` (`-` when empty) immediately before `thinking_chars`; no `query_log` schema change.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Task 01 — The `llm_thinking_level` setting + the catalogue row + `.env.example`
|
||||
|
||||
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The new setting exists in `app/config.py` (default `low`, free-form like `llm_chat_model`), in the admin catalogue (so the build's completeness pin stays green — a field without a `SettingMeta` fails the build, and the per-task gate runs the full suite), and in `.env.example`, so `BOR_LLM_THINKING_LEVEL` is a first-class, documented knob before anything consumes it.
|
||||
|
||||
## Work
|
||||
1. `app/config.py` — in the LLM section, **directly after** the `llm_summary_model` block (L103–107) and **before** `stream_thinking` (L108), add ONE field, house docstring style (env var named, default, wire mapping, off rule):
|
||||
```python
|
||||
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
|
||||
#: pi-style reasoning effort, free-form text exactly like
|
||||
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
|
||||
#: truth). Default ``low``. Rides every chat-completions request to
|
||||
#: aipi as the top-level ``reasoning_effort`` parameter
|
||||
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
|
||||
#: parameter is not sent at all.
|
||||
llm_thinking_level: str = "low"
|
||||
```
|
||||
No validator, no coercion — a plain `str` (ASSUMPTION A4, phase-locked). Verify by reading: `llm_chat_model` (L101) keeps its shape byte-identical.
|
||||
2. `.env.example` — in the LLM block (L57–65), directly under `BOR_LLM_CHAT_MODEL=turbo` (L60), add:
|
||||
```
|
||||
BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent
|
||||
```
|
||||
(one-line comment, the house style of L61–63; `#`-prefixed optional vars stay commented — this one is set, like L60).
|
||||
3. `app/core/settings_catalog.py` — in the `LLM (aipi)` catalogue group, **directly after** the `llm_summary_model` entry (L150–151) and **before** `llm_retries` (L156), add ONE entry:
|
||||
```python
|
||||
"llm_thinking_level": SettingMeta(
|
||||
"llm_thinking_level", "LLM (aipi)", "Thinking level"
|
||||
),
|
||||
```
|
||||
Nothing else — `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` are DERIVED from `CATALOG` (module docstring), so no other list changes. The row is plain: not `secret`, not `restart_required` (the LLM client reads the setting per request via `get_settings()` — live), overridable (the overlay is generic over `str` fields, `app/core/settings_overlay.py`).
|
||||
4. `tests/unit/test_settings_catalog.py` — the completeness pin (`set(CATALOG) == set(Settings.model_fields)`) must stay green with the new pair; add the house-style assertion that `CATALOG["llm_thinking_level"]` sits in group `LLM (aipi)` with label `Thinking level` and is **not** in `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` (the derived sets).
|
||||
5. `tests/unit/test_config.py` — add the house-style cases: (a) the DEFAULT is `"low"` (no env set); (b) the env override — `BOR_LLM_THINKING_LEVEL=high` resolves to `"high"`; (c) the empty env value (`BOR_LLM_THINKING_LEVEL=`) resolves to `""` (no coercion to a default, no allowlist — the off switch stays expressible).
|
||||
|
||||
- ASSUMPTION: A4 (phase-locked) — free-form text, no allowlist/validator; A2 (phase-locked) — empty = off switch, so the setting must NOT strip/normalize away the empty value at load (it is trimmed only at the wire, task 03); A5 (phase-locked) — the panel row falls out of the catalogue with zero `settings.js` code.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_config.py` + `tests/unit/test_settings_catalog.py` green (`uv run pytest tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov`).
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` green: default `low`, env override `high`, empty stays `""`, the completeness pin holds with the new pair.
|
||||
- [ ] `grep -n "BOR_LLM_THINKING_LEVEL" .env.example` shows the line under `BOR_LLM_CHAT_MODEL=turbo`.
|
||||
- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, effective `low`) — the pin + a quick curl, or the task-02 integration test.
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (no other `Settings` field touched; `settings.js` untouched).
|
||||
@@ -0,0 +1,24 @@
|
||||
# Task 02 — The admin row's API contract (set / reset / live-apply for `llm_thinking_level`)
|
||||
|
||||
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. … Update .env and admin settings."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Prove, at the API/overlay level, that the new key behaves exactly like `llm_chat_model`: it appears in the catalogue, `PUT` stores the raw string and the overlay resolves it live (no restart), `DELETE` resets to env, and the effective value the LLM layer will read follows the overlay.
|
||||
|
||||
## Work
|
||||
1. `tests/integration/test_admin_settings_api.py` — extend the existing key-parameterized patterns (read how the phase-168 suite picks a `str` key, e.g. `llm_chat_model`) with the house-style cases for `llm_thinking_level`: (a) `GET /api/admin/settings` → the item is present, group `LLM (aipi)`, `type str`, effective value `low` (no row in `app_settings`), `env_var` `BOR_LLM_THINKING_LEVEL`; (b) `PUT` with `{"value": "high"}` → 200, the row is stored **raw** in `app_settings` (env-native string form, D1 of the overlay), the response reports the new effective value; (c) `PUT` with an empty string `""` → 200 (the off switch is storable — the API stores as-is, no allowlist); (d) `DELETE /api/admin/settings/llm_thinking_level` → the row is gone, effective value back to the env default `low`; (e) the non-admin 403 gate still applies (the phase-168 router dependency — assert once, mirroring the existing suite).
|
||||
2. `tests/integration/test_admin_settings_live.py` — add the live-apply case in the house pattern: `PUT llm_thinking_level=high` → `get_settings().llm_thinking_level` is `"high"` **without** a restart (the `invalidate_settings()` path); `DELETE` → back to `"low"`. (This is the proof the LLM layer — which reads `get_settings()` per request, task 03 — picks the override up live.)
|
||||
3. If the phase-168 suites parameterize over `CATALOG` keys generically (check `tests/integration/test_admin_settings_api.py` first — read it, do not assume): the new key may already be covered by the parameterization; in that case add only the empty-string (c) case and the live-apply (2) case, and note it in the test's comment.
|
||||
|
||||
- ASSUMPTION: A4 (phase-locked) — the empty string is storable (the off switch must be settable from the panel, not only via `.env`); the API "stores as-is" per the phase-168 module docstring — no value validation to add.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` green (`uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov`).
|
||||
- Coverage: **>90%** on `app/` (unchanged `app/` code — the gate must stay green, not drop).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/admin/settings` (admin) lists the row (`LLM (aipi)`, `str`, `low`); `PUT high` → live effective `high`; `PUT ""` → live effective `""`; `DELETE` → back to `low` (the integration cases green).
|
||||
- [ ] `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` green; full suite green.
|
||||
- [ ] ruff + pyright clean; no `app/` code changed by this task.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Task 03 — The thinking level rides every chat-completions request (`reasoning_effort` via `extra_body`)
|
||||
|
||||
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
`LLMClient` sends the operator's thinking level to aipi on every chat-completions request — `chat()` (the one-shot) and `chat_stream()` (the chat turn) — as the top-level `reasoning_effort` parameter via the SDK's `extra_body`; when the (trimmed) value is empty the request is byte-identical to today's (no `extra_body` at all).
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py` — ONE small helper (module-level, near the top, after the imports):
|
||||
```python
|
||||
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
|
||||
"""The phase-172 thinking level as the chat-completions extra body.
|
||||
|
||||
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
|
||||
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
|
||||
(the off switch — the parameter is ABSENT from the request, A2).
|
||||
"""
|
||||
level = settings.llm_thinking_level.strip()
|
||||
return {"reasoning_effort": level} if level else None
|
||||
```
|
||||
Then in **BOTH** request sites, add the parameter only when present:
|
||||
- `chat()` (L446, the `await self._client.chat.completions.create(...)` at L416–421): compute `extra = _thinking_extra_body(self.settings)` and pass the kwarg **only when present** — `**({"extra_body": extra} if extra is not None else {})` spread into the `create(...)` call, so `extra_body` is ABSENT (never `None`) when off.
|
||||
- `chat_stream()` (L511, the kwargs dict L592–600): same pattern — after the dict is built (and after the `if tools is not None:` block, L601–602), `if (extra := _thinking_extra_body(self.settings)) is not None: kwargs["extra_body"] = extra`.
|
||||
The value is the **trimmed** setting (`.strip()` in the helper) — the raw stored string is never sent with surrounding whitespace. The embeddings path (`_post_embeddings` L256, `embed` L346, `embed_one` L400) is **untouched** (A3) — verify by reading: no `extra_body` appears anywhere in the embedding code.
|
||||
2. `app/rag/llm.py` — update the module docstring (L1–L30) with ONE sentence: the thinking level (phase 172, `BOR_LLM_THINKING_LEVEL`) rides chat completions as `reasoning_effort`; empty = absent. (The house style: wire facts live in the module docstring, cf. the `reasoning_content` note at L10–12.)
|
||||
3. `tests/unit/test_llm_client.py` — extend the mocked-SDK-client family (read the existing `chat` / `chat_stream` cases first and mirror their mocking):
|
||||
- `chat()` with `llm_thinking_level="low"` → `create` is awaited with `extra_body={"reasoning_effort": "low"}` (assert on the call kwargs).
|
||||
- `chat()` with `llm_thinking_level=" "` (whitespace) → `extra_body` is **absent from the call kwargs** (assert `"extra_body" not in kwargs`).
|
||||
- `chat_stream()` with `"high"` → same `extra_body` assertion on the `create` call.
|
||||
- `chat_stream()` with `""` → `extra_body` absent (the off switch — the kwargs dict is byte-identical in shape to pre-phase-172).
|
||||
- one `embed()` / `embed_one()` case with the level set → the HTTP POST body (the mocked transport) contains **no** `reasoning_effort` (the embeddings path is untouched, A3).
|
||||
- the TRIM case: `llm_thinking_level="low "` (trailing spaces — reachable via a hand-edited `.env`) → the sent value is `"low"`.
|
||||
|
||||
- ASSUMPTION: A1 (phase-locked) — the wire key is the OpenAI-standard top-level `reasoning_effort` via `extra_body`; if aipi exposes a different key, ONLY this task's helper changes (the setting, the row, the log, the E2E are key-agnostic). A2 (phase-locked) — absence is the off switch (never `extra_body=None`, never `"off"`). A3 (phase-locked) — `chat()` + `chat_stream()` only; embeddings untouched.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_llm_client.py` green (`uv run pytest tests/unit/test_llm_client.py -v --no-cov`).
|
||||
- Coverage: **>90%** on `app/` (the new helper is fully covered by the cases above).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With the setting non-empty, a `chat()` / `chat_stream()` request carries `extra_body={"reasoning_effort": <trimmed value>}` (the unit cases green).
|
||||
- [ ] With it empty/whitespace, `extra_body` is absent from the request kwargs (byte-identical shape to today); the embeddings POST body never contains `reasoning_effort`.
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean.
|
||||
- [ ] no behavior change in completed work (temperature / max_tokens / tools / stream kwargs untouched; the `stream_thinking` kill-switch untouched).
|
||||
@@ -0,0 +1,24 @@
|
||||
# Task 04 — The per-turn log line records the effective thinking level
|
||||
|
||||
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The per-turn log line (PLAN §9, AGENTS rule 10) records the effective thinking level of the turn, so "which level did this turn run at" is one `grep` away, next to the `thinking_chars` it governs.
|
||||
|
||||
## Work
|
||||
1. `app/api/chat.py` — the per-turn `logger.info(...)` (L1268–1291): insert `thinking=%s` **immediately before** the existing `thinking_chars=%d` in the format string, with the argument `settings.llm_thinking_level.strip() or "-"` (the effective overlay-resolved value — `settings` is already in scope, the line already logs `settings.relevance_threshold` L1280; trimmed, `-` when empty, per A6). No reordering of the existing fields; the `query_log` row (L1250–1260) is **untouched** (A6: the table is retrieval-scoped, the line is the home of turn knobs).
|
||||
2. `app/api/chat.py` — the module docstring: the existing per-turn-line notes (L14, L46, L64, L78, L141, L161) set the house pattern — add ONE line where the log line's fields are described: the per-turn log line records `thinking=<level>` (the effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172).
|
||||
3. The existing per-turn-line test (find it: `grep -rn "thinking_chars" tests/` — the house pattern pins the line's fields in one place; extend that test, do not invent a new file): assert the new `thinking=` field is present, carries the effective value in the non-empty case, and is `-` when the setting is empty.
|
||||
|
||||
- ASSUMPTION: A6 (phase-locked) — `thinking=<trimmed value>` or `-`, placed immediately before `thinking_chars`; no `query_log` schema change, no new `Settings` read (the per-turn `settings` object is already resolved through the overlay).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the per-turn-line test (above) green; `uv run pytest` green (the SSE turn machinery is untouched — the change is one format field + one argument).
|
||||
- Coverage: **>90%** on `app/` (the gate must stay green, not drop).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A test turn's per-turn log line (the test's captured log) contains `thinking=low` (default) and `-` when the setting is empty; `thinking_chars` stays in its position after it.
|
||||
- [ ] `grep -rn "thinking=" app/api/chat.py` shows the field exactly once in the format string.
|
||||
- [ ] `uv run pytest` green; ruff + pyright clean; no `query_log` schema change.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 05 — The phase's dedicated Playwright suite (AGENTS rule 4) + the full gates + the atomic commit
|
||||
|
||||
**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
`tests/e2e/test_thinking_level.py` proves the owner's acceptance in a real browser — the Thinking level row exists in the admin Settings panel as a **free-form text** entry (like the chat model's), defaults to `low`, saves and persists, and resets to env — run **in isolation**; then the full gates, then ONE atomic commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_thinking_level.py` — the phase's dedicated suite, run in isolation (DB must be up: `podman compose up -d db`; `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov`). **Copy the harness from `tests/e2e/test_admin_settings_panel.py`** (read it first, then mirror: the `_boot_app`/`app_server`/`app_url` fixtures with the mock LLM, the `_clean` autouse fixture (its `app_settings` truncation pattern — the new key's rows must be wiped between tests), the `_cookies`/`_admin_client` helpers, the settings-page nav + Save/Reset drives at L561ff). Three tests:
|
||||
- `test_row_renders_free_form_with_default_low` — admin opens `/settings.html`; the `LLM (aipi)` group band is visible (the phase-171 band contract); a row labelled **Thinking level** (`BOR_LLM_THINKING_LEVEL` env-var display, the catalogue pattern) is present, its input is a **free-form text** input (`type="text"` — NOT `number`/`select`, the A4 contract), shows `low` with the `env` badge. Assert the `llm_chat_model` row's input is the same `type="text"` control (the "like chat model" reference, in one assertion pair).
|
||||
- `test_saved_level_persists_without_restart` — type `high` into the row (free-form: also type a custom value `think-soft` once, to prove no allowlist — A4), Save (the phase-168 §7.4 Save lifecycle: dirty → save → `DB` badge), the effective value reads `think-soft`; reload `/settings.html` → still `think-soft` with the `DB` badge (no restart, the phase-168 contract).
|
||||
- `test_reset_returns_to_env_default` — the row's Reset (per-row `DELETE`) → the value reads `low` again with the `env` badge (the overlay reset, task-02's contract, in the browser).
|
||||
2. Run the **full gates** in order, all from the repo root:
|
||||
- `uv run pytest -v` (unit + integration) and `uv run pytest --cov=app --cov-report=term-missing` → `app/` coverage **>90%**.
|
||||
- `uv run ruff check . && uv run pyright`.
|
||||
- `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` → green **in isolation**.
|
||||
- Regression, in isolation: `uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov` (phase 168) and `uv run pytest tests/e2e/test_admin_settings_layout.py -v --no-cov` (phase 171) → both green.
|
||||
3. The **atomic commit** (AGENTS rule 8 — one commit per phase, Conventional Commits, `--no-gpg-sign`): the whole phase — `app/config.py`, `app/core/settings_catalog.py`, `app/rag/llm.py`, `app/api/chat.py`, `.env.example`, the four test files, and `.agents/phases/todo/172_thinking_level/` → moved to `.agents/phases/complete/172_thinking_level/` in the SAME commit. Message: `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort` + the `LLM-Generated: true` trailer. The pre-existing dirty `TODO.md` (the now-converted entry, cleared in Phase 4 of this conversion) rides this commit ONLY if the conversion commit has not already committed it — check `git status` first; the conversion commit is the one that clears TODO.md (see the skill's Phase 4), so the phase commit should be the phase files + code only.
|
||||
|
||||
- ASSUMPTION: A5 (phase-locked) — no `settings.js` / `styles.css` change; the row renders from the catalogue. A6 (phase-locked) — the per-turn-line assertion lives in the unit/integration layer (task 04), not the E2E (the mock LLM turn does not exercise the wire; the E2E proves the settings surface only).
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `tests/e2e/test_thinking_level.py` green **in isolation**; the phase-168 + phase-171 suites green in isolation (regression).
|
||||
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green in isolation (the three contracts).
|
||||
- [ ] `uv run pytest` green; `app/` coverage >90%; ruff + pyright clean.
|
||||
- [ ] `test_admin_settings_panel.py` + `test_admin_settings_layout.py` green in isolation (no regression in the settings machinery).
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit with the `LLM-Generated: true` trailer; the phase dir moved to `.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user