diff --git a/.agents/phases/todo/172_thinking_level/00_phase.md b/.agents/phases/todo/172_thinking_level/00_phase.md new file mode 100644 index 0000000..42d2ee1 --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/00_phase.md @@ -0,0 +1,50 @@ +# Phase 172 — Thinking level: a pi-style, operator-tunable LLM reasoning knob + +**Source:** `TODO.md` L3 — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings." +**Story:** n/a (TODO-derived, owner roadmap confirmation 2026-10-10) +**Context:** `app/config.py` — the LLM section (L97–125: `llm_base_url`, `llm_api_key`, `llm_chat_model` L101, `llm_embed_model`, `llm_summary_model` L107, `stream_thinking` L112, `llm_retries` / `llm_retry_delay` / `llm_timeout` L122 — the house pattern this setting joins), `.env.example` L57–65 (the LLM block — `BOR_LLM_CHAT_MODEL=turbo` L60), `app/core/settings_catalog.py` L137–161 (the `LLM (aipi)` catalogue group; the completeness pin in `tests/unit/test_settings_catalog.py` — `set(CATALOG) == set(Settings.model_fields)` — makes the new field a build failure until it has a `SettingMeta`), `app/core/settings_overlay.py` (the generic DB-over-env overlay — a plain `str` field rides it with zero extra code: `PUT` stores the raw string, `invalidate_settings()` re-reads), `app/rag/llm.py` — `LLMClient.chat()` (L446, the one-shot `chat.completions.create` L416, `temperature=0.2`, 2048 tokens) and `LLMClient.chat_stream()` (L511, the kwargs dict L592–600: `model` / `messages` / `temperature=0.4` / `max_tokens` / `stream`), `app/api/chat.py` — the per-turn log line (L1268–1291: `…thinking_chars=%d tool_calls=%d…`), `frontend/assets/settings.js` (the phase-168 generated panel — **no code change here**: a new `str` catalogue entry renders as a free-form text row automatically, exactly like `llm_chat_model`), `tests/unit/test_config.py`, `tests/unit/test_llm_client.py` (the mocked-SDK-client family), `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` (the set/reset/live-apply contract), `tests/e2e/test_admin_settings_panel.py` (the phase-168 suite — the pattern to copy). + +## Objective +The operator can pick the model's thinking level — pi-style, free-form text, default `low` — from `.env` (`BOR_LLM_THINKING_LEVEL`) or the admin Settings panel, and the value rides every chat-completions request to aipi as the OpenAI-standard `reasoning_effort` parameter, so the same knob works as the model's reasoning support evolves. The empty value is the off switch (the parameter is not sent at all). + +## Design (shared by all tasks — the executor reads this, not the chat) +- **D1 — one `Settings` field (task 01).** `llm_thinking_level: str = "low"` in the LLM section of `app/config.py` (placed directly after the `llm_summary_model` block, before `stream_thinking` — model-identity settings stay together). Free-form, exactly like `llm_chat_model` (A4: **no allowlist, no validator** — any non-empty text is accepted; the endpoint is the source of truth). The docstring names the env var (`BOR_LLM_THINKING_LEVEL`), the default, the wire mapping (D2) and the empty-off rule (D3). `.env.example` gains `BOR_LLM_THINKING_LEVEL=low` directly under `BOR_LLM_CHAT_MODEL=turbo` (L60) with a one-line comment (the house style of L61–63). +- **D2 — the wire (task 03).** The value is sent as a **top-level chat-completions parameter** `reasoning_effort` via the OpenAI SDK's `extra_body` (`extra_body={"reasoning_effort": ""}` — the SDK merges `extra_body` into the request JSON body). Applied in BOTH `LLMClient.chat()` (the one-shot: document summaries, KB overview) and `LLMClient.chat_stream()` (the chat turn) — A3. The embeddings path (`_post_embeddings` / `embed` / `embed_one`) is **untouched**. `ASSUMPTION: A1 — aipi accepts the OpenAI-standard `reasoning_effort` top-level parameter (pi's `reasoningEffort`); if aipi exposes a different key (e.g. `chat_template_kwargs`), only task 03's key changes — the rest of the phase is key-agnostic by design.** +- **D3 — the off switch (tasks 03/04).** The **trimmed** value decides: non-empty → the parameter is sent with the trimmed value; empty/whitespace → `extra_body` is absent from the request (byte-identical to today's payload). Nothing is sent as `"off"` or `null` — absence is the off. +- **D4 — the admin row (task 02).** One `SettingMeta("llm_thinking_level", "LLM (aipi)", "Thinking level")` entry in `CATALOG` (placed after `llm_summary_model` L150–151, before `llm_retries` L156). Nothing else: the panel (`settings.js`), the `GET` catalogue, the `PUT` / `DELETE` routes, and the DB-over-env overlay are all **generic over the catalogue** — the row, its free-form text input, its save/reset and its live-apply fall out with zero extra code (A5). The completeness pin is the enforcement: a missing entry fails the build. +- **D5 — the per-turn line (task 04).** The per-turn log line (PLAN §9, `app/api/chat.py` L1268) gains `thinking=` immediately before the existing `thinking_chars=%d` — the effective (overlay-resolved) value, trimmed, or `-` when empty, so "which level did this turn run at" stays one `grep` away (AGENTS rule 10). No `query_log` schema change (the table is retrieval-scoped; the line is the home of turn knobs). +- **NOT touched:** `frontend/assets/settings.js` (generated from the catalogue), `frontend/assets/styles.css`, the `app_settings` table shape, `app/core/settings_overlay.py` (generic), the embeddings path, `stream_thinking` (the phase-17 kill-switch for `thinking` SSE events — different concern, stays as-is), every other setting. + +## Dependencies +— (none: the app is fully functional at the head of `complete/`; this phase builds on the committed phase-168/171 settings machinery) + +## Tasks +1. `01_setting_and_env.md` — `Settings.llm_thinking_level: str = "low"` + the `CATALOG` row (the completeness pin keeps the build green) + `BOR_LLM_THINKING_LEVEL=low` in `.env.example`; the config/catalog unit tests. +2. `02_admin_row_contract.md` — the integration set / reset / live-apply contract for the new key (stored raw, live without restart, empty storable as the off switch). +3. `03_llm_extra_body_passthrough.md` — `extra_body={"reasoning_effort": …}` on `chat()` + `chat_stream()` when trimmed-non-empty; absent when empty; the mocked-client unit tests. +4. `04_per_turn_log_field.md` — `thinking=` (or `-`) on the per-turn log line; the log-line test. +5. `05_e2e_admin_settings_row.md` — `tests/e2e/test_thinking_level.py` (the phase's dedicated Playwright suite, AGENTS rule 4), the regression set, the full gates, the atomic commit. + +## Testing & Quality +- **Unit:** `tests/unit/test_config.py` — the default is `low`, the env override works, an empty env value yields an empty string (no coercion, no allowlist — D3/A4). `tests/unit/test_settings_catalog.py` — the completeness pin stays green (the new field has its `SettingMeta`; group/label/flags correct). `tests/unit/test_llm_client.py` — `chat()` and `chat_stream()` send `extra_body={"reasoning_effort": }` when the setting is non-empty, send **no** `extra_body` when it is empty/whitespace, and the embeddings path is unaffected (the mocked-SDK-client family). +- **Integration:** `tests/integration/test_admin_settings_api.py` / `test_admin_settings_live.py` — the row appears in `GET /api/admin/settings` (group, `str`, effective `low`); `PUT` stores the raw string, the empty value is storable (the off switch), the overlay resolves live without a restart; `DELETE` resets to env. +- **E2E (AGENTS rule 4, task 05):** `tests/e2e/test_thinking_level.py`, run **in isolation** — the row is visible in the `LLM (aipi)` group with default `low`, is a free-form text input (type `high`, save, survives reload), reset-to-env restores `low`, and the `test_admin_settings_panel.py` regression set stays green. +- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`). +- **Lint/types:** `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] `uv run pytest -k "config and thinking" -v` shows the default `low` + the env override; `.env.example` contains `BOR_LLM_THINKING_LEVEL=low` under `BOR_LLM_CHAT_MODEL`. +- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, default `low`); `PUT` / `DELETE` round-trip per the overlay contract. +- [ ] With the setting non-empty, a `chat_stream()` / `chat()` request carries `reasoning_effort` (unit-asserted via the mocked client); with it empty, the request kwargs contain no `extra_body` (byte-identical to today). +- [ ] The per-turn log line carries `thinking=` (`-` when empty) next to `thinking_chars`. +- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean. +- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green **in isolation**; `test_admin_settings_panel.py` + `test_admin_settings_layout.py` (phase 171) green in isolation. +- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort`); the phase dir moves to `.agents/phases/complete/`. + +## Locked decisions (owner-confirmed 2026-10-10, roadmap confirmation) +- **A1 — the wire key is `reasoning_effort`** (the OpenAI-standard top-level chat-completions parameter, pi's `reasoningEffort`), sent via the SDK's `extra_body`. If aipi exposes a different key, only task 03 changes. +- **A2 — the off switch is absence:** empty/whitespace value → the parameter is not sent at all; the default `low` is always sent. +- **A3 — scope:** the parameter rides BOTH `chat_stream()` (the chat turn) and `chat()` (the one-shot summaries / KB overview); the embeddings path is untouched. +- **A4 — free-form, no allowlist:** any non-empty text is accepted (like `llm_chat_model`); no validator, no value coercion. +- **A5 — no frontend code change:** the panel is generated from the catalogue; the row appears automatically as a free-form text input. +- **A6 — the per-turn line** gains `thinking=` (`-` when empty) immediately before `thinking_chars`; no `query_log` schema change. diff --git a/.agents/phases/todo/172_thinking_level/01_setting_and_env.md b/.agents/phases/todo/172_thinking_level/01_setting_and_env.md new file mode 100644 index 0000000..11acaf9 --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/01_setting_and_env.md @@ -0,0 +1,49 @@ +# Task 01 — The `llm_thinking_level` setting + the catalogue row + `.env.example` + +**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings." +**Story:** n/a + +## Objective +The new setting exists in `app/config.py` (default `low`, free-form like `llm_chat_model`), in the admin catalogue (so the build's completeness pin stays green — a field without a `SettingMeta` fails the build, and the per-task gate runs the full suite), and in `.env.example`, so `BOR_LLM_THINKING_LEVEL` is a first-class, documented knob before anything consumes it. + +## Work +1. `app/config.py` — in the LLM section, **directly after** the `llm_summary_model` block (L103–107) and **before** `stream_thinking` (L108), add ONE field, house docstring style (env var named, default, wire mapping, off rule): + ```python + #: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) — + #: pi-style reasoning effort, free-form text exactly like + #: ``llm_chat_model`` (no allowlist — the endpoint is the source of + #: truth). Default ``low``. Rides every chat-completions request to + #: aipi as the top-level ``reasoning_effort`` parameter + #: (``app/rag/llm.py``); the EMPTY value is the off switch — the + #: parameter is not sent at all. + llm_thinking_level: str = "low" + ``` + No validator, no coercion — a plain `str` (ASSUMPTION A4, phase-locked). Verify by reading: `llm_chat_model` (L101) keeps its shape byte-identical. +2. `.env.example` — in the LLM block (L57–65), directly under `BOR_LLM_CHAT_MODEL=turbo` (L60), add: + ``` + BOR_LLM_THINKING_LEVEL=low # the model's thinking level (pi-style; free-form, rides reasoning_effort); empty = the parameter is not sent + ``` + (one-line comment, the house style of L61–63; `#`-prefixed optional vars stay commented — this one is set, like L60). +3. `app/core/settings_catalog.py` — in the `LLM (aipi)` catalogue group, **directly after** the `llm_summary_model` entry (L150–151) and **before** `llm_retries` (L156), add ONE entry: + ```python + "llm_thinking_level": SettingMeta( + "llm_thinking_level", "LLM (aipi)", "Thinking level" + ), + ``` + Nothing else — `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` are DERIVED from `CATALOG` (module docstring), so no other list changes. The row is plain: not `secret`, not `restart_required` (the LLM client reads the setting per request via `get_settings()` — live), overridable (the overlay is generic over `str` fields, `app/core/settings_overlay.py`). +4. `tests/unit/test_settings_catalog.py` — the completeness pin (`set(CATALOG) == set(Settings.model_fields)`) must stay green with the new pair; add the house-style assertion that `CATALOG["llm_thinking_level"]` sits in group `LLM (aipi)` with label `Thinking level` and is **not** in `SECRETS` / `RESTART_REQUIRED` / `NON_OVERRIDABLE` (the derived sets). +5. `tests/unit/test_config.py` — add the house-style cases: (a) the DEFAULT is `"low"` (no env set); (b) the env override — `BOR_LLM_THINKING_LEVEL=high` resolves to `"high"`; (c) the empty env value (`BOR_LLM_THINKING_LEVEL=`) resolves to `""` (no coercion to a default, no allowlist — the off switch stays expressible). + +- ASSUMPTION: A4 (phase-locked) — free-form text, no allowlist/validator; A2 (phase-locked) — empty = off switch, so the setting must NOT strip/normalize away the empty value at load (it is trimmed only at the wire, task 03); A5 (phase-locked) — the panel row falls out of the catalogue with zero `settings.js` code. + +## Testing & Quality +- Unit: `tests/unit/test_config.py` + `tests/unit/test_settings_catalog.py` green (`uv run pytest tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov`). +- Coverage: **>90%** on `app/` (the gate must stay green). +- Lint/types: `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] `uv run pytest -k "thinking" tests/unit/test_config.py tests/unit/test_settings_catalog.py -v --no-cov` green: default `low`, env override `high`, empty stays `""`, the completeness pin holds with the new pair. +- [ ] `grep -n "BOR_LLM_THINKING_LEVEL" .env.example` shows the line under `BOR_LLM_CHAT_MODEL=turbo`. +- [ ] `GET /api/admin/settings` (admin) lists `llm_thinking_level` in the `LLM (aipi)` group (`str`, effective `low`) — the pin + a quick curl, or the task-02 integration test. +- [ ] `uv run pytest` green; ruff + pyright clean. +- [ ] no behavior change in completed work (no other `Settings` field touched; `settings.js` untouched). diff --git a/.agents/phases/todo/172_thinking_level/02_admin_row_contract.md b/.agents/phases/todo/172_thinking_level/02_admin_row_contract.md new file mode 100644 index 0000000..cf685e0 --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/02_admin_row_contract.md @@ -0,0 +1,24 @@ +# Task 02 — The admin row's API contract (set / reset / live-apply for `llm_thinking_level`) + +**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. … Update .env and admin settings." +**Story:** n/a + +## Objective +Prove, at the API/overlay level, that the new key behaves exactly like `llm_chat_model`: it appears in the catalogue, `PUT` stores the raw string and the overlay resolves it live (no restart), `DELETE` resets to env, and the effective value the LLM layer will read follows the overlay. + +## Work +1. `tests/integration/test_admin_settings_api.py` — extend the existing key-parameterized patterns (read how the phase-168 suite picks a `str` key, e.g. `llm_chat_model`) with the house-style cases for `llm_thinking_level`: (a) `GET /api/admin/settings` → the item is present, group `LLM (aipi)`, `type str`, effective value `low` (no row in `app_settings`), `env_var` `BOR_LLM_THINKING_LEVEL`; (b) `PUT` with `{"value": "high"}` → 200, the row is stored **raw** in `app_settings` (env-native string form, D1 of the overlay), the response reports the new effective value; (c) `PUT` with an empty string `""` → 200 (the off switch is storable — the API stores as-is, no allowlist); (d) `DELETE /api/admin/settings/llm_thinking_level` → the row is gone, effective value back to the env default `low`; (e) the non-admin 403 gate still applies (the phase-168 router dependency — assert once, mirroring the existing suite). +2. `tests/integration/test_admin_settings_live.py` — add the live-apply case in the house pattern: `PUT llm_thinking_level=high` → `get_settings().llm_thinking_level` is `"high"` **without** a restart (the `invalidate_settings()` path); `DELETE` → back to `"low"`. (This is the proof the LLM layer — which reads `get_settings()` per request, task 03 — picks the override up live.) +3. If the phase-168 suites parameterize over `CATALOG` keys generically (check `tests/integration/test_admin_settings_api.py` first — read it, do not assume): the new key may already be covered by the parameterization; in that case add only the empty-string (c) case and the live-apply (2) case, and note it in the test's comment. + +- ASSUMPTION: A4 (phase-locked) — the empty string is storable (the off switch must be settable from the panel, not only via `.env`); the API "stores as-is" per the phase-168 module docstring — no value validation to add. + +## Testing & Quality +- Integration: `tests/integration/test_admin_settings_api.py` + `test_admin_settings_live.py` green (`uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov`). +- Coverage: **>90%** on `app/` (unchanged `app/` code — the gate must stay green, not drop). +- Lint/types: `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] `GET /api/admin/settings` (admin) lists the row (`LLM (aipi)`, `str`, `low`); `PUT high` → live effective `high`; `PUT ""` → live effective `""`; `DELETE` → back to `low` (the integration cases green). +- [ ] `uv run pytest tests/integration/test_admin_settings_api.py tests/integration/test_admin_settings_live.py -v --no-cov` green; full suite green. +- [ ] ruff + pyright clean; no `app/` code changed by this task. diff --git a/.agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md b/.agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md new file mode 100644 index 0000000..fdcc5a0 --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/03_llm_extra_body_passthrough.md @@ -0,0 +1,46 @@ +# Task 03 — The thinking level rides every chat-completions request (`reasoning_effort` via `extra_body`) + +**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …" +**Story:** n/a + +## Objective +`LLMClient` sends the operator's thinking level to aipi on every chat-completions request — `chat()` (the one-shot) and `chat_stream()` (the chat turn) — as the top-level `reasoning_effort` parameter via the SDK's `extra_body`; when the (trimmed) value is empty the request is byte-identical to today's (no `extra_body` at all). + +## Work +1. `app/rag/llm.py` — ONE small helper (module-level, near the top, after the imports): + ```python + def _thinking_extra_body(settings: Settings) -> dict[str, str] | None: + """The phase-172 thinking level as the chat-completions extra body. + + Non-empty (trimmed) value → ``{"reasoning_effort": }`` (the + OpenAI-standard top-level parameter, A1); empty/whitespace → ``None`` + (the off switch — the parameter is ABSENT from the request, A2). + """ + level = settings.llm_thinking_level.strip() + return {"reasoning_effort": level} if level else None + ``` + Then in **BOTH** request sites, add the parameter only when present: + - `chat()` (L446, the `await self._client.chat.completions.create(...)` at L416–421): compute `extra = _thinking_extra_body(self.settings)` and pass the kwarg **only when present** — `**({"extra_body": extra} if extra is not None else {})` spread into the `create(...)` call, so `extra_body` is ABSENT (never `None`) when off. + - `chat_stream()` (L511, the kwargs dict L592–600): same pattern — after the dict is built (and after the `if tools is not None:` block, L601–602), `if (extra := _thinking_extra_body(self.settings)) is not None: kwargs["extra_body"] = extra`. + The value is the **trimmed** setting (`.strip()` in the helper) — the raw stored string is never sent with surrounding whitespace. The embeddings path (`_post_embeddings` L256, `embed` L346, `embed_one` L400) is **untouched** (A3) — verify by reading: no `extra_body` appears anywhere in the embedding code. +2. `app/rag/llm.py` — update the module docstring (L1–L30) with ONE sentence: the thinking level (phase 172, `BOR_LLM_THINKING_LEVEL`) rides chat completions as `reasoning_effort`; empty = absent. (The house style: wire facts live in the module docstring, cf. the `reasoning_content` note at L10–12.) +3. `tests/unit/test_llm_client.py` — extend the mocked-SDK-client family (read the existing `chat` / `chat_stream` cases first and mirror their mocking): + - `chat()` with `llm_thinking_level="low"` → `create` is awaited with `extra_body={"reasoning_effort": "low"}` (assert on the call kwargs). + - `chat()` with `llm_thinking_level=" "` (whitespace) → `extra_body` is **absent from the call kwargs** (assert `"extra_body" not in kwargs`). + - `chat_stream()` with `"high"` → same `extra_body` assertion on the `create` call. + - `chat_stream()` with `""` → `extra_body` absent (the off switch — the kwargs dict is byte-identical in shape to pre-phase-172). + - one `embed()` / `embed_one()` case with the level set → the HTTP POST body (the mocked transport) contains **no** `reasoning_effort` (the embeddings path is untouched, A3). + - the TRIM case: `llm_thinking_level="low "` (trailing spaces — reachable via a hand-edited `.env`) → the sent value is `"low"`. + +- ASSUMPTION: A1 (phase-locked) — the wire key is the OpenAI-standard top-level `reasoning_effort` via `extra_body`; if aipi exposes a different key, ONLY this task's helper changes (the setting, the row, the log, the E2E are key-agnostic). A2 (phase-locked) — absence is the off switch (never `extra_body=None`, never `"off"`). A3 (phase-locked) — `chat()` + `chat_stream()` only; embeddings untouched. + +## Testing & Quality +- Unit: `tests/unit/test_llm_client.py` green (`uv run pytest tests/unit/test_llm_client.py -v --no-cov`). +- Coverage: **>90%** on `app/` (the new helper is fully covered by the cases above). +- Lint/types: `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] With the setting non-empty, a `chat()` / `chat_stream()` request carries `extra_body={"reasoning_effort": }` (the unit cases green). +- [ ] With it empty/whitespace, `extra_body` is absent from the request kwargs (byte-identical shape to today); the embeddings POST body never contains `reasoning_effort`. +- [ ] `uv run pytest` green; ruff + pyright clean. +- [ ] no behavior change in completed work (temperature / max_tokens / tools / stream kwargs untouched; the `stream_thinking` kill-switch untouched). diff --git a/.agents/phases/todo/172_thinking_level/04_per_turn_log_field.md b/.agents/phases/todo/172_thinking_level/04_per_turn_log_field.md new file mode 100644 index 0000000..437dbd7 --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/04_per_turn_log_field.md @@ -0,0 +1,24 @@ +# Task 04 — The per-turn log line records the effective thinking level + +**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "Brain of Reese needs to support thinking levels like pi does. …" +**Story:** n/a + +## Objective +The per-turn log line (PLAN §9, AGENTS rule 10) records the effective thinking level of the turn, so "which level did this turn run at" is one `grep` away, next to the `thinking_chars` it governs. + +## Work +1. `app/api/chat.py` — the per-turn `logger.info(...)` (L1268–1291): insert `thinking=%s` **immediately before** the existing `thinking_chars=%d` in the format string, with the argument `settings.llm_thinking_level.strip() or "-"` (the effective overlay-resolved value — `settings` is already in scope, the line already logs `settings.relevance_threshold` L1280; trimmed, `-` when empty, per A6). No reordering of the existing fields; the `query_log` row (L1250–1260) is **untouched** (A6: the table is retrieval-scoped, the line is the home of turn knobs). +2. `app/api/chat.py` — the module docstring: the existing per-turn-line notes (L14, L46, L64, L78, L141, L161) set the house pattern — add ONE line where the log line's fields are described: the per-turn log line records `thinking=` (the effective `BOR_LLM_THINKING_LEVEL`, `-` when empty) before `thinking_chars` (phase 172). +3. The existing per-turn-line test (find it: `grep -rn "thinking_chars" tests/` — the house pattern pins the line's fields in one place; extend that test, do not invent a new file): assert the new `thinking=` field is present, carries the effective value in the non-empty case, and is `-` when the setting is empty. + +- ASSUMPTION: A6 (phase-locked) — `thinking=` or `-`, placed immediately before `thinking_chars`; no `query_log` schema change, no new `Settings` read (the per-turn `settings` object is already resolved through the overlay). + +## Testing & Quality +- Unit/integration: the per-turn-line test (above) green; `uv run pytest` green (the SSE turn machinery is untouched — the change is one format field + one argument). +- Coverage: **>90%** on `app/` (the gate must stay green, not drop). +- Lint/types: `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] A test turn's per-turn log line (the test's captured log) contains `thinking=low` (default) and `-` when the setting is empty; `thinking_chars` stays in its position after it. +- [ ] `grep -rn "thinking=" app/api/chat.py` shows the field exactly once in the format string. +- [ ] `uv run pytest` green; ruff + pyright clean; no `query_log` schema change. diff --git a/.agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md b/.agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md new file mode 100644 index 0000000..8774a0e --- /dev/null +++ b/.agents/phases/todo/172_thinking_level/05_e2e_admin_settings_row.md @@ -0,0 +1,32 @@ +# Task 05 — The phase's dedicated Playwright suite (AGENTS rule 4) + the full gates + the atomic commit + +**Phase:** `172_thinking_level` · **Source:** `TODO.md:3` — "… should be a free-form text entry like chat model. Default should be 'low'. Update .env and admin settings." +**Story:** n/a + +## Objective +`tests/e2e/test_thinking_level.py` proves the owner's acceptance in a real browser — the Thinking level row exists in the admin Settings panel as a **free-form text** entry (like the chat model's), defaults to `low`, saves and persists, and resets to env — run **in isolation**; then the full gates, then ONE atomic commit. + +## Work +1. `tests/e2e/test_thinking_level.py` — the phase's dedicated suite, run in isolation (DB must be up: `podman compose up -d db`; `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov`). **Copy the harness from `tests/e2e/test_admin_settings_panel.py`** (read it first, then mirror: the `_boot_app`/`app_server`/`app_url` fixtures with the mock LLM, the `_clean` autouse fixture (its `app_settings` truncation pattern — the new key's rows must be wiped between tests), the `_cookies`/`_admin_client` helpers, the settings-page nav + Save/Reset drives at L561ff). Three tests: + - `test_row_renders_free_form_with_default_low` — admin opens `/settings.html`; the `LLM (aipi)` group band is visible (the phase-171 band contract); a row labelled **Thinking level** (`BOR_LLM_THINKING_LEVEL` env-var display, the catalogue pattern) is present, its input is a **free-form text** input (`type="text"` — NOT `number`/`select`, the A4 contract), shows `low` with the `env` badge. Assert the `llm_chat_model` row's input is the same `type="text"` control (the "like chat model" reference, in one assertion pair). + - `test_saved_level_persists_without_restart` — type `high` into the row (free-form: also type a custom value `think-soft` once, to prove no allowlist — A4), Save (the phase-168 §7.4 Save lifecycle: dirty → save → `DB` badge), the effective value reads `think-soft`; reload `/settings.html` → still `think-soft` with the `DB` badge (no restart, the phase-168 contract). + - `test_reset_returns_to_env_default` — the row's Reset (per-row `DELETE`) → the value reads `low` again with the `env` badge (the overlay reset, task-02's contract, in the browser). +2. Run the **full gates** in order, all from the repo root: + - `uv run pytest -v` (unit + integration) and `uv run pytest --cov=app --cov-report=term-missing` → `app/` coverage **>90%**. + - `uv run ruff check . && uv run pyright`. + - `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` → green **in isolation**. + - Regression, in isolation: `uv run pytest tests/e2e/test_admin_settings_panel.py -v --no-cov` (phase 168) and `uv run pytest tests/e2e/test_admin_settings_layout.py -v --no-cov` (phase 171) → both green. +3. The **atomic commit** (AGENTS rule 8 — one commit per phase, Conventional Commits, `--no-gpg-sign`): the whole phase — `app/config.py`, `app/core/settings_catalog.py`, `app/rag/llm.py`, `app/api/chat.py`, `.env.example`, the four test files, and `.agents/phases/todo/172_thinking_level/` → moved to `.agents/phases/complete/172_thinking_level/` in the SAME commit. Message: `feat(llm): operator-tunable thinking level (BOR_LLM_THINKING_LEVEL, default low) sent as reasoning_effort` + the `LLM-Generated: true` trailer. The pre-existing dirty `TODO.md` (the now-converted entry, cleared in Phase 4 of this conversion) rides this commit ONLY if the conversion commit has not already committed it — check `git status` first; the conversion commit is the one that clears TODO.md (see the skill's Phase 4), so the phase commit should be the phase files + code only. + +- ASSUMPTION: A5 (phase-locked) — no `settings.js` / `styles.css` change; the row renders from the catalogue. A6 (phase-locked) — the per-turn-line assertion lives in the unit/integration layer (task 04), not the E2E (the mock LLM turn does not exercise the wire; the E2E proves the settings surface only). + +## Testing & Quality +- E2E: `tests/e2e/test_thinking_level.py` green **in isolation**; the phase-168 + phase-171 suites green in isolation (regression). +- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`). +- Lint/types: `uv run ruff check . && uv run pyright`. + +## Completion Criteria +- [ ] `uv run pytest tests/e2e/test_thinking_level.py -v --no-cov` green in isolation (the three contracts). +- [ ] `uv run pytest` green; `app/` coverage >90%; ruff + pyright clean. +- [ ] `test_admin_settings_panel.py` + `test_admin_settings_layout.py` green in isolation (no regression in the settings machinery). +- [ ] One atomic `--no-gpg-sign` Conventional Commit with the `LLM-Generated: true` trailer; the phase dir moved to `.agents/phases/complete/`.