phase: 178_split_thinking_levels
## Phase 178 — final verification pass: 100% green
**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation
**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.
**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.
**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).
**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
+30
-18
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
|
||||
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
|
||||
wire convention, verified live 2026-08-23) **before** the answer's
|
||||
``delta.content`` chunks, and reasoning counts against ``max_tokens``
|
||||
(an answer can in principle be empty). The operator's thinking level
|
||||
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
|
||||
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
|
||||
as the top-level ``reasoning_effort`` parameter (via the SDK's
|
||||
``extra_body``); an empty value is the off switch — the parameter is
|
||||
absent, the request is byte-identical to pre-phase-172, and the
|
||||
embeddings path never carries it.
|
||||
(an answer can in principle be empty). The operator's thinking levels
|
||||
(phase 178) ride chat-completions requests as the top-level
|
||||
``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
|
||||
per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
|
||||
default ``low``; the phase-172 knob, renamed) rides streaming
|
||||
``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
|
||||
default EMPTY = off) rides ``chat`` — document summaries, folder
|
||||
summaries, the KB overview, and the ``check_models`` probe (a
|
||||
reachability ping, off by default, no special-casing). An empty value is
|
||||
the off switch — the parameter is absent (never a literal ``"none"``),
|
||||
the request is byte-identical to pre-phase-172, and the embeddings path
|
||||
never carries it.
|
||||
|
||||
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
|
||||
dimensions when the table is created, so a model that returns a different
|
||||
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger("app.llm")
|
||||
|
||||
|
||||
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
|
||||
"""The phase-172 thinking level as the chat-completions extra body.
|
||||
def _thinking_extra_body(level: str) -> dict[str, str] | None:
|
||||
"""The thinking level as the chat-completions extra body (phase 178).
|
||||
|
||||
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
|
||||
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
|
||||
(the off switch — the parameter is ABSENT from the request, A2).
|
||||
OpenAI-standard top-level parameter, phase-172 A1); empty /
|
||||
whitespace → ``None`` (the off switch — the parameter is ABSENT
|
||||
from the request, phase-172 A2; never a literal ``"none"`` or
|
||||
``"off"``).
|
||||
"""
|
||||
level = settings.llm_thinking_level.strip()
|
||||
return {"reasoning_effort": level} if level else None
|
||||
trimmed = level.strip()
|
||||
return {"reasoning_effort": trimmed} if trimmed else None
|
||||
|
||||
|
||||
class EmbeddingError(RuntimeError):
|
||||
@@ -429,7 +436,7 @@ class LLMClient:
|
||||
``finish_reason``) for an empty reply — the one failure class
|
||||
:meth:`chat` retries — and :class:`LLMError` for everything else.
|
||||
"""
|
||||
extra = _thinking_extra_body(self.settings)
|
||||
extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
|
||||
extra_kwargs: dict[str, Any] = (
|
||||
{"extra_body": extra} if extra is not None else {}
|
||||
)
|
||||
@@ -440,8 +447,11 @@ class LLMClient:
|
||||
temperature=0.2,
|
||||
max_tokens=2048,
|
||||
stream=False,
|
||||
# Phase 172 (A1/A2): the thinking level rides as
|
||||
# reasoning_effort; the key is absent (never None) when off.
|
||||
# Phase 178 (A3/A2): the **summary** level rides as
|
||||
# reasoning_effort (document summaries, folder summaries,
|
||||
# KB overview, the ``check_models`` probe); the key is
|
||||
# absent (never None) when off — the default (empty)
|
||||
# keeps one-shot requests byte-identical to pre-phase-172.
|
||||
**extra_kwargs,
|
||||
)
|
||||
except LLMError:
|
||||
@@ -622,8 +632,10 @@ class LLMClient:
|
||||
}
|
||||
if tools is not None:
|
||||
kwargs["tools"] = tools
|
||||
if (extra := _thinking_extra_body(self.settings)) is not None:
|
||||
# Phase 172 (A1/A2): the thinking level rides as
|
||||
if (
|
||||
extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
|
||||
) is not None:
|
||||
# Phase 178 (A3/A2): the **chat** level rides as
|
||||
# reasoning_effort; the key is absent (never None) when off.
|
||||
kwargs["extra_body"] = extra
|
||||
stream: AsyncStream[ChatCompletionChunk] | None = None
|
||||
|
||||
Reference in New Issue
Block a user