phase: 178_split_thinking_levels
## Phase 178 — final verification pass: 100% green
**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only
**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation
**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.
**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.
**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).
**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
+8
-3
@@ -15,8 +15,9 @@ turn's thinking is counted in the per-turn log line
|
||||
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
|
||||
``thinking`` frames (the pieces are still counted). The per-turn log
|
||||
line records ``thinking=<level>`` (the effective
|
||||
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before
|
||||
``thinking_chars`` (phase 172, A6).
|
||||
``BOR_LLM_CHAT_THINKING_LEVEL``, trimmed, ``-`` when empty) before
|
||||
``thinking_chars`` (phase 172, A6; phase 178, A5 — the **chat**
|
||||
level; the one-shot summary level never appears on the line).
|
||||
|
||||
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
|
||||
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
|
||||
@@ -1286,7 +1287,11 @@ async def chat(
|
||||
settings.relevance_threshold,
|
||||
plan.deflected,
|
||||
source_paths,
|
||||
settings.llm_thinking_level.strip() or "-", # phase 172 (A6)
|
||||
# phase 178 (A5): the effective chat thinking
|
||||
# level, trimmed, `-` when empty — the level this
|
||||
# turn ran at; the summary level is a one-shot
|
||||
# concern and gains no field here.
|
||||
settings.llm_chat_thinking_level.strip() or "-",
|
||||
thinking_chars,
|
||||
holder.tool_calls,
|
||||
holder.rounds,
|
||||
|
||||
+15
-6
@@ -105,14 +105,23 @@ class Settings(BaseSettings):
|
||||
#: Served by the same OpenAI-compatible endpoint — no new model
|
||||
#: management. Called via ``LLMClient.chat()``.
|
||||
llm_summary_model: str = "lite"
|
||||
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
|
||||
#: The chat turn's thinking level (phase 178,
|
||||
#: ``BOR_LLM_CHAT_THINKING_LEVEL``; the phase-172 knob, renamed) —
|
||||
#: pi-style reasoning effort, free-form text exactly like
|
||||
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
|
||||
#: truth). Default ``low``. Rides every chat-completions request to
|
||||
#: aipi as the top-level ``reasoning_effort`` parameter
|
||||
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
|
||||
#: parameter is not sent at all.
|
||||
llm_thinking_level: str = "low"
|
||||
#: truth). Default ``low``. Rides the streaming chat-completions
|
||||
#: requests to aipi as the top-level ``reasoning_effort`` parameter
|
||||
#: (``app/rag/llm.py`` ``chat_stream``); the EMPTY value is the off
|
||||
#: switch — the parameter is not sent at all.
|
||||
llm_chat_thinking_level: str = "low"
|
||||
#: The one-shot summary thinking level (phase 178,
|
||||
#: ``BOR_LLM_SUMMARY_THINKING_LEVEL``) — rides the one-shot
|
||||
#: ``chat()`` calls (document summaries, folder summaries, KB
|
||||
#: overview, the ``check_models`` probe) as ``reasoning_effort``.
|
||||
#: Default EMPTY = the off switch (phase-172 A2): the parameter is
|
||||
#: not sent at all; the TODO's "none" means absence, never a
|
||||
#: literal "none" string.
|
||||
llm_summary_thinking_level: str = ""
|
||||
#: Operator kill-switch for the ``thinking`` SSE events (phase 17,
|
||||
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
|
||||
#: pieces are still counted for the per-turn log line but never
|
||||
|
||||
@@ -151,8 +151,11 @@ CATALOG: dict[str, SettingMeta] = {
|
||||
"llm_summary_model": SettingMeta(
|
||||
"llm_summary_model", "LLM (aipi)", "LLM summary model"
|
||||
),
|
||||
"llm_thinking_level": SettingMeta(
|
||||
"llm_thinking_level", "LLM (aipi)", "Thinking level"
|
||||
"llm_chat_thinking_level": SettingMeta(
|
||||
"llm_chat_thinking_level", "LLM (aipi)", "Chat thinking level"
|
||||
),
|
||||
"llm_summary_thinking_level": SettingMeta(
|
||||
"llm_summary_thinking_level", "LLM (aipi)", "Summary thinking level"
|
||||
),
|
||||
"stream_thinking": SettingMeta(
|
||||
"stream_thinking", "LLM (aipi)", "Stream thinking"
|
||||
|
||||
+30
-18
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
|
||||
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
|
||||
wire convention, verified live 2026-08-23) **before** the answer's
|
||||
``delta.content`` chunks, and reasoning counts against ``max_tokens``
|
||||
(an answer can in principle be empty). The operator's thinking level
|
||||
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
|
||||
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
|
||||
as the top-level ``reasoning_effort`` parameter (via the SDK's
|
||||
``extra_body``); an empty value is the off switch — the parameter is
|
||||
absent, the request is byte-identical to pre-phase-172, and the
|
||||
embeddings path never carries it.
|
||||
(an answer can in principle be empty). The operator's thinking levels
|
||||
(phase 178) ride chat-completions requests as the top-level
|
||||
``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
|
||||
per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
|
||||
default ``low``; the phase-172 knob, renamed) rides streaming
|
||||
``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
|
||||
default EMPTY = off) rides ``chat`` — document summaries, folder
|
||||
summaries, the KB overview, and the ``check_models`` probe (a
|
||||
reachability ping, off by default, no special-casing). An empty value is
|
||||
the off switch — the parameter is absent (never a literal ``"none"``),
|
||||
the request is byte-identical to pre-phase-172, and the embeddings path
|
||||
never carries it.
|
||||
|
||||
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
|
||||
dimensions when the table is created, so a model that returns a different
|
||||
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger("app.llm")
|
||||
|
||||
|
||||
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
|
||||
"""The phase-172 thinking level as the chat-completions extra body.
|
||||
def _thinking_extra_body(level: str) -> dict[str, str] | None:
|
||||
"""The thinking level as the chat-completions extra body (phase 178).
|
||||
|
||||
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
|
||||
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
|
||||
(the off switch — the parameter is ABSENT from the request, A2).
|
||||
OpenAI-standard top-level parameter, phase-172 A1); empty /
|
||||
whitespace → ``None`` (the off switch — the parameter is ABSENT
|
||||
from the request, phase-172 A2; never a literal ``"none"`` or
|
||||
``"off"``).
|
||||
"""
|
||||
level = settings.llm_thinking_level.strip()
|
||||
return {"reasoning_effort": level} if level else None
|
||||
trimmed = level.strip()
|
||||
return {"reasoning_effort": trimmed} if trimmed else None
|
||||
|
||||
|
||||
class EmbeddingError(RuntimeError):
|
||||
@@ -429,7 +436,7 @@ class LLMClient:
|
||||
``finish_reason``) for an empty reply — the one failure class
|
||||
:meth:`chat` retries — and :class:`LLMError` for everything else.
|
||||
"""
|
||||
extra = _thinking_extra_body(self.settings)
|
||||
extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
|
||||
extra_kwargs: dict[str, Any] = (
|
||||
{"extra_body": extra} if extra is not None else {}
|
||||
)
|
||||
@@ -440,8 +447,11 @@ class LLMClient:
|
||||
temperature=0.2,
|
||||
max_tokens=2048,
|
||||
stream=False,
|
||||
# Phase 172 (A1/A2): the thinking level rides as
|
||||
# reasoning_effort; the key is absent (never None) when off.
|
||||
# Phase 178 (A3/A2): the **summary** level rides as
|
||||
# reasoning_effort (document summaries, folder summaries,
|
||||
# KB overview, the ``check_models`` probe); the key is
|
||||
# absent (never None) when off — the default (empty)
|
||||
# keeps one-shot requests byte-identical to pre-phase-172.
|
||||
**extra_kwargs,
|
||||
)
|
||||
except LLMError:
|
||||
@@ -622,8 +632,10 @@ class LLMClient:
|
||||
}
|
||||
if tools is not None:
|
||||
kwargs["tools"] = tools
|
||||
if (extra := _thinking_extra_body(self.settings)) is not None:
|
||||
# Phase 172 (A1/A2): the thinking level rides as
|
||||
if (
|
||||
extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
|
||||
) is not None:
|
||||
# Phase 178 (A3/A2): the **chat** level rides as
|
||||
# reasoning_effort; the key is absent (never None) when off.
|
||||
kwargs["extra_body"] = extra
|
||||
stream: AsyncStream[ChatCompletionChunk] | None = None
|
||||
|
||||
Reference in New Issue
Block a user