phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Build and Push Containers / build-and-push-db (push) Successful in 14s

## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
2026-10-11 23:38:18 -04:00
parent c6223c93dc
commit 149e753952
45 changed files with 1947 additions and 544 deletions
+8 -3
View File
@@ -15,8 +15,9 @@ turn's thinking is counted in the per-turn log line
(``thinking_chars=N``); ``BOR_STREAM_THINKING=0`` suppresses the
``thinking`` frames (the pieces are still counted). The per-turn log
line records ``thinking=<level>`` (the effective
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6).
``BOR_LLM_CHAT_THINKING_LEVEL``, trimmed, ``-`` when empty) before
``thinking_chars`` (phase 172, A6; phase 178, A5 — the **chat**
level; the one-shot summary level never appears on the line).
Honesty gate (A8, revised 2026-09-14): LOW — deflection — only when
``best_cosine < BOR_RELEVANCE_THRESHOLD`` **and** (``fts_hits == 0``
@@ -1286,7 +1287,11 @@ async def chat(
settings.relevance_threshold,
plan.deflected,
source_paths,
settings.llm_thinking_level.strip() or "-", # phase 172 (A6)
# phase 178 (A5): the effective chat thinking
# level, trimmed, `-` when empty — the level this
# turn ran at; the summary level is a one-shot
# concern and gains no field here.
settings.llm_chat_thinking_level.strip() or "-",
thinking_chars,
holder.tool_calls,
holder.rounds,
+15 -6
View File
@@ -105,14 +105,23 @@ class Settings(BaseSettings):
#: Served by the same OpenAI-compatible endpoint — no new model
#: management. Called via ``LLMClient.chat()``.
llm_summary_model: str = "lite"
#: The model's thinking level (phase 172, ``BOR_LLM_THINKING_LEVEL``) —
#: The chat turn's thinking level (phase 178,
#: ``BOR_LLM_CHAT_THINKING_LEVEL``; the phase-172 knob, renamed) —
#: pi-style reasoning effort, free-form text exactly like
#: ``llm_chat_model`` (no allowlist — the endpoint is the source of
#: truth). Default ``low``. Rides every chat-completions request to
#: aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py``); the EMPTY value is the off switch — the
#: parameter is not sent at all.
llm_thinking_level: str = "low"
#: truth). Default ``low``. Rides the streaming chat-completions
#: requests to aipi as the top-level ``reasoning_effort`` parameter
#: (``app/rag/llm.py`` ``chat_stream``); the EMPTY value is the off
#: switch — the parameter is not sent at all.
llm_chat_thinking_level: str = "low"
#: The one-shot summary thinking level (phase 178,
#: ``BOR_LLM_SUMMARY_THINKING_LEVEL``) — rides the one-shot
#: ``chat()`` calls (document summaries, folder summaries, KB
#: overview, the ``check_models`` probe) as ``reasoning_effort``.
#: Default EMPTY = the off switch (phase-172 A2): the parameter is
#: not sent at all; the TODO's "none" means absence, never a
#: literal "none" string.
llm_summary_thinking_level: str = ""
#: Operator kill-switch for the ``thinking`` SSE events (phase 17,
#: ``BOR_STREAM_THINKING``; ``0``/``false`` → off). When off, thinking
#: pieces are still counted for the per-turn log line but never
+5 -2
View File
@@ -151,8 +151,11 @@ CATALOG: dict[str, SettingMeta] = {
"llm_summary_model": SettingMeta(
"llm_summary_model", "LLM (aipi)", "LLM summary model"
),
"llm_thinking_level": SettingMeta(
"llm_thinking_level", "LLM (aipi)", "Thinking level"
"llm_chat_thinking_level": SettingMeta(
"llm_chat_thinking_level", "LLM (aipi)", "Chat thinking level"
),
"llm_summary_thinking_level": SettingMeta(
"llm_summary_thinking_level", "LLM (aipi)", "Summary thinking level"
),
"stream_thinking": SettingMeta(
"stream_thinking", "LLM (aipi)", "Stream thinking"
+30 -18
View File
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty). The operator's thinking level
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
as the top-level ``reasoning_effort`` parameter (via the SDK's
``extra_body``); an empty value is the off switch — the parameter is
absent, the request is byte-identical to pre-phase-172, and the
embeddings path never carries it.
(an answer can in principle be empty). The operator's thinking levels
(phase 178) ride chat-completions requests as the top-level
``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
default ``low``; the phase-172 knob, renamed) rides streaming
``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
default EMPTY = off) rides ``chat`` — document summaries, folder
summaries, the KB overview, and the ``check_models`` probe (a
reachability ping, off by default, no special-casing). An empty value is
the off switch — the parameter is absent (never a literal ``"none"``),
the request is byte-identical to pre-phase-172, and the embeddings path
never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
def _thinking_extra_body(level: str) -> dict[str, str] | None:
"""The thinking level as the chat-completions extra body (phase 178).
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
OpenAI-standard top-level parameter, phase-172 A1); empty /
whitespace → ``None`` (the off switch — the parameter is ABSENT
from the request, phase-172 A2; never a literal ``"none"`` or
``"off"``).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
trimmed = level.strip()
return {"reasoning_effort": trimmed} if trimmed else None
class EmbeddingError(RuntimeError):
@@ -429,7 +436,7 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else.
"""
extra = _thinking_extra_body(self.settings)
extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {}
)
@@ -440,8 +447,11 @@ class LLMClient:
temperature=0.2,
max_tokens=2048,
stream=False,
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
# Phase 178 (A3/A2): the **summary** level rides as
# reasoning_effort (document summaries, folder summaries,
# KB overview, the ``check_models`` probe); the key is
# absent (never None) when off — the default (empty)
# keeps one-shot requests byte-identical to pre-phase-172.
**extra_kwargs,
)
except LLMError:
@@ -622,8 +632,10 @@ class LLMClient:
}
if tools is not None:
kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None:
# Phase 172 (A1/A2): the thinking level rides as
if (
extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
) is not None:
# Phase 178 (A3/A2): the **chat** level rides as
# reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None