phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Build and Push Containers / build-and-push-db (push) Successful in 14s

## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
2026-10-11 23:38:18 -04:00
parent c6223c93dc
commit 149e753952
45 changed files with 1947 additions and 544 deletions
+30 -18
View File
@@ -10,13 +10,18 @@ pipeline. Chat streaming yields typed :class:`StreamPiece` values
its reasoning as ``delta.reasoning_content`` chunks (deepseek/litellm
wire convention, verified live 2026-08-23) **before** the answer's
``delta.content`` chunks, and reasoning counts against ``max_tokens``
(an answer can in principle be empty). The operator's thinking level
(phase 172, ``BOR_LLM_THINKING_LEVEL``, default ``low``) rides every
chat-completions request (one-shot ``chat`` and streaming ``chat_stream``)
as the top-level ``reasoning_effort`` parameter (via the SDK's
``extra_body``); an empty value is the off switch — the parameter is
absent, the request is byte-identical to pre-phase-172, and the
embeddings path never carries it.
(an answer can in principle be empty). The operator's thinking levels
(phase 178) ride chat-completions requests as the top-level
``reasoning_effort`` parameter (via the SDK's ``extra_body``), split
per surface (A3): the chat turn's level (``BOR_LLM_CHAT_THINKING_LEVEL``,
default ``low``; the phase-172 knob, renamed) rides streaming
``chat_stream``, and the one-shot level (``BOR_LLM_SUMMARY_THINKING_LEVEL``,
default EMPTY = off) rides ``chat`` — document summaries, folder
summaries, the KB overview, and the ``check_models`` probe (a
reachability ping, off by default, no special-casing). An empty value is
the off switch — the parameter is absent (never a literal ``"none"``),
the request is byte-identical to pre-phase-172, and the embeddings path
never carries it.
Fail-loud rule (PLAN A6): the ``chunks.embedding`` column is fixed at 768
dimensions when the table is created, so a model that returns a different
@@ -46,15 +51,17 @@ if TYPE_CHECKING:
logger = logging.getLogger("app.llm")
def _thinking_extra_body(settings: Settings) -> dict[str, str] | None:
"""The phase-172 thinking level as the chat-completions extra body.
def _thinking_extra_body(level: str) -> dict[str, str] | None:
"""The thinking level as the chat-completions extra body (phase 178).
Non-empty (trimmed) value → ``{"reasoning_effort": <value>}`` (the
OpenAI-standard top-level parameter, A1); empty/whitespace → ``None``
(the off switch — the parameter is ABSENT from the request, A2).
OpenAI-standard top-level parameter, phase-172 A1); empty /
whitespace → ``None`` (the off switch — the parameter is ABSENT
from the request, phase-172 A2; never a literal ``"none"`` or
``"off"``).
"""
level = settings.llm_thinking_level.strip()
return {"reasoning_effort": level} if level else None
trimmed = level.strip()
return {"reasoning_effort": trimmed} if trimmed else None
class EmbeddingError(RuntimeError):
@@ -429,7 +436,7 @@ class LLMClient:
``finish_reason``) for an empty reply — the one failure class
:meth:`chat` retries — and :class:`LLMError` for everything else.
"""
extra = _thinking_extra_body(self.settings)
extra = _thinking_extra_body(self.settings.llm_summary_thinking_level)
extra_kwargs: dict[str, Any] = (
{"extra_body": extra} if extra is not None else {}
)
@@ -440,8 +447,11 @@ class LLMClient:
temperature=0.2,
max_tokens=2048,
stream=False,
# Phase 172 (A1/A2): the thinking level rides as
# reasoning_effort; the key is absent (never None) when off.
# Phase 178 (A3/A2): the **summary** level rides as
# reasoning_effort (document summaries, folder summaries,
# KB overview, the ``check_models`` probe); the key is
# absent (never None) when off — the default (empty)
# keeps one-shot requests byte-identical to pre-phase-172.
**extra_kwargs,
)
except LLMError:
@@ -622,8 +632,10 @@ class LLMClient:
}
if tools is not None:
kwargs["tools"] = tools
if (extra := _thinking_extra_body(self.settings)) is not None:
# Phase 172 (A1/A2): the thinking level rides as
if (
extra := _thinking_extra_body(self.settings.llm_chat_thinking_level)
) is not None:
# Phase 178 (A3/A2): the **chat** level rides as
# reasoning_effort; the key is absent (never None) when off.
kwargs["extra_body"] = extra
stream: AsyncStream[ChatCompletionChunk] | None = None