phase: 178_split_thinking_levels
Build and Push Containers / build-and-push-app (push) Successful in 1m53s
Build and Push Containers / build-and-push-db (push) Successful in 14s

## Phase 178 — final verification pass: 100% green

**Verified (all 5 tasks complete in `complete/178_split_thinking_levels/`):**
- `config.py` rename (`llm_chat_thinking_level` default `low`) + new `llm_summary_thinking_level` (default `""`); two `CATALOG` rows; `.env.example` replaced; no stale old-key references (only intentional negative pins in tests)
- Wire split: `chat_stream()` → chat level, one-shot `chat()` → summary level (absent `extra_body` when empty); per-turn line reads chat level only

**Gates:**
- `uv run pytest -k "config and thinking" -v` → 7 passed (defaults + env overrides both knobs)
- `uv run pytest` (unit + integration) → 4170 tests, exit 0; `--cov=app` → **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright` clean
- `uv run pytest tests/e2e/test_split_thinking_levels.py -v --no-cov` → 3 passed in isolation
- `test_admin_settings_panel.py` + `test_admin_settings_layout.py` → 17 passed in isolation

**Completion criteria:** all 6 test/doc criteria ✅ (admin GET lists both keys, old key gone — integration + E2E pins); commit + phase-dir move = harness's job.

**Documentation:** README.md — both new env vars correctly documented in the LLM (aipi) table with defaults/wire-mapping/off-rule, defaults match `config.py`, all `.env.example` vars covered: verified consistent, no changes needed. AGENTS.md — all referenced files/commands verified to exist, no phase-178 impact (no UI code change, A4): verified consistent, no changes needed.

**Decisions:** none required — no live changes made (E2E booted its own app subprocess against the existing DB).

**Next pending phase:** none — 178 is the last phase in `todo/`.
This commit is contained in:
2026-10-11 23:38:18 -04:00
parent c6223c93dc
commit 149e753952
45 changed files with 1947 additions and 544 deletions
+122 -18
View File
@@ -518,11 +518,13 @@ def test_catalog_covers_every_settings_field(admin_client) -> None:
assert set(CATALOG) == set(Settings.model_fields)
def test_llm_thinking_level_row_contract(
def test_llm_chat_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the new ``llm_thinking_level`` row behaves
exactly like ``llm_chat_model`` — the machinery is generic over
"""Phase 178 (task 01; task 02 extends for the summary row): the
renamed ``llm_chat_thinking_level`` row (the phase-172 knob, the
rename per locked A1) behaves exactly like ``llm_chat_model`` —
the machinery is generic over
the catalogue (D4 / A5), so the per-key contract is: (a) present in
the GET body (group ``LLM (aipi)``, ``str``, effective ``low``
with no row in ``app_settings``, ``env_var`` named); (b) PUT stores
@@ -538,20 +540,20 @@ def test_llm_thinking_level_row_contract(
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, type, env var, effective ``low`` with NO row
# in app_settings, not secret, not restart-required (the LLM layer
# reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_thinking_level")
item = _item(r.json(), "llm_chat_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Thinking level"
assert item["label"] == "Chat thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_THINKING_LEVEL"
assert item["env_var"] == "BOR_LLM_CHAT_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == "low"
@@ -562,48 +564,150 @@ def test_llm_thinking_level_row_contract(
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": "high"})
r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_thinking_level == "high" # live, no restart
assert get_settings().llm_chat_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable (the off switch, A2/A4 — the
# API stores as-is; no allowlist, no value validation): the row
# holds the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_thinking_level", json={"value": ""})
r = admin_client.put(f"{API}/llm_chat_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT value FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_thinking_level == ""
assert get_settings().llm_chat_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``low`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_thinking_level")
r = admin_client.delete(f"{API}/llm_chat_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == "low"
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_thinking_level'")
text("SELECT count(*) FROM app_settings WHERE key = 'llm_chat_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_thinking_level", json={"value": "high"})
r = anon.put(f"{API}/llm_chat_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
def test_llm_summary_thinking_level_row_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the ``llm_summary_thinking_level`` row (the
new one-shot knob) behaves like the chat row — the machinery is
generic over the catalogue (D4 / A4) — with the summary specifics:
(a) present in the GET body (group ``LLM (aipi)``, ``str``, label
``"Summary thinking level"``, ``env_var``
``BOR_LLM_SUMMARY_THINKING_LEVEL``, effective ``""`` with no row in
``app_settings``); (b) PUT stores the RAW string in
``app_settings`` and the value resolves live WITHOUT a restart;
(c) the EMPTY value is storable (the summary off switch, A2 — the
API stores as-is, no allowlist, an empty stored value is never
rejected as "no value"); (d) DELETE resets to the env value
(``""`` — the code default); (e) the non-admin 403 gate applies;
(f) INDEPENDENCE — the two overlays resolve independently (the
whole point of the split): a saved summary value never changes the
chat row's effective value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): BOTH knobs at their code defaults, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
assert get_settings().llm_chat_thinking_level == "low"
# (a) the row: group, label, type, env var, effective ``""`` with NO
# row in app_settings, not secret, not restart-required (the LLM
# layer reads the setting per request — live).
r = admin_client.get(API)
assert r.status_code == 200, r.text
item = _item(r.json(), "llm_summary_thinking_level")
assert item["group"] == "LLM (aipi)"
assert item["label"] == "Summary thinking level"
assert item["type"] == "str"
assert item["env_var"] == "BOR_LLM_SUMMARY_THINKING_LEVEL"
assert item["secret"] is False
assert item["restart_required"] is False
assert item["value"] == ""
assert item["source"] == "default" # the env var is unset, no row
assert item["has_override"] is False
assert item["db_value"] is None
assert item["default"] == ""
# (b) PUT stores the RAW string (the overlay's D1 — env-native
# string form) and the response reports the new effective value.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "high"
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == "high"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "high" # stored raw, exactly as sent
assert get_settings().llm_summary_thinking_level == "high" # live, no restart
# (c) the EMPTY value is storable — the summary off switch (A2; the
# API stores as-is; no allowlist, no value validation): an empty
# stored value must not be rejected as "no value"; the row holds
# the empty string and the effective value is ``""``.
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": ""})
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_value"] is False # the effective value is empty
assert db.execute(
text("SELECT value FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == "" # stored as-is (the off switch)
assert get_settings().llm_summary_thinking_level == ""
# (d) DELETE: the row is gone and the effective value is back to
# the env value (``""`` — the base pinned above).
r = admin_client.delete(f"{API}/llm_summary_thinking_level")
assert r.status_code == 200, r.text
assert r.json()["value"] == ""
assert r.json()["has_override"] is False
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = 'llm_summary_thinking_level'")
).scalar() == 0 # the row is gone
assert get_settings().llm_summary_thinking_level == ""
# (e) the non-admin 403 gate still applies (asserted once for this
# key — the router-wide dependency is pinned on ALL four routes in
# ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
# (f) INDEPENDENCE — the whole point of the split: save the summary
# row "high" again; the chat row's effective value is STILL "low"
# (the two overlays resolve independently — a saved summary value
# never touches the chat level).
r = admin_client.put(f"{API}/llm_summary_thinking_level", json={"value": "high"})
assert r.status_code == 200, r.text
assert get_settings().llm_summary_thinking_level == "high"
assert get_settings().llm_chat_thinking_level == "low" # untouched
chat_item = _item(admin_client.get(API).json(), "llm_chat_thinking_level")
assert chat_item["value"] == "low" # the panel agrees, too
admin_client.delete(f"{API}/llm_summary_thinking_level") # no row survives
def test_docs_token_is_write_only_masked_end_to_end(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
+61 -16
View File
@@ -10,9 +10,10 @@ snapshot, ``app/api/chat.py::get_llm``), and the chat concurrency
semaphore (``chat_max_concurrent``). Plus the kill-switch counterpart
(``BOR_SETTINGS_DB=0`` with rows present → env values are back) and
the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_thinking_level`` — the value the LLM
layer reads per request via ``get_settings()``; the off switch is
live-settable, A2).
phase-172 thinking level (``llm_chat_thinking_level`` — the value the
LLM layer reads per request via ``get_settings()``; the off switch is
live-settable, A2) and its phase-178 summary twin (task 02:
``llm_summary_thinking_level``, default ``""`` = off, A2).
The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` —
@@ -164,44 +165,88 @@ def test_llm_client_repoints_on_the_next_save(db: Session) -> None:
# ---------------------------------------------------------------------------
# the thinking level: save/reset resolve the live value (phase 172, task 02)
# the chat thinking level: save/reset resolve the live value
# (phase 178, task 01 — the phase-172 test, renamed)
# ---------------------------------------------------------------------------
def test_llm_thinking_level_live_apply_no_restart(
def test_llm_chat_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 172 (task 02): the thinking level follows the overlay
"""Phase 178 (task 01): the chat thinking level follows the overlay
live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE
call after the write) resolves
``get_settings().llm_thinking_level`` WITHOUT a restart (the proof
the LLM layer — task 03 reads the setting per request via
``get_settings().llm_chat_thinking_level`` WITHOUT a restart (the
proof the LLM layer — task 03 reads the setting per request via
``get_settings()`` — picks the override up live), the EMPTY value
resolves live too (the off switch is live-settable, A2/A4), and a
reset (the DELETE) returns the env value.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_THINKING_LEVEL", raising=False)
monkeypatch.delenv("BOR_LLM_CHAT_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
_save(db, "llm_thinking_level", "high")
assert get_settings().llm_chat_thinking_level == "low"
_save(db, "llm_chat_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_thinking_level == "high" # NO restart
assert get_settings().llm_chat_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_thinking_level", "")
_save(db, "llm_chat_thinking_level", "")
invalidate_settings()
assert get_settings().llm_thinking_level == ""
assert get_settings().llm_chat_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_thinking_level'")
text("DELETE FROM app_settings WHERE key = 'llm_chat_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_thinking_level == "low"
assert get_settings().llm_chat_thinking_level == "low"
# ---------------------------------------------------------------------------
# the summary thinking level: save/reset resolve the live value
# (phase 178, task 02 — the chat twin's summary counterpart)
# ---------------------------------------------------------------------------
def test_llm_summary_thinking_level_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 178 (task 02): the summary thinking level follows the
overlay live — a save (the PUT's write, minus the API: ``_save`` +
``invalidate_settings()`` is exactly what task 04's PUT/DELETE call
after the write) resolves ``get_settings().llm_summary_thinking_level``
WITHOUT a restart (the proof the LLM layer — task 03 reads the
setting per request via ``get_settings()`` — picks the override up
live), the EMPTY value resolves live too (the off switch is
live-settable, A2/A4), and a reset (the DELETE) returns the env
value (``""`` — the code default; the chat twin's reset returns
``"low"``, the two levels are independent).
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code default, no row in the table.
monkeypatch.delenv("BOR_LLM_SUMMARY_THINKING_LEVEL", raising=False)
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
_save(db, "llm_summary_thinking_level", "high")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == "high" # NO restart
# The off switch is live-settable too (A2/A4): the empty value
# resolves through the overlay, not stripped or coerced.
_save(db, "llm_summary_thinking_level", "")
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# The reset returns the env value (no process restart in between —
# the LLM layer re-reads per request, task 03).
db.execute(
text("DELETE FROM app_settings WHERE key = 'llm_summary_thinking_level'")
)
db.commit()
invalidate_settings()
assert get_settings().llm_summary_thinking_level == ""
# ---------------------------------------------------------------------------