phase: 172_thinking_level
All verification complete. Final report: **Phase 172 final verification pass — all tasks (01–05) were already complete; verified the whole phase, no code fixes needed.** - **Verified implementation:** `Settings.llm_thinking_level: str = "low"` + `CATALOG` row (`LLM (aipi)` / "Thinking level", plain: not secret/restart/overridable-exempt) + `.env.example` line under `BOR_LLM_CHAT_MODEL` + README table row; `_thinking_extra_body()` → `reasoning_effort` via `extra_body` on `chat()` + `chat_stream()` when trimmed-non-empty, key absent (byte-identical) when empty, embeddings untouched; per-turn line gains `thinking=<value>` (`-` when empty) before `thinking_chars`. - **Tests/lint/coverage (all green):** `uv run pytest` → 4084 passed · `uv run pytest --cov=app --cov-report=term-missing` → 4084 passed, `app/` **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings. - **Completion criteria:** (1) `pytest -k "config and thinking"` 4 passed + `.env.example` grep ✓ (2) live `GET /api/admin/settings` curl shows the row (str, `low`, `default`) + integration PUT/DELETE round-trip ✓ (3) 9 mocked-client unit tests assert `extra_body` present/absent ✓ (4) log-line tests assert `low`/`high`/`-`/whitespace ✓ (5) suite + coverage + lint ✓ (6) `test_thinking_level.py` 3/3 in isolation; `test_admin_settings_panel.py` 12/12; `test_admin_settings_layout.py` 5/5 ✓ (7) atomic commit + phase move = the harness's job (task files already in `complete/`; I ran no git index/history commands). - **Documentation:** README.md reviewed in full — verified consistent (env table complete vs `.env.example`, catalog restart/non-overridable claims re-verified at runtime); AGENTS.md reviewed in full — verified consistent (no rule affected, A5 = no UI change). No changes needed. **Flag (not fixable here):** PLAN.md §9's per-turn-line spec predates the phase-172 `thinking=` field; PLAN.md is off-limits per protocol. - **Next pending phase:** none — `todo/` contains only phase 172 (the harness moves it on this pass's success).
This commit is contained in:
@@ -1547,22 +1547,33 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
|
||||
``suggested=N`` immediately AFTER ``summary_hits=N`` — the field
|
||||
order of every existing field is untouched (the phase-114
|
||||
``retries=N scaffold_stripped=N`` tail stays last). The value is the
|
||||
seeded suggestion tier's size (``len(plan.suggested_docs)``)."""
|
||||
seeded suggestion tier's size (``len(plan.suggested_docs)``).
|
||||
Phase 172 (A6): the line gains ``thinking=<level>`` immediately
|
||||
BEFORE ``thinking_chars=N`` — the effective (overlay-resolved)
|
||||
``BOR_LLM_THINKING_LEVEL``, trimmed, ``-`` when empty (the off
|
||||
switch)."""
|
||||
_session, _llm = gate_env
|
||||
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
|
||||
chunks = [_chunk(doc, 0.30), _chunk(doc, 0.20, is_summary=True)]
|
||||
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever(chunks))
|
||||
|
||||
with caplog.at_level("INFO", logger="app.chat"):
|
||||
_ask(client, "How is my Kubernetes cluster set up?")
|
||||
def _last_log_line() -> str:
|
||||
caplog.clear()
|
||||
with caplog.at_level("INFO", logger="app.chat"):
|
||||
_ask(client, "How is my Kubernetes cluster set up?")
|
||||
log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert log_lines
|
||||
return log_lines[-1]
|
||||
|
||||
log_lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert log_lines
|
||||
line = log_lines[-1]
|
||||
# The default (the fixture's settings carry the field default):
|
||||
# thinking=low, immediately before thinking_chars.
|
||||
line = _last_log_line()
|
||||
assert "thinking=low thinking_chars=" in line
|
||||
# The new slot: suggested=N right after summary_hits=N (one suggested
|
||||
# doc — the single fixture doc — and one summary hit on it).
|
||||
assert "summary_hits=1 suggested=1" in line
|
||||
# The full field order (the phase-114 tail stays last).
|
||||
# The full field order (the phase-114 tail stays last; phase 172
|
||||
# inserts thinking= between sources= and thinking_chars=).
|
||||
order = (
|
||||
"question=",
|
||||
"embed_ms=",
|
||||
@@ -1576,6 +1587,7 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
|
||||
"threshold=",
|
||||
"deflected=",
|
||||
"sources=",
|
||||
"thinking=",
|
||||
"thinking_chars=",
|
||||
"tool_calls=",
|
||||
"rounds=",
|
||||
@@ -1589,3 +1601,31 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
|
||||
pos = line.find(field)
|
||||
assert pos > idx, f"{field} out of order in the per-turn log line"
|
||||
idx = pos
|
||||
|
||||
# The effective value: a non-default setting rides the line as-is.
|
||||
monkeypatch.setattr(
|
||||
chat_api,
|
||||
"get_settings",
|
||||
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level="high"), # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
line = _last_log_line()
|
||||
assert "thinking=high thinking_chars=" in line
|
||||
|
||||
# The off switch: an empty setting logs ``-`` ...
|
||||
monkeypatch.setattr(
|
||||
chat_api,
|
||||
"get_settings",
|
||||
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=""), # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
line = _last_log_line()
|
||||
assert "thinking=- thinking_chars=" in line
|
||||
|
||||
# ... and so does a whitespace-only one (A6: the TRIMMED value
|
||||
# decides, the log field mirrors the wire's trim).
|
||||
monkeypatch.setattr(
|
||||
chat_api,
|
||||
"get_settings",
|
||||
lambda: Settings(_env_file=None, relevance_threshold=0.30, llm_thinking_level=" "), # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
line = _last_log_line()
|
||||
assert "thinking=- thinking_chars=" in line
|
||||
|
||||
Reference in New Issue
Block a user