phase: 96_oneshot_resilience
All checks complete. Final report: **Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design) - `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff - `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe - `.env.example` comments updated (chat-turn stream + one-shot summary calls) **Test / lint / coverage results** - `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%) - `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated) - Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed - `uv run ruff check . && uv run pyright` → clean (0 errors) **Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files). **Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
This commit is contained in:
+131
-1
@@ -385,7 +385,38 @@ test_llm_retry.py``). The mock is single-conversation per e2e server, so
|
||||
SDK-level retries), so one POST per attempt: the counter is per
|
||||
POST here, unlike the chat counter above.
|
||||
Non-streaming requests (document summaries, KB overview) never 500 —
|
||||
the retry scope is the chat turn only (owner-locked A1).
|
||||
the retry scope is the chat turn only (owner-locked A1). The one
|
||||
non-streaming injection is phase 96's incident shape below (it
|
||||
answers 200 with EMPTY content — the semantic failure class, not a
|
||||
dead endpoint).
|
||||
|
||||
Failure injection (phase 96, one-shot resilience, task 04) — the
|
||||
2026-09-11 incident shape for the folder-summary one-shot path
|
||||
(``tests/e2e/test_oneshot_llm_retry.py``): a NON-stream
|
||||
``chat/completions`` request whose system prompt carries
|
||||
``FOLDER_SUMMARY_MODE`` (the folder-summary marker — ``chat()`` is
|
||||
the mock's only non-streaming consumer of it) and whose user
|
||||
message's ``Folder: …`` header (the branch's existing parse, the
|
||||
``FOLDER_HEADER_PREFIX`` tail) labels this suite's own fixture
|
||||
folders:
|
||||
- the label ends with ``/e2e_empty_once``: the FIRST non-stream POST
|
||||
for that label answers the incident envelope — the mock's normal
|
||||
OpenAI chat-completion shape with ``choices[0].message.content =
|
||||
""`` and ``choices[0].finish_reason = "length"`` (the exact wire
|
||||
shape of the empty ``lite`` reply: the budget spent in
|
||||
``reasoning_content``) — and every later POST returns the normal
|
||||
``Fixture folder summary for <folder>.`` line (the one-shot retry,
|
||||
phase 96 task 01, recovers the row).
|
||||
- the label ends with ``/e2e_empty_always``: EVERY non-stream POST
|
||||
for that label answers the empty envelope (the 1 +
|
||||
``BOR_LLM_RETRIES`` exhaustion → per-folder fail-soft → the row
|
||||
stays absent while the sync stays green, phase 94 contract).
|
||||
The once-sequence is driven by a module-level per-label counter that
|
||||
resets after the success it guards (the phase-67 ``_fail_posts``
|
||||
pattern — the mock is single-conversation per e2e server), so a
|
||||
second sync re-drives the sequence deterministically. The trigger
|
||||
strings are this suite's own folder names, so no other E2E can hit
|
||||
them (they seed different trees).
|
||||
|
||||
``max_tokens`` is honored deterministically (token ≈ whitespace word),
|
||||
like a real endpoint: an answer longer than the cap is truncated. This
|
||||
@@ -1496,6 +1527,70 @@ def _history_echo(body: dict[str, Any]) -> str:
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 96 (task 04, one-shot resilience): the incident-shape injection
|
||||
# for the folder-summary one-shot path — see the module docstring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The folder-label triggers (phase 96, task 04): the ``FOLDER_SUMMARY_MODE``
|
||||
#: branch's folder label (the user message's ``Folder: …`` tail — the
|
||||
#: ``FOLDER_HEADER_PREFIX`` parse) ending with these suffixes is THIS
|
||||
#: SUITE'S own fixture folder (``tests/e2e/test_oneshot_llm_retry.py``
|
||||
#: seeds the names — no other E2E can hit them; they seed different
|
||||
#: trees, and the triggers carry the E2E prefix).
|
||||
ONESHOT_EMPTY_ONCE_SUFFIX = "/e2e_empty_once"
|
||||
ONESHOT_EMPTY_ALWAYS_SUFFIX = "/e2e_empty_always"
|
||||
|
||||
#: Module-level per-label incident counter — the mock is single-
|
||||
#: conversation per e2e server (the phase-67 ``_fail_posts``
|
||||
#: convention). Counts the non-stream folder-summary POSTs served per
|
||||
#: trigger label; the once-sequence resets after the success it guards
|
||||
#: (the first normal reply), so a second sync re-drives the sequence
|
||||
#: deterministically.
|
||||
_empty_once_posts: dict[str, int] = {}
|
||||
|
||||
|
||||
def _folder_summary_incident(body: dict[str, Any]) -> bool:
|
||||
"""Should this NON-stream folder-summary POST answer with the
|
||||
2026-09-11 incident envelope (``content=""`` +
|
||||
``finish_reason="length"`` — the exact wire shape of the empty
|
||||
``lite`` reply phase 96's one-shot retry targets)?
|
||||
|
||||
* the system prompt lacks ``FOLDER_SUMMARY_MODE`` → never (only the
|
||||
folder-summary one-shot carries the marker — ``chat()`` is the
|
||||
mock's only non-streaming consumer of it, so the counter counts
|
||||
exactly the one-shot POSTs the app's retry policy drives);
|
||||
* the label ends with ``/e2e_empty_always`` → EVERY POST (the
|
||||
exhaustion path — 1 + ``BOR_LLM_RETRIES`` empty attempts, the
|
||||
per-folder fail-soft leaves the row absent);
|
||||
* the label ends with ``/e2e_empty_once`` → the FIRST non-stream
|
||||
POST for that label only — every later POST returns the normal
|
||||
line (the retry recovers the row), and the counter resets on
|
||||
that first normal reply (the phase-67 pattern).
|
||||
|
||||
The folder label is the ``FOLDER_SUMMARY_MODE`` branch's existing
|
||||
parse (the user message's first line, the ``FOLDER_HEADER_PREFIX``
|
||||
tail) — the injection is a pure function of the request plus the
|
||||
per-label counter (the house marker-flow convention).
|
||||
"""
|
||||
if "FOLDER_SUMMARY_MODE" not in _system(body):
|
||||
return False
|
||||
user = _user(body)
|
||||
header = user.splitlines()[0] if user else ""
|
||||
if not header.startswith(FOLDER_HEADER_PREFIX):
|
||||
return False
|
||||
label = header.removeprefix(FOLDER_HEADER_PREFIX).strip()
|
||||
if label.endswith(ONESHOT_EMPTY_ALWAYS_SUFFIX):
|
||||
return True
|
||||
if label.endswith(ONESHOT_EMPTY_ONCE_SUFFIX):
|
||||
n = _empty_once_posts.get(label, 0) + 1
|
||||
_empty_once_posts[label] = n
|
||||
if n == 1:
|
||||
return True # the first POST: the incident envelope
|
||||
_empty_once_posts[label] = 0 # the retry went out — restart
|
||||
return False
|
||||
|
||||
|
||||
def compose_answer(body: dict[str, Any]) -> str:
|
||||
system = _system(body)
|
||||
user = _user(body)
|
||||
@@ -1516,6 +1611,12 @@ def compose_answer(body: dict[str, Any]) -> str:
|
||||
# shadow every folder-summary call. Checked BEFORE the
|
||||
# DEFLECT_MODE branch, like the other lite-mode markers (a
|
||||
# deflection prompt never carries one).
|
||||
# Phase 96 (task 04): the incident-shape injection keys on THIS
|
||||
# branch's label (``_folder_summary_incident`` — the module
|
||||
# docstring) and overrides the NON-STREAM response envelope in
|
||||
# ``chat_completions`` (``content=""`` +
|
||||
# ``finish_reason="length"``); the line below is what the later
|
||||
# / normal POSTs return.
|
||||
header = user.splitlines()[0] if user else ""
|
||||
folder = (
|
||||
header.removeprefix(FOLDER_HEADER_PREFIX).strip()
|
||||
@@ -2167,6 +2268,35 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
)
|
||||
|
||||
if not body.get("stream"):
|
||||
# Phase 96 (task 04): the incident-shape injection (the module
|
||||
# docstring) — the trigger-labelled folder summary answers the
|
||||
# exact 2026-09-11 envelope: the mock's normal OpenAI
|
||||
# chat-completion shape with ``content=""`` and
|
||||
# ``finish_reason="length"`` (the budget spent in
|
||||
# ``reasoning_content``). ``chat()`` is the mock's only
|
||||
# non-streaming folder-summary consumer, so this is the one-shot
|
||||
# retry path (``app.rag.llm.LLMClient.chat``, phase 96 task 01)
|
||||
# and nothing else — every other non-stream response is
|
||||
# byte-identical to pre-phase-96.
|
||||
if _folder_summary_incident(body):
|
||||
return {
|
||||
"id": f"chatcmpl-{uuid.uuid4()}",
|
||||
"object": "chat.completion",
|
||||
"created": int(time.time()),
|
||||
"model": body.get("model", "turbo"),
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": ""},
|
||||
"finish_reason": "length",
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 2048,
|
||||
"total_tokens": 2148,
|
||||
},
|
||||
}
|
||||
message: dict[str, Any] = {"role": "assistant", "content": answer}
|
||||
if thinking:
|
||||
# Harmless future-proofing: the app only uses streaming, but a
|
||||
|
||||
Reference in New Issue
Block a user