phase: 96_oneshot_resilience
Build and Push Containers / build-and-push-app (push) Successful in 1m34s
Build and Push Containers / build-and-push-db (push) Successful in 10s

All checks complete. Final report:

**Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design)

- `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff
- `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe
- `.env.example` comments updated (chat-turn stream + one-shot summary calls)

**Test / lint / coverage results**
- `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%)
- `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated)
- Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed
- `uv run ruff check . && uv run pyright` → clean (0 errors)

**Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files).

**Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
This commit is contained in:
2026-09-11 13:16:20 -04:00
parent bcaef800c5
commit a49be80b8e
42 changed files with 2893 additions and 143 deletions
+106 -28
View File
@@ -33,6 +33,14 @@ Chat turns never generate folder summaries — the agent's ``ls`` output
caller's job at sync time (phase 94, task 02), and :func:`generate_
folder_summaries` only flushes — the sync path owns the transaction
(the phase-53 ``bump_sources_version`` convention).
Self-heal (phase 96): an exhausted one-shot retry can still leave a
candidate folder without a row. :func:`missing_folder_summaries`
names those gaps (the generator's exact candidate computation minus
the stored keys — one concept, as with :func:`group_by_folder`), and
:func:`generate_folder_summaries`'s ``only_missing=True`` fills
EXACTLY those on the next sync — existing rows stay
byte-identical (text AND ``updated_at``), the prune pass still runs.
"""
from __future__ import annotations
@@ -293,22 +301,85 @@ def _upsert(db: Session, source: str, folder_path: str, summary: str) -> None:
row.updated_at = now
def folder_summary_table_empty(db: Session) -> bool:
"""Whether ``folder_summaries`` holds no rows (the sync-path gate).
def _catalog_rows(db: Session) -> list[DocRow]:
"""The document catalogue in catalogue order (one query).
Phase 94 (task 02): the ``_overview_row_exists`` pattern
(``scripts.import_docs``) extended to a table-emptiness check —
after an unchanged re-sync, an EMPTY table (the first full sync
after migration 0017, or after a ``--limit`` debug walk that
skipped generation) still gets a fresh batch, while a populated
table is left untouched until the KB actually changes. One
bounded ``LIMIT 1`` probe, never a count scan.
``(source, path, title, summary)`` ordered by ``(source, path)`` —
the EXACT query :func:`generate_folder_summaries` runs, so the gap
detector and the fill can never disagree about what the catalogue
contains (phase 96, task 02: one concept, as with
:func:`group_by_folder`).
"""
return db.execute(select(FolderSummary.source).limit(1)).first() is None
result = db.execute(
select(Document.source, Document.path, Document.title, Document.summary)
.order_by(Document.source, Document.path)
).all()
return [
(source, path, title, summary)
for source, path, title, summary in result
]
def _candidates(rows: Sequence[DocRow]) -> dict[tuple[str, str], list[DocRow]]:
"""The generator's candidate map (recursive subtree ≥ 2 docs).
The :func:`group_by_folder` groups filtered by
:data:`MIN_DOCS_PER_FOLDER` — the EXACT set a full regeneration
would cover, so a "missing" folder can never disagree with what a
regeneration would (re)generate.
"""
groups = group_by_folder(rows)
return {
key: docs for key, docs in groups.items() if len(docs) >= MIN_DOCS_PER_FOLDER
}
def _stored_keys(db: Session) -> set[tuple[str, str]]:
"""The ``(source, folder_path)`` keys that already hold a row.
One bounded key select — no count scan (the phase-94
table-emptiness probe's house pattern).
"""
return {
(source, folder_path)
for source, folder_path in db.execute(
select(FolderSummary.source, FolderSummary.folder_path)
).all()
}
def missing_folder_summaries(db: Session) -> list[tuple[str, str]]:
"""The candidate folders whose stored row is ABSENT (phase 96, 02).
The unchanged-sync self-heal gate: a one-shot reply can still
exhaust its retries and leave a candidate folder without a row, and
the phase-94 change gate only regenerated on a KB change — so the
gap persisted until the next KB change. This function names the gap:
the generator's candidate folders (the EXACT candidate computation
— one catalogue query in catalogue order, :func:`group_by_folder`,
recursive subtree ≥ :data:`MIN_DOCS_PER_FOLDER`) minus the stored
keys (one bounded select). The gate probe and the
:func:`generate_folder_summaries` fill both key off this one
concept.
Returns the missing keys sorted by ``(source, folder_path)``;
``[]`` when there is no gap — including an empty catalogue over an
empty table (no candidates, no gap). A single-document folder is
never listed (it is not a candidate), and a stored row for a folder
that dropped below the minimum is NOT missing (it is stale — the
prune pass owns it).
"""
return sorted(
key for key in _candidates(_catalog_rows(db)) if key not in _stored_keys(db)
)
async def generate_folder_summaries(
db: Session, llm: FolderSummaryLLM, *, skip: bool = False
db: Session,
llm: FolderSummaryLLM,
*,
skip: bool = False,
only_missing: bool = False,
) -> dict[str, int]:
"""Regenerate the stored folder summaries for the current catalogue.
@@ -328,38 +399,45 @@ async def generate_folder_summaries(
FOLDER: one folder's :class:`LLMError` is logged and counted,
its previous row (if any) is kept, and the remaining folders
still land (a ``lite`` outage must never fail the sync — the KB
is the product, the summaries are auxiliary).
is the product, the summaries are auxiliary). With
``only_missing=True`` the iteration is restricted to the
candidates that have NO stored row (the same gap
:func:`missing_folder_summaries` reports, derived from the SAME
catalogue pass — one bounded stored-key select, no second
catalogue query): existing rows stay byte-identical (summary
text AND ``updated_at`` — never re-stamped, even a stale-looking
one; staleness is the changed-KB regeneration's job) and no
``lite`` call is burned for a folder that already has a summary
— the unchanged-sync self-heal fill (phase 96, task 02).
4. DELETE rows whose folder no longer has ≥ 2 documents — a
pruned/renamed folder's summary goes stale and is dropped.
Rows for folders that still qualify persist (regenerated in
step 3 — an unchanged folder's summary is still true).
step 3 — an unchanged folder's summary is still true). The
prune pass runs in BOTH modes: under ``only_missing`` on an
unchanged catalogue it is a no-op (the invariant kept), and it
still drops rows whose folder fell below the minimum.
Only flushes — the CALLER commits (the phase-53
``bump_sources_version`` convention: the sync path owns the
transaction, so a failed sync rolls the summaries back with it).
Returns the small stats dict ``{"generated", "failed", "pruned"}``
for the caller's summary-line logging (PLAN §9 ample logging).
for the caller's summary-line logging (PLAN §9 ample logging) —
the caller logs the mode, not the generator.
"""
stats = {"generated": 0, "failed": 0, "pruned": 0}
if skip:
return stats
result = db.execute(
select(Document.source, Document.path, Document.title, Document.summary)
.order_by(Document.source, Document.path)
).all()
rows: list[DocRow] = [
(source, path, title, summary)
for source, path, title, summary in result
]
groups = group_by_folder(rows)
candidates = {
key: docs for key, docs in groups.items() if len(docs) >= MIN_DOCS_PER_FOLDER
}
candidates = _candidates(_catalog_rows(db))
keys = sorted(candidates)
if only_missing:
stored = _stored_keys(db)
keys = [key for key in keys if key not in stored]
for source, folder_path in sorted(candidates):
docs = candidates[(source, folder_path)]
for key in keys:
source, folder_path = key
docs = candidates[key]
try:
summary = await summarize_folder(source, folder_path, docs, llm)
except LLMError as e: