phase: 96_oneshot_resilience
All checks complete. Final report: **Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design) - `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff - `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe - `.env.example` comments updated (chat-turn stream + one-shot summary calls) **Test / lint / coverage results** - `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%) - `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated) - Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed - `uv run ruff check . && uv run pyright` → clean (0 errors) **Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files). **Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
This commit is contained in:
+83
-37
@@ -56,14 +56,22 @@ than none).
|
||||
|
||||
The stored folder summaries (phase 94 — the drill-down ``ls``'s
|
||||
per-level descriptions, the ``folder_summaries`` table) regenerate in
|
||||
the same run under the same gate: a changed KB (added + updated > 0),
|
||||
or an empty table after an unchanged walk (the first full run after
|
||||
migration 0017, or after a ``--limit`` first walk that skipped them).
|
||||
Same contract — **best-effort, per-folder fail-soft**: a ``lite``
|
||||
failure keeps the failed folders' previous rows and only counts into
|
||||
the stats; ``--limit`` debug runs skip them entirely (never burning a
|
||||
``lite`` call). The stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>`` (or
|
||||
the same run under the same gate: a changed KB (added + updated > 0 —
|
||||
full regeneration), or, after an unchanged walk, a GAP — a candidate
|
||||
folder (≥ 2 docs) with no stored row (phase 96: this subsumes the old
|
||||
table-empty trigger exactly — an empty table leaves EVERY candidate
|
||||
missing, as after the first full run after migration 0017 or a
|
||||
``--limit`` first walk that skipped them — and catches the single row
|
||||
an exhausted one-shot retry lost mid-run). A gap after an unchanged
|
||||
walk fills ONLY the missing rows (``only_missing`` — every other row
|
||||
stays byte-identical, summary text AND ``updated_at``), and the run's
|
||||
stats token carries `` (gap-fill)`` behind the numbers. Same contract
|
||||
— **best-effort, per-folder fail-soft**: a ``lite`` failure keeps the
|
||||
failed folders' previous rows (or leaves the row absent) and only
|
||||
counts into the stats; ``--limit`` debug runs skip them entirely
|
||||
(never burning a ``lite`` call). The stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>`` (`` (gap-fill)``
|
||||
appended after the stats when the run took the targeted-fill path;
|
||||
``folder_summaries=skipped`` when the gate did not fire), and the rows
|
||||
commit in the run's own short-lived session (the phase-53 convention —
|
||||
a failed commit rolls them back with it).
|
||||
@@ -92,7 +100,7 @@ from app.core.debugging import configure_debugging
|
||||
from app.core.logging import configure_logging
|
||||
from app.db import SessionLocal
|
||||
from app.models import KbOverview
|
||||
from app.rag.folder_summaries import folder_summary_table_empty, generate_folder_summaries
|
||||
from app.rag.folder_summaries import generate_folder_summaries, missing_folder_summaries
|
||||
from app.rag.git_sources import effective_sources
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient
|
||||
@@ -231,18 +239,20 @@ def _overview_row_exists() -> bool:
|
||||
return session.get(KbOverview, 1) is not None
|
||||
|
||||
|
||||
def _folder_summaries_table_empty() -> bool:
|
||||
"""Whether the ``folder_summaries`` table holds any row (phase 94).
|
||||
def _folder_summaries_gap() -> list[tuple[str, str]]:
|
||||
"""The folder-summary gaps (phase 96, task 03): the candidate
|
||||
folders (≥ 2 docs) with no stored row, sorted.
|
||||
|
||||
The ``_overview_row_exists`` pattern extended to a table-emptiness
|
||||
check (one bounded ``LIMIT 1`` probe): an empty table after an
|
||||
unchanged re-import — e.g. the first full run after migration 0017,
|
||||
or after a ``--limit`` first walk that skipped generation — still
|
||||
gets a fresh batch of folder summaries, while a populated table is
|
||||
left untouched until the KB actually changes.
|
||||
The unchanged-walk self-heal trigger, replacing the phase-94
|
||||
table-emptiness probe (which the gap subsumes exactly: an empty
|
||||
table leaves every candidate missing, so the targeted fill over
|
||||
all candidates IS a full generation — the first full run after
|
||||
migration 0017, or after a ``--limit`` first walk that skipped
|
||||
generation, still generates — and a single row an exhausted
|
||||
one-shot retry lost mid-run is healed on the next sync).
|
||||
"""
|
||||
with SessionLocal() as session:
|
||||
return folder_summary_table_empty(session)
|
||||
return missing_folder_summaries(session)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
@@ -275,7 +285,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
|
||||
llm = LLMClient()
|
||||
|
||||
async def _run() -> tuple[ImportSummary, str, str, dict[str, int] | None]:
|
||||
async def _run(
|
||||
) -> tuple[ImportSummary, str, str, dict[str, int] | None, bool]:
|
||||
"""Import, then (change-gated) advance the sources version,
|
||||
refresh the stored KB overview, and regenerate the stored folder
|
||||
summaries.
|
||||
@@ -299,16 +310,24 @@ def main(argv: list[str] | None = None) -> int:
|
||||
runs and unchanged re-runs never bump. The returned token is
|
||||
the new version, or ``"skipped"``.
|
||||
|
||||
The folder summaries (phase 94, task 02) follow the same gate
|
||||
— a changed KB, or an empty ``folder_summaries`` table after an
|
||||
unchanged walk (the first full run after migration 0017, or
|
||||
after a ``--limit`` first walk) — with per-folder fail-soft
|
||||
The folder summaries (phase 94, task 02; phase 96, task 03)
|
||||
follow the same gate — a changed KB (full regeneration), or,
|
||||
after an unchanged walk, a GAP: a candidate folder (≥ 2 docs)
|
||||
with no stored row (the subsumed table-empty trigger — an
|
||||
empty table leaves every candidate missing — plus a row an
|
||||
exhausted one-shot retry lost) — with per-folder fail-soft
|
||||
inside the generator (a ``lite`` failure keeps the failed
|
||||
folders' previous rows). The generator only flushes: this
|
||||
folders' previous rows or leaves the row absent). The gap path
|
||||
passes ``only_missing=True`` (existing rows stay
|
||||
byte-identical) and its stats token gains the `` (gap-fill)``
|
||||
suffix on the summary line. The generator only flushes: this
|
||||
run's own short-lived session commits (the phase-53
|
||||
convention), and the stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>``
|
||||
(``None`` — rendered ``skipped`` — when the gate did not fire).
|
||||
(``None`` — rendered ``skipped`` — when the gate did not
|
||||
fire). The returned fifth element names the mode the stats
|
||||
were taken in (the `` (gap-fill)`` suffix trigger — ``True``
|
||||
only when the unchanged-walk gap fired the targeted fill).
|
||||
"""
|
||||
summary = await import_sources(
|
||||
sources, llm, prune=args.prune, limit=args.limit,
|
||||
@@ -339,24 +358,30 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# folder summaries (the --limit skip, mirrored above for the
|
||||
# sources version).
|
||||
logger.info("overview: skipped (--limit)")
|
||||
return summary, "skipped", sources_version, None
|
||||
return summary, "skipped", sources_version, None, False
|
||||
changed = summary.added + summary.updated > 0
|
||||
overview_due = changed
|
||||
folders_due = changed
|
||||
folder_gap_fill = False
|
||||
if not changed:
|
||||
if summary.files == 0:
|
||||
logger.info("overview: skipped (nothing imported)")
|
||||
return summary, "skipped", sources_version, None
|
||||
# The unchanged-walk first-run triggers: the overview when
|
||||
# no row exists yet (the first run after migration 0005),
|
||||
# the folder summaries when the table is empty (the first
|
||||
# full run after migration 0017, or after a --limit first
|
||||
# walk that skipped them).
|
||||
return summary, "skipped", sources_version, None, False
|
||||
# The unchanged-walk triggers: the overview when no row
|
||||
# exists yet (the first run after migration 0005), the
|
||||
# folder summaries when the table has a GAP — a candidate
|
||||
# folder (>= 2 docs) with no stored row (phase 96, task
|
||||
# 03; the old table-empty trigger is the special case
|
||||
# where every candidate is missing). The gap path is
|
||||
# ALWAYS the targeted fill: only the missing rows
|
||||
# regenerate (only_missing=True), every other row stays
|
||||
# byte-identical.
|
||||
overview_due = not _overview_row_exists()
|
||||
folders_due = _folder_summaries_table_empty()
|
||||
folders_due = bool(_folder_summaries_gap())
|
||||
folder_gap_fill = folders_due
|
||||
if not overview_due and not folders_due:
|
||||
logger.info("overview: skipped (KB unchanged)")
|
||||
return summary, "skipped", sources_version, None
|
||||
return summary, "skipped", sources_version, None, False
|
||||
overview_status = "skipped"
|
||||
if overview_due:
|
||||
ok = await regenerate_overview(llm)
|
||||
@@ -371,20 +396,41 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# summaries back with it.
|
||||
folder_stats: dict[str, int] | None = None
|
||||
if folders_due:
|
||||
# Phase 96 (task 03): a changed-KB run is a full
|
||||
# regeneration (today's behavior, byte-identical); the
|
||||
# unchanged-walk gap run is the targeted fill (only the
|
||||
# missing candidates burn a lite call).
|
||||
session = SessionLocal()
|
||||
try:
|
||||
folder_stats = await generate_folder_summaries(session, llm)
|
||||
folder_stats = await generate_folder_summaries(
|
||||
session, llm, only_missing=folder_gap_fill
|
||||
)
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
return summary, overview_status, sources_version, folder_stats
|
||||
return (
|
||||
summary, overview_status, sources_version, folder_stats,
|
||||
folder_gap_fill,
|
||||
)
|
||||
|
||||
summary, overview_status, sources_version, folder_stats = asyncio.run(_run())
|
||||
(
|
||||
summary,
|
||||
overview_status,
|
||||
sources_version,
|
||||
folder_stats,
|
||||
folder_gap_fill,
|
||||
) = asyncio.run(_run())
|
||||
folder_token = (
|
||||
"skipped"
|
||||
if folder_stats is None
|
||||
else f"{folder_stats['generated']}/{folder_stats['failed']}/{folder_stats['pruned']}"
|
||||
)
|
||||
if folder_stats is not None and folder_gap_fill:
|
||||
# PLAN §9 greppable-cron-safe line — the line-extension house
|
||||
# rule: the targeted fill (phase 96, task 03) is named on the
|
||||
# summary line; the full-regeneration token stays
|
||||
# byte-identical to phase 94.
|
||||
folder_token += " (gap-fill)"
|
||||
print(
|
||||
f"import_docs: files={summary.files} added={summary.added} "
|
||||
f"updated={summary.updated} unchanged={summary.unchanged} "
|
||||
|
||||
Reference in New Issue
Block a user