phase: 96_oneshot_resilience
Build and Push Containers / build-and-push-app (push) Successful in 1m34s
Build and Push Containers / build-and-push-db (push) Successful in 10s

All checks complete. Final report:

**Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design)

- `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff
- `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe
- `.env.example` comments updated (chat-turn stream + one-shot summary calls)

**Test / lint / coverage results**
- `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%)
- `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated)
- Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed
- `uv run ruff check . && uv run pyright` → clean (0 errors)

**Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files).

**Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
This commit is contained in:
2026-09-11 13:16:20 -04:00
parent bcaef800c5
commit a49be80b8e
42 changed files with 2893 additions and 143 deletions
@@ -0,0 +1,43 @@
# Task 03 — The gap gate in both sync paths (replace the table-empty probe)
**Phase:** `96_oneshot_resilience` · **Story:** n/a.
## Objective
Wire task 02's primitives into the folder-summary gate in BOTH sync paths so an unchanged-KB sync self-heals missing rows (targeted fill) while a no-gap unchanged sync still burns zero `lite` calls and a changed-KB sync still regenerates everything exactly as today. This is the half of the fix that turns "a failed folder summary persists until a KB change" into "the next sync heals it".
## Work
1. `scripts/import_docs.py` — the `_run()` folder-summary block (the change-gated region next to the KB-overview regeneration):
- Keep the `--limit` full skip exactly as today (an incomplete debug walk must never regenerate or gap-fill).
- Changed KB (`summary.added + summary.updated > 0`) → `generate_folder_summaries(session, llm)` — full regeneration (unchanged).
- Unchanged KB → replace `folders_due = _folder_summaries_table_empty()` with `missing = missing_folder_summaries(session)` and `folders_due = bool(missing)`; when due, call `generate_folder_summaries(session, llm, only_missing=True)`. (This subsumes the table-empty first-run case exactly: empty table ⇒ every candidate is missing ⇒ `only_missing` over all candidates == a full generation.)
- The run's summary line token stays `folder_summaries=<g>/<f>/<p>`; when the run took the gap-fill path, append ` (gap-fill)` to the token (the PLAN §9 greppable-cron-safe line — the line-extension house rule). The `folder_stats is None` → `skipped` rendering is unchanged.
- Update the module docstring's folder-summary paragraph (the phase-94 text) to name the gap-fill trigger alongside the table-empty one.
2. `app/api/sync.py` — `_run_sync()` (the `fs_db` block, ~L271–283):
- `changed = summary.added + summary.updated > 0`. `changed` → full regeneration (unchanged). Unchanged → `missing = missing_folder_summaries(fs_db)`; non-empty → `generate_folder_summaries(fs_db, llm, only_missing=True)` + `logger.info("sync: folder_summaries gap-fill stats=%s", …)`; empty → the existing `sync: folder_summaries skipped (KB unchanged)` log.
- Update the block's docstring comment (the phase-94 text) the same way.
- No `sync_status` surface change (folder stats stay log-only — the phase-94 contract).
3. `app/rag/folder_summaries.py` — remove `folder_summary_table_empty` now that BOTH call sites use `missing_folder_summaries` (it becomes dead code). If anything else imports it (grep first), keep it and note why in the commit body; the expectation is exactly the two sync paths.
4. Imports: `missing_folder_summaries` in both sync modules (replace the `folder_summary_table_empty` imports).
- ASSUMPTION: the gate's "unchanged" decision is the importer's `added + updated` count exactly as today — a prune-only run (files deleted, none added/updated) is unchanged for the gate's purposes, so a prune that drops a folder below 2 docs is healed by the FULL regeneration on the next CHANGED run, not by a gap-fill (prune and gap-fill never race).
- ASSUMPTION: transaction convention unchanged — the generator flushes, each path commits in its own short-lived session (phase-53), so a gap-fill never half-writes and never blocks the `bump_sources_version` (the bump stays change-gated on the KB, not the summaries).
## Testing & Quality
- Integration — extend `tests/integration/test_sync_folder_summaries.py` (the phase-94 suite) and mirror the `test_import_docs_overview.py` fake-`LLMClient`-keyed-on-`FOLDER_SUMMARY_MODE` pattern; the fake records its `FOLDER_SUMMARY_MODE` calls so zero-burn and targeted-fill are directly asserted:
- **Script path, unchanged + gap:** import a changed KB (rows land), delete one stored row directly, re-run with no KB change → exactly **one** new `FOLDER_SUMMARY_MODE` call (the deleted row's folder only), that row is back with the deterministic fake text, every other row byte-identical (summary + `updated_at`), summary line carries `folder_summaries=…(gap-fill)`.
- **Script path, unchanged + no gap:** re-run with no change and a complete table → **zero** `FOLDER_SUMMARY_MODE` calls, token `folder_summaries=skipped` (the phase-94 zero-burn invariant, unchanged).
- **Script path, changed:** a KB change still triggers FULL regeneration (call count == candidate count, all rows re-stamped) — today's behavior.
- **Script path, `--limit`:** still skips generation entirely (zero calls, `skipped`).
- **API path** (`tests/integration/test_sync_api.py` extension or the folder suite's API variant): the same three branches via `POST /api/sync` — unchanged+gap → targeted fill + the `gap-fill` log; unchanged+no-gap → zero calls; changed → full.
- **First-run subsumption:** a fresh table (no rows) + populated KB on an unchanged walk → the gap-fill path generates every candidate (equivalent to the old table-empty trigger) — assert the full candidate set lands.
- The existing `test_sync_api.py` + `test_import_docs_git.py` + the phase-94 suite stay green (status shape + full-regeneration behavior unchanged).
- Coverage: **>90%** on the modified `app/` lines (`app/api/sync.py`; `scripts/` is outside the `app/` coverage gate but the integration tests exercise it).
## Completion Criteria
- [ ] unchanged-KB sync with one deleted row regenerates exactly that row on BOTH paths, leaves the rest byte-identical, and logs the gap-fill (integration-pinned)
- [ ] unchanged-KB sync with a complete table burns zero folder-summary `lite` calls (both paths)
- [ ] changed-KB sync regenerates all candidates (both paths) — byte-identical to today's behavior
- [ ] `--limit` (script) skips generation entirely; `sync_status` shape unchanged (API)
- [ ] `folder_summary_table_empty` removed (or retained with a documented reason); no dead imports
- [ ] `uv run pytest` green, coverage >90%, `uv run ruff check . && uv run pyright` clean
- [ ] no behavior change in completed work (phase 94 + phase 53 bump contract intact)