phase: 96_oneshot_resilience
All checks complete. Final report: **Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design) - `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff - `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe - `.env.example` comments updated (chat-turn stream + one-shot summary calls) **Test / lint / coverage results** - `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%) - `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated) - Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed - `uv run ruff check . && uv run pyright` → clean (0 errors) **Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files). **Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
# Phase 96 — One-shot LLM resilience: retry empty replies, self-heal missing folder summaries
|
||||
|
||||
**Source:** 2026-09-11 incident (owner chat) — the then-`lite` model intermittently returned empty completions: its internal monologue (the wire's `reasoning_content` field) consumed the entire `max_tokens=2048` budget, so the reply arrived as `content=""` + `finish_reason="length"`. Measured 2 empties in 90 folder-summary calls on the `deploy` source sync (≈2–7% per call in follow-up probes), losing two stored rows: `deploy/Deployments` (283 docs) and `deploy/Deployments/stackexpected/website/app` (7 docs). The app handled it exactly as designed (`LLMClient.chat` refuses to store a silent summary; `generate_folder_summaries` is per-folder fail-soft) — but nothing retried, and nothing ever filled the hole: the folder-summary gate (`changed KB or empty table`) skips regeneration on unchanged re-syncs, so missing rows persist until a KB change. The owner replaced the `lite` model server-side (it no longer emits `reasoning_content` — verified live: 16/16 clean, no `reasoning_content` field); this phase removes the remaining app-side exposure so ANY future transient empty reply from ANY one-shot consumer cannot silently lose data.
|
||||
**Story:** n/a (incident-driven resilience — one Playwright file per phase, run in isolation, A16).
|
||||
**Context:**
|
||||
- `app/rag/llm.py` — `LLMClient.chat()` (~L335–378) is the one-shot surface: transport failure → `LLMError` (wrapped), choiceless reply → `LLMError`, `content` None/blank → `LLMError("…returned empty content — refusing to store a silent summary")`. No retry. Phase 67's retry machinery (`BOR_LLM_RETRIES`=3 / `BOR_LLM_RETRY_DELAY`=5 s, flat; `chat_stream_retried`, `RetryPiece`) covers the **streaming** chat-turn path only — phase 67's owner-locked A1 explicitly scoped it to the chat turn and left `chat()` out ("admin/import paths with their own fail-fast behavior").
|
||||
- `chat()` consumers (all fail-soft per call, all protected once `chat()` retries): `app/rag/summarizer.py` L113 (document summaries — failure leaves `documents.summary` NULL + a `summary_errors` count), `app/rag/overview.py` L182 (KB overview — failure keeps the old row; its `_overview_row_exists` gate already self-heals a missing row on the next unchanged walk), `app/rag/folder_summaries.py` L267 via `summarize_folder` → `generate_folder_summaries` (per-folder fail-soft; **no self-heal** — the gap this phase fills), `app/rag/llm.py` L641 `check_models` (the sync-time "ping" probe).
|
||||
- `app/config.py` — `llm_retries: int = 3`, `llm_retry_delay: float = 5` (phase 67; validators already enforce the house kill-switch pattern: 0 disables, negative fails startup).
|
||||
- The openai SDK's own `max_retries=2` (kept by `LLMClient`) already re-POSTs **wire-level** 500s — the gap is the **semantic** failure: the endpoint is healthy, answers, and says nothing.
|
||||
- `scripts/import_docs.py` `_run()` (~L335–360) and `app/api/sync.py` `_run_sync()` (L276–281) — the folder-summary gate, one copy per path: `changed = summary.added + summary.updated > 0`; script path `folders_due = changed or _folder_summaries_table_empty()` (`--limit` skips entirely); API path `if changed or folder_summary_table_empty(fs_db)`. `folder_summary_table_empty` (`app/rag/folder_summaries.py`) is the bounded `LIMIT 1` probe.
|
||||
- `app/rag/folder_summaries.py` — `group_by_folder` (the one recursive-subtree concept), `MIN_DOCS_PER_FOLDER = 2`, `generate_folder_summaries` (sorted candidate iteration, per-folder fail-soft upsert, then a prune pass deleting rows whose folder lost ≥ 2 docs; flushes only — the sync path owns the transaction).
|
||||
- `tests/unit/test_llm_client.py` — the fake-client harness (`_FakeCompletion`, `_make_chat_client`, `SimpleNamespace` completions); existing pins `test_chat_missing_content_raises_llm_error` / `test_chat_whitespace_only_content_raises_llm_error` (~L989–1000) match `"empty content"`.
|
||||
- `tests/unit/test_folder_summaries.py` — the duck-typed fake LLM with a `.calls` counter (the generator's unit pattern).
|
||||
- `tests/integration/test_sync_folder_summaries.py` + `tests/integration/test_import_docs_overview.py` — the change-gate integration pattern (fake `LLMClient` keyed on the mode marker; zero-call assertions on unchanged re-runs).
|
||||
- `tests/e2e/mock_llm.py` — `compose_answer`'s `FOLDER_SUMMARY_MODE` branch (L1504) returns the deterministic one-liner `Fixture folder summary for <folder>.` from the user message's `Folder: …` header (imported `FOLDER_HEADER_PREFIX` — the mock can never drift from it); phase 67's failure-injection machinery: module-level `_fail_posts` counters + `_bump_fail`, keyed by trigger phrase, **reset after the success they guard**.
|
||||
- `tests/e2e/test_ls_tree_drilldown.py` (phase 94) — the scripted `ls` drill-down turn pattern (the mock echoes the tool result into its grounded answer — the E2E's only lens on the LLM context) and the temp-local-source seed + `POST /api/sync` pattern; `tests/e2e/test_llm_retry.py` (phase 67) — the test-server env-override pattern (`BOR_LLM_RETRY_DELAY=0`).
|
||||
|
||||
## Objective
|
||||
A transient empty one-shot LLM reply can no longer silently lose data: `LLMClient.chat()` retries an empty-content reply under the existing `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` policy (logged per attempt) before raising, and an **unchanged**-KB sync detects folder-summary gaps (candidate folders with no stored row) and fills **only** the missing rows — so a failed folder summary self-heals on the next sync instead of persisting until a KB change.
|
||||
|
||||
## Dependencies
|
||||
- `67_llm_retry` (complete) — the retry policy knobs (`BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY`), the logging/kill-switch conventions, and the mock's failure-injection pattern.
|
||||
- `94_ls_tree_drilldown` (complete) — `folder_summaries`, `generate_folder_summaries`, the change-gate, the E2E drill-down + sync patterns.
|
||||
|
||||
## Decisions recorded here (no PLAN.md change needed — no new technology, no new env vars, no new SSE events)
|
||||
- **D1 — knob reuse:** the one-shot path honors the **existing** `BOR_LLM_RETRIES` (default 3) / `BOR_LLM_RETRY_DELAY` (default 5 s) — no new settings. Phase 67's owner-locked A3 named these the house LLM-retry policy; extending them to `chat()` is a deliberate extension of that policy, not a deviation (the lock scoped *what phase 67 shipped*, it does not prohibit `chat()` from honoring the same knobs). `.env.example` comments are updated to say so. (If the owner prefers a separate one-shot knob, task 01 is a one-line change.)
|
||||
- **D2 — retry condition:** `chat()` retries **empty content only** (`content` None or whitespace — the exact incident failure class). A choiceless reply still raises immediately (no retry), and transport failures keep today's behavior (the SDK's `max_retries=2` already covers wire-level 500s; an app-level transport retry would double-stack the SDK policy and multiply the 300 s timeout).
|
||||
- **D3 — exhaustion contract:** after `1 + llm_retries` empty attempts, `chat()` raises the same `LLMError` type with an updated message naming the attempts (`…returned empty content on all {N} attempts — refusing to store a silent summary`); with `llm_retries=0` the message stays **byte-identical** to today's (the feature is fully off — phase 67 kill-switch convention). All consumers' fail-soft behavior is untouched (document summary → NULL + count, overview → old row, folder → skip + old row, probe → sync fails loudly as today).
|
||||
- **D4 — targeted gap-fill:** `generate_folder_summaries` gains `only_missing=False`. The changed-KB path is **unchanged** (full regeneration + prune). The unchanged-KB gate changes from "table empty?" to "any candidate row missing?" (subsumes the table-empty first-run case exactly); when a gap exists it runs `only_missing=True` — only the missing candidates are generated, existing rows stay byte-identical (text AND `updated_at`), and the prune pass still runs (a no-op on an unchanged KB, the invariant kept). `--limit` (script only) still skips generation entirely.
|
||||
- **Non-goals:** the streaming path (`chat_stream` / `chat_stream_retried`) is untouched (phase 67 territory); no `documents.summary`-NULL catch-up (the retry eliminates the failure class — expected per-doc loss p⁴; the importer's unchanged-skip would be a separate, larger change); no KB-overview catch-up (its `_overview_row_exists` gate already self-heals); no UI/SSE/sync-status surface change (folder stats stay log-only, phase 94 contract); no aipi/model-side change (owner already done).
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
|
||||
### The one-shot retry (task 01)
|
||||
`app/rag/llm.py` `LLMClient`:
|
||||
- Extract the single-attempt body of `chat()` into a private `_chat_once(messages, model) -> str` (the existing try/except transport wrap + choiceless check + empty-content check, verbatim behavior). `chat()` becomes: attempt 1 via `_chat_once`; on the **empty-content** `LLMError` only, while attempts remain: `logger.warning` (PLAN §9: model, `finish_reason` of the empty reply when available, `attempt n of N`) + `await asyncio.sleep(settings.llm_retry_delay)` (flat, phase 67) + retry via `_chat_once`. Any other `LLMError` propagates immediately.
|
||||
- `N = 1 + settings.llm_retries` total attempts. Exhaustion → the D3 message. `llm_retries=0` → one attempt, the byte-identical legacy message.
|
||||
- The empty-content check must capture the reply's `finish_reason` for the log line (the incident's signature was `finish_reason="length"` — it makes the warning greppable and self-explaining).
|
||||
- `chat()` docstring: the phase-30 "a silent empty summary must never be stored" contract stays, extended with the retry policy (D1/D2/D3). `.env.example`: the two phase-67 comments gain "(chat-turn stream and one-shot summary calls)".
|
||||
- `check_models` benefits automatically (no change).
|
||||
|
||||
### The gap detector + targeted fill (task 02)
|
||||
`app/rag/folder_summaries.py`:
|
||||
- `missing_folder_summaries(db) -> list[tuple[str, str]]` — the candidate folders (the generator's exact candidate computation: one catalog query ordered by `(source, path)`, `group_by_folder`, subtrees with ≥ `MIN_DOCS_PER_FOLDER`) that have **no stored row**, sorted by `(source, folder_path)`. The gate probe and the fill both use this one function — one concept, as with `group_by_folder`.
|
||||
- `generate_folder_summaries(db, llm, *, skip=False, only_missing=False)` — `only_missing=True` restricts the sorted candidate iteration to the keys from `missing_folder_summaries(db)` (computed from the SAME catalog pass — no second full query inside the generator: derive the missing set from the candidates already grouped). Upsert, fail-soft, and the prune pass are otherwise unchanged; stats dict shape unchanged (the caller logs the mode).
|
||||
- `folder_summary_table_empty` is deleted in task 03 (both call sites replaced — the function becomes dead code; its unit tests move to `missing_folder_summaries`).
|
||||
|
||||
### The sync gates (task 03)
|
||||
Both paths, identical shape (the phase-94 "one gate, two call sites" convention):
|
||||
- Changed KB → `generate_folder_summaries(session, llm)` — full regeneration (today's behavior, byte-identical).
|
||||
- Unchanged KB → `missing = missing_folder_summaries(session)`; non-empty → `generate_folder_summaries(session, llm, only_missing=True)` + a log line naming the gap-fill (script: the summary line token becomes `folder_summaries=<g>/<f>/<p> (gap-fill)`; API: `sync: folder_summaries gap-fill stats=…`); empty → today's skip log.
|
||||
- `--limit` (script) → unchanged full skip. Transaction convention unchanged (generator flushes, the path commits).
|
||||
|
||||
### The E2E (task 04)
|
||||
`tests/e2e/mock_llm.py` — the `FOLDER_SUMMARY_MODE` branch gains the incident-shape injection, keyed by the folder LABEL (the seeded KB's folder names carry the trigger — deterministic, no env, no request-order state beyond the phase-67 per-key counter, reset after the success it guards):
|
||||
- label ends with `/e2e_empty_once` → the FIRST non-stream POST for that label returns the incident envelope: same OpenAI shape, `message.content=""`, `finish_reason="length"`; later POSTs return the normal `Fixture folder summary for <folder>.` line.
|
||||
- label ends with `/e2e_empty_always` → every non-stream POST for that label returns the empty envelope.
|
||||
`tests/e2e/test_oneshot_llm_retry.py` (new; `app_server` + `mock_llm` conftest fixtures; admin login via `tests/e2e/auth_helpers.py`; sync via `POST /api/sync` per `test_sync_button.py`; the test server boots with `BOR_LLM_RETRY_DELAY=0` per the phase-67 pattern; default `BOR_LLM_RETRIES=3`):
|
||||
1. Seed a temp local-dir source with three ≥ 2-doc folders: `e2e_empty_once/`, `e2e_empty_always/`, `normal/`.
|
||||
2. Sync #1 → a scripted `ls <source>` drill-down turn (the phase-94 echo pattern) asserts: the `e2e_empty_once/` line carries `: Fixture folder summary for <src>/e2e_empty_once.` (the retry recovered the row — without task 01 it would be absent), the `normal/` line does too, and the `e2e_empty_always/` line has NO `: …` suffix (exhausted → fail-soft → absent) while the sync still reports success (the folder-stats failure never flips the run, phase 94).
|
||||
3. Delete the `normal` stored row directly (simulating a historical failure) → Sync #2 (KB unchanged) → the `normal` row is back with its deterministic text; the `e2e_empty_once` row's `updated_at` is **unchanged** (targeted fill — no full regeneration); `e2e_empty_always` remains absent (the gap-fill attempts it, exhausts, stays absent, sync still succeeds).
|
||||
|
||||
## Tasks
|
||||
1. `01_chat_retry.md` — `chat()` empty-content retry under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` + logging + unit pins
|
||||
2. `02_gapfill_generator.md` — `missing_folder_summaries` + `generate_folder_summaries(only_missing=…)` + unit pins
|
||||
3. `03_sync_gates.md` — the gap gate in `scripts/import_docs.py` + `app/api/sync.py` (replacing the table-empty probe) + integration pins
|
||||
4. `04_e2e_oneshot_retry.md` — the mock's incident-shape injection + `tests/e2e/test_oneshot_llm_retry.py` + regressions + commit
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_llm_client.py` (retry matrix: empty-then-success = 2 attempts + 1 sleep; all-empty = `1 + retries` attempts + the exhaustion message; first-success = 1 attempt, 0 sleeps, byte-identical return; choiceless reply = no retry, immediate error; transport error = no app-level retry; `llm_retries=0` = legacy byte-identical message; sleep value honored — record `asyncio.sleep`); `tests/unit/test_folder_summaries.py` (`missing_folder_summaries` shapes: fresh table → all candidates, full table → empty, one row deleted → that folder; `only_missing=True` → only missing keys generated, existing rows byte-identical incl. `updated_at`, zero LLM calls when there is no gap, prune still runs; `only_missing=False` unchanged).
|
||||
- Integration: `tests/integration/test_sync_folder_summaries.py` + the `test_import_docs_overview.py` pattern (both sync paths: unchanged + gap → targeted fill of exactly the missing rows; unchanged + no gap → zero `FOLDER_SUMMARY_MODE` calls; changed → full regeneration as today; `--limit` skips).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_oneshot_llm_retry.py`, in isolation, deterministic mock, `BOR_LLM_RETRY_DELAY=0`.
|
||||
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] a one-shot `chat()` call whose first reply is empty content (None or whitespace) is retried up to `BOR_LLM_RETRIES` times with `BOR_LLM_RETRY_DELAY` between attempts, one `WARNING` per retry (model + finish_reason + attempt count), and succeeds when a later attempt returns content (unit-pinned)
|
||||
- [ ] an all-empty `chat()` raises `LLMError` naming the attempts after exactly `1 + BOR_LLM_RETRIES` attempts; `BOR_LLM_RETRIES=0` reproduces today's single-attempt behavior and message byte-identically
|
||||
- [ ] the streaming path is untouched: `test_llm_retry.py` (phase 67) green in isolation
|
||||
- [ ] an unchanged-KB sync with a missing folder-summary row regenerates exactly that row (both `scripts/import_docs.py` and `POST /api/sync`), leaves every other row byte-identical (incl. `updated_at`), and logs the gap-fill (integration-pinned)
|
||||
- [ ] an unchanged-KB sync with no gap burns zero `lite` folder-summary calls (the phase-94 zero-burn invariant holds)
|
||||
- [ ] `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` green in isolation: the `e2e_empty_once` folder summary is recovered by the retry (visible in the `ls` drill-down), `e2e_empty_always` stays absent with the sync green, and the deleted-row gap self-heals on the next unchanged sync
|
||||
- [ ] regression E2E green in isolation: `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py`
|
||||
- [ ] `uv run pytest` green, coverage >90%, `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] no behavior change in completed phases; one atomic Conventional Commit, `--no-gpg-sign`
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agents/ app/ scripts/ tests/ && git commit --no-gpg-sign -m "fix(rag): retry empty one-shot LLM replies and self-heal missing folder summaries on unchanged syncs"
|
||||
```
|
||||
@@ -0,0 +1,38 @@
|
||||
# Task 01 — `chat()` retries an empty one-shot reply under the house retry policy
|
||||
|
||||
**Phase:** `96_oneshot_resilience` · **Story:** n/a (incident-driven resilience — phase 67's owner-locked A1 left the one-shot path out of scope; this task closes that gap per owner request 2026-09-11).
|
||||
|
||||
## Objective
|
||||
`LLMClient.chat()` (the one-shot surface used by document summaries, the KB overview, folder summaries, and the sync probe) retries an **empty-content** reply up to `BOR_LLM_RETRIES` times with `BOR_LLM_RETRY_DELAY` between attempts, logs each retry, and only then raises — so a transient "the model answered but said nothing" reply (the 2026-09-11 incident: `content=""`, `finish_reason="length"`, the whole budget spent in `reasoning_content`) no longer loses a summary on its first attempt.
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py` — `LLMClient`:
|
||||
- Extract `chat()`'s single-attempt body verbatim into a private `_chat_once(self, messages, model) -> str`: the existing `try/except` transport wrap (→ `LLMError` with the sanitized base URL), the choiceless check (`…returned no choices`), and the empty-content check (`content` None or `not content.strip()`). `_chat_once` must also surface the empty reply's `finish_reason` to its caller for the log line — e.g. raise a small internal signal or return it alongside; keep the public error messages exactly as they are today for the no-retry cases.
|
||||
- Rewrite `chat()` around it: `N = 1 + self.settings.llm_retries` total attempts. Attempt 1 → on the **empty-content** failure only: while attempts remain, `logger.warning(…)` with the model name, the empty reply's `finish_reason`, and `attempt {n} of {N}` (PLAN §9 ample logging — this line is the greppable record of the incident class), then `await asyncio.sleep(self.settings.llm_retry_delay)` (flat delay — the phase-67 convention), then the next attempt. Every other `LLMError` (transport, no-choices) propagates immediately — **no** app-level retry (the openai SDK's own `max_retries=2` already re-POSTs wire-level failures; an app-level transport retry would stack on top of it).
|
||||
- Exhaustion: raise `LLMError` with the updated message `f"…returned empty content on all {N} attempts — refusing to store a silent summary"` (same base-URL sanitization as today). When `llm_retries == 0`, raise the **current** message verbatim (`…returned empty content — refusing to store a silent summary`) — the kill-switch must be byte-identical to pre-phase-96 behavior (house byte-identical convention).
|
||||
- Update `chat()`'s docstring: the phase-30 contract ("a silent empty summary must never be stored") stands; add the retry policy (D1/D2/D3 of `00_phase.md`): which failures retry (empty content only), the knobs, the flat delay, the exhaustion message.
|
||||
- `check_models` (the "ping" probe) needs no change — it benefits automatically.
|
||||
2. `.env.example` — the `BOR_LLM_RETRIES` / `BOR_LLM_RETRY_DELAY` comments gain that they now cover "the chat-turn stream (phase 67) and one-shot summary calls (phase 96)". No new settings, no `app/config.py` change (the validators for both knobs already exist from phase 67).
|
||||
|
||||
- ASSUMPTION: the retry loop lives inside `chat()` itself (one place, every consumer protected) — NOT in each caller; the generators' per-folder fail-soft semantics are untouched (they still catch the post-exhaustion `LLMError` exactly as today).
|
||||
- ASSUMPTION: `finish_reason` availability — the empty reply is already parsed (`resp.choices[0]`); reading `.finish_reason` off it is free. If a provider omits it, log `finish_reason=None` (the line must never crash on the diagnostic path).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit — `tests/unit/test_llm_client.py` (extend the existing `_FakeCompletion`/`_make_chat_client` harness; the fake completions object can be scripted to yield a sequence of replies per `create()` call):
|
||||
- empty-then-success: 2 attempts, exactly 1 sleep of `llm_retry_delay`, returns the second reply's trimmed content; the `WARNING` fired once (caplog).
|
||||
- all-empty with `llm_retries=3` (default): 4 attempts, 3 sleeps, `LLMError` matching `all 4 attempts`; with a custom `llm_retries=1`: 2 attempts, message names 2.
|
||||
- first-attempt success: exactly 1 `create()` call, **zero** sleeps, return value byte-identical to today's behavior (the happy path is untouched).
|
||||
- choiceless reply: 1 attempt, no retry, the existing `no choices` error.
|
||||
- transport failure: 1 attempt, no retry, the existing wrapped error.
|
||||
- `llm_retries=0`: empty reply → 1 attempt, zero sleeps, the **legacy** message byte-identical (assert the exact string).
|
||||
- sleep value: record `asyncio.sleep` calls (monkeypatch) — flat `llm_retry_delay` each time, never a growing backoff.
|
||||
- update the two existing pins (`test_chat_missing_content_raises_llm_error`, `test_chat_whitespace_only_content_raises_llm_error`, ~L989–1000) to the new contract — either pin the exhaustion path with a low-retry setting or set `llm_retries=0` there to keep the legacy-message assertion; both `None` and whitespace-only content must still refuse.
|
||||
- Coverage: **>90%** on the modified `app/rag/llm.py` lines (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- No integration/E2E in this task (tasks 03/04 cover the paths end to end).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_llm_client.py -v` green with the new pins above
|
||||
- [ ] full suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] the streaming path (`chat_stream`, `chat_stream_retried`) is byte-identical — no diff outside `chat()`/`_chat_once` in `app/rag/llm.py`
|
||||
- [ ] `.env.example` comments updated; no new settings
|
||||
- [ ] no behavior change in completed work (phase 67's retry suite green)
|
||||
@@ -0,0 +1,36 @@
|
||||
# Task 02 — Gap detector + targeted fill in the folder-summary generator
|
||||
|
||||
**Phase:** `96_oneshot_resilience` · **Story:** n/a.
|
||||
|
||||
## Objective
|
||||
Give `app/rag/folder_summaries.py` the two primitives the sync gate (task 03) and the self-heal need: `missing_folder_summaries(db)` — the candidate folders whose stored row is absent (one concept, the generator's exact candidate computation) — and `generate_folder_summaries(…, only_missing=False)`, which restricts the upsert pass to those missing keys so an unchanged-KB gap-fill never re-burns `lite` for a folder that already has a good summary.
|
||||
|
||||
## Work
|
||||
1. `app/rag/folder_summaries.py`:
|
||||
- `missing_folder_summaries(db) -> list[tuple[str, str]]`:
|
||||
- Run the generator's catalog query verbatim (`select(Document.source, Document.path, Document.title, Document.summary).order_by(Document.source, Document.path)`), group via `group_by_folder`, keep the candidates (subtree ≥ `MIN_DOCS_PER_FOLDER`) — the SAME computation `generate_folder_summaries` does, so the gap can never disagree with what a regeneration would cover.
|
||||
- Subtract the stored keys: one `select(FolderSummary.source, FolderSummary.folder_path)` (bounded, no count scan — the `folder_summary_table_empty` house pattern).
|
||||
- Return `sorted(missing)` by `(source, folder_path)`. Empty list when nothing is missing (including an empty table over an empty KB — no candidates, no gap).
|
||||
- `generate_folder_summaries(db, llm, *, skip=False, only_missing=False)`:
|
||||
- When `only_missing=True`, after computing `groups`/`candidates` (the existing pass), intersect the iteration keys with `missing_folder_summaries(db)` computed from the SAME grouped rows (do not re-run the catalog query — derive the missing set from the candidates minus the one stored-key select). Upsert loop, per-folder fail-soft, and the prune pass are otherwise **unchanged**; the stats dict keeps its `{"generated", "failed", "pruned"}` shape (the caller logs the mode, not the generator).
|
||||
- When `only_missing=False` (default) the function is byte-identical in behavior to today.
|
||||
- Update the docstring: the new parameter, the "existing rows stay byte-identical (text AND `updated_at`) under `only_missing`" contract, and that the prune pass still runs in both modes (a no-op on an unchanged KB, the invariant kept).
|
||||
- Do NOT delete `folder_summary_table_empty` here — task 03 replaces both call sites and removes it (its unit tests move there).
|
||||
|
||||
- ASSUMPTION: "candidate" means exactly what `generate_folder_summaries` already means (the `00_phase.md`/phase-94 recursive-subtree rule) — a single-document folder is still not a candidate, so it can never appear as a "gap".
|
||||
- ASSUMPTION: `only_missing` is a **fill**, not a refresh — a stored row is never regenerated or re-stamped by an `only_missing` run, even a stale-looking one (staleness is the changed-KB regeneration's job, as today).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit — `tests/unit/test_folder_summaries.py` (extend the existing duck-typed fake-LLM + in-memory/SQLite-DB patterns used by that file):
|
||||
- `missing_folder_summaries`: fresh table + populated KB → exactly the candidate set; fully-populated table → `[]`; delete one stored row → `[(that source, that folder)]`; a single-document folder present in the KB and absent from the table → NOT listed (not a candidate); a stored row for a folder that dropped below 2 docs → not listed as missing (it's stale, not missing — the prune pass owns it).
|
||||
- `only_missing=True` with two missing candidates + two present ones: exactly 2 `llm.chat` calls (the missing keys, sorted order); the two present rows are byte-identical afterward (summary text **and** `updated_at` unchanged — capture both before/after); stats `generated=2, failed=0, pruned=0`.
|
||||
- `only_missing=True` with no gap: **zero** `llm.chat` calls, no rows touched (the zero-burn invariant the gate relies on).
|
||||
- `only_missing=True` + a folder that lost ≥ 2 docs since its row was written (seed a stale row manually): the prune pass still deletes it (`pruned=1`) while the genuine missing folders are filled.
|
||||
- `only_missing=False` regression: existing generator tests stay green unchanged (full regeneration + prune).
|
||||
- Coverage: **>90%** on the new/modified `app/rag/folder_summaries.py` lines.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_folder_summaries.py -v` green with the pins above
|
||||
- [ ] `generate_folder_summaries` default path is behavior-identical (all pre-existing tests unmodified and green)
|
||||
- [ ] full suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] no behavior change in completed work (phase 94's generator contract intact)
|
||||
@@ -0,0 +1,43 @@
|
||||
# Task 03 — The gap gate in both sync paths (replace the table-empty probe)
|
||||
|
||||
**Phase:** `96_oneshot_resilience` · **Story:** n/a.
|
||||
|
||||
## Objective
|
||||
Wire task 02's primitives into the folder-summary gate in BOTH sync paths so an unchanged-KB sync self-heals missing rows (targeted fill) while a no-gap unchanged sync still burns zero `lite` calls and a changed-KB sync still regenerates everything exactly as today. This is the half of the fix that turns "a failed folder summary persists until a KB change" into "the next sync heals it".
|
||||
|
||||
## Work
|
||||
1. `scripts/import_docs.py` — the `_run()` folder-summary block (the change-gated region next to the KB-overview regeneration):
|
||||
- Keep the `--limit` full skip exactly as today (an incomplete debug walk must never regenerate or gap-fill).
|
||||
- Changed KB (`summary.added + summary.updated > 0`) → `generate_folder_summaries(session, llm)` — full regeneration (unchanged).
|
||||
- Unchanged KB → replace `folders_due = _folder_summaries_table_empty()` with `missing = missing_folder_summaries(session)` and `folders_due = bool(missing)`; when due, call `generate_folder_summaries(session, llm, only_missing=True)`. (This subsumes the table-empty first-run case exactly: empty table ⇒ every candidate is missing ⇒ `only_missing` over all candidates == a full generation.)
|
||||
- The run's summary line token stays `folder_summaries=<g>/<f>/<p>`; when the run took the gap-fill path, append ` (gap-fill)` to the token (the PLAN §9 greppable-cron-safe line — the line-extension house rule). The `folder_stats is None` → `skipped` rendering is unchanged.
|
||||
- Update the module docstring's folder-summary paragraph (the phase-94 text) to name the gap-fill trigger alongside the table-empty one.
|
||||
2. `app/api/sync.py` — `_run_sync()` (the `fs_db` block, ~L271–283):
|
||||
- `changed = summary.added + summary.updated > 0`. `changed` → full regeneration (unchanged). Unchanged → `missing = missing_folder_summaries(fs_db)`; non-empty → `generate_folder_summaries(fs_db, llm, only_missing=True)` + `logger.info("sync: folder_summaries gap-fill stats=%s", …)`; empty → the existing `sync: folder_summaries skipped (KB unchanged)` log.
|
||||
- Update the block's docstring comment (the phase-94 text) the same way.
|
||||
- No `sync_status` surface change (folder stats stay log-only — the phase-94 contract).
|
||||
3. `app/rag/folder_summaries.py` — remove `folder_summary_table_empty` now that BOTH call sites use `missing_folder_summaries` (it becomes dead code). If anything else imports it (grep first), keep it and note why in the commit body; the expectation is exactly the two sync paths.
|
||||
4. Imports: `missing_folder_summaries` in both sync modules (replace the `folder_summary_table_empty` imports).
|
||||
|
||||
- ASSUMPTION: the gate's "unchanged" decision is the importer's `added + updated` count exactly as today — a prune-only run (files deleted, none added/updated) is unchanged for the gate's purposes, so a prune that drops a folder below 2 docs is healed by the FULL regeneration on the next CHANGED run, not by a gap-fill (prune and gap-fill never race).
|
||||
- ASSUMPTION: transaction convention unchanged — the generator flushes, each path commits in its own short-lived session (phase-53), so a gap-fill never half-writes and never blocks the `bump_sources_version` (the bump stays change-gated on the KB, not the summaries).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration — extend `tests/integration/test_sync_folder_summaries.py` (the phase-94 suite) and mirror the `test_import_docs_overview.py` fake-`LLMClient`-keyed-on-`FOLDER_SUMMARY_MODE` pattern; the fake records its `FOLDER_SUMMARY_MODE` calls so zero-burn and targeted-fill are directly asserted:
|
||||
- **Script path, unchanged + gap:** import a changed KB (rows land), delete one stored row directly, re-run with no KB change → exactly **one** new `FOLDER_SUMMARY_MODE` call (the deleted row's folder only), that row is back with the deterministic fake text, every other row byte-identical (summary + `updated_at`), summary line carries `folder_summaries=…(gap-fill)`.
|
||||
- **Script path, unchanged + no gap:** re-run with no change and a complete table → **zero** `FOLDER_SUMMARY_MODE` calls, token `folder_summaries=skipped` (the phase-94 zero-burn invariant, unchanged).
|
||||
- **Script path, changed:** a KB change still triggers FULL regeneration (call count == candidate count, all rows re-stamped) — today's behavior.
|
||||
- **Script path, `--limit`:** still skips generation entirely (zero calls, `skipped`).
|
||||
- **API path** (`tests/integration/test_sync_api.py` extension or the folder suite's API variant): the same three branches via `POST /api/sync` — unchanged+gap → targeted fill + the `gap-fill` log; unchanged+no-gap → zero calls; changed → full.
|
||||
- **First-run subsumption:** a fresh table (no rows) + populated KB on an unchanged walk → the gap-fill path generates every candidate (equivalent to the old table-empty trigger) — assert the full candidate set lands.
|
||||
- The existing `test_sync_api.py` + `test_import_docs_git.py` + the phase-94 suite stay green (status shape + full-regeneration behavior unchanged).
|
||||
- Coverage: **>90%** on the modified `app/` lines (`app/api/sync.py`; `scripts/` is outside the `app/` coverage gate but the integration tests exercise it).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] unchanged-KB sync with one deleted row regenerates exactly that row on BOTH paths, leaves the rest byte-identical, and logs the gap-fill (integration-pinned)
|
||||
- [ ] unchanged-KB sync with a complete table burns zero folder-summary `lite` calls (both paths)
|
||||
- [ ] changed-KB sync regenerates all candidates (both paths) — byte-identical to today's behavior
|
||||
- [ ] `--limit` (script) skips generation entirely; `sync_status` shape unchanged (API)
|
||||
- [ ] `folder_summary_table_empty` removed (or retained with a documented reason); no dead imports
|
||||
- [ ] `uv run pytest` green, coverage >90%, `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] no behavior change in completed work (phase 94 + phase 53 bump contract intact)
|
||||
@@ -0,0 +1,41 @@
|
||||
# Task 04 — E2E: the incident shape, retried and healed (the dedicated story suite)
|
||||
|
||||
**Phase:** `96_oneshot_resilience` · **Source:** 2026-09-11 incident (owner chat — the whole item: the empty-reply failure class + the missing-row gap). **Story:** n/a (A16 — one Playwright file per phase, run in isolation).
|
||||
|
||||
## Objective
|
||||
`tests/e2e/test_oneshot_llm_retry.py` proves the whole fix end to end against the deterministic mock: a folder whose FIRST one-shot reply arrives in the exact incident shape (`content=""`, `finish_reason="length"`) still ends up with its stored summary (the task-01 retry recovered it, visible in the `ls` drill-down), a folder whose replies are ALWAYS empty stays absent without failing the sync (task-01 exhaustion + the fail-soft contract), and a row deleted behind the app's back is self-healed by the next unchanged sync with the other rows untouched (tasks 02/03 targeted fill).
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/mock_llm.py` — the `compose_answer` `FOLDER_SUMMARY_MODE` branch (L1504) gains the incident-shape injection, keyed by the folder LABEL parsed from the `Folder: …` header (the existing parse — the seeded KB's folder names carry the trigger, so the injection is a pure function of the request, the house marker-flow convention):
|
||||
- A module-level per-label counter (the phase-67 `_fail_posts`/`_bump_fail` pattern; the mock is single-conversation per E2E server — reset each label's counter after the success it guards, so a second sync re-drives the sequence deterministically).
|
||||
- Label ends with `/e2e_empty_once` → the FIRST non-stream POST for that label returns the incident envelope: the mock's normal OpenAI chat-completion shape with `choices[0].message.content = ""` and `choices[0].finish_reason = "length"`; every later POST returns the normal `Fixture folder summary for {folder}.` line. (Inject at the NON-STREAM response path — `chat()` posts `stream=false`; the phase-67 `_llm_500` injection shows where the non-stream branch answers.)
|
||||
- Label ends with `/e2e_empty_always` → EVERY non-stream POST for that label returns the empty envelope.
|
||||
- No other mock branch changes; the trigger strings are this suite's own folder names, so no other E2E can hit them (they seed different trees).
|
||||
2. `tests/e2e/test_oneshot_llm_retry.py` (new — isolated: `tests/e2e/conftest.py` fixtures `app_server` + `mock_llm`; admin login via `tests/e2e/auth_helpers.py`; the test server boots with `BOR_LLM_RETRY_DELAY=0` per the phase-67 pattern in `test_llm_retry.py`, default `BOR_LLM_RETRIES=3`):
|
||||
- **Seed:** one temp local-dir source (the `test_local_directory_sources.py` registration + Sync pattern) with three ≥ 2-doc folders — `e2e_empty_once/` (2 `.md` files), `e2e_empty_always/` (2), `normal/` (2) — small, distinct contents.
|
||||
- **Sync #1** via `POST /api/sync` (the `test_sync_button.py` pattern; wait for completion) → then a scripted `ls <source>` drill-down turn (the phase-94 `test_ls_tree_drilldown.py` echo pattern — the mock's grounded answer carries the `ls` result, the E2E's only lens on the LLM context) asserting:
|
||||
- the `e2e_empty_once/` line is `e2e_empty_once/ — 2 documents: Fixture folder summary for <src>/e2e_empty_once.` — the retry recovered the row (without task 01 this line has no `: …` suffix);
|
||||
- the `normal/` line carries its summary;
|
||||
- the `e2e_empty_always/` line is `e2e_empty_always/ — 2 documents` with **no** `: …` suffix (all 4 attempts empty → `LLMError` → per-folder fail-soft → no row);
|
||||
- the sync reported success (the status endpoint / UI success state — a folder-summary exhaustion never flips the run, phase 94).
|
||||
- **Gap-fill:** delete the `normal` stored row directly from the DB (the test process has DB access via the conftest engine — the same connection the app uses), with no KB change, capture `e2e_empty_once`'s row `updated_at` → **Sync #2** → assert:
|
||||
- a second scripted `ls <source>` turn: `normal/` carries `: Fixture folder summary for <src>/normal.` again (healed); `e2e_empty_once/` still carries its summary;
|
||||
- the DB: the `normal` row's summary is the deterministic text; the `e2e_empty_once` row's `updated_at` is **unchanged** (targeted fill — a full regeneration would have re-stamped it);
|
||||
- `e2e_empty_always` is STILL absent (the gap-fill attempted it, exhausted, stayed absent) and the sync still succeeded.
|
||||
3. Run in isolation: `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` (db up: `podman compose up -d db`).
|
||||
4. Regression gate, in isolation: `test_ls_tree_drilldown.py` (phase 94 — the drill-down + folder-summary surface), `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py` (phase 67 — the streaming retry path must be untouched).
|
||||
|
||||
- ASSUMPTION: assertions on what the LLM's context carried go through the deterministic mock echoing the `ls` result into its answer (the house pattern from the agent-tool suites); DB-level assertions (row presence, `updated_at`) use the test process's direct DB access — the E2E's two established lenses.
|
||||
- ASSUMPTION: the mock's per-label counters reset per sync (after the success they guard), so sync #2 re-drives `e2e_empty_once` from "first POST empty" — which is FINE here: the row already exists, so the gap-fill never calls it at all (the `updated_at` assertion proves that); `e2e_empty_always` exhausts again by design.
|
||||
- ASSUMPTION: `BOR_LLM_RETRY_DELAY=0` on the test server keeps the 4-attempt exhaustion paths instant (phase 67's fast-suite convention); default `BOR_LLM_RETRIES=3` means the `e2e_empty_always` folder costs 4 POSTs per sync that attempts it — the suite stays fast.
|
||||
|
||||
## Testing & Quality
|
||||
- This IS the E2E task; the mock's new branch is covered by the suite itself (the mock is a real OpenAI-compatible server — no separate unit needed, per the existing agent/tool suites' practice).
|
||||
- Coverage: **>90%** on `app/` (this task adds test code only — keep the gate green).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` green in isolation and encoding: incident-shaped empty reply → retried → summary visible in the `ls` drill-down; always-empty folder → absent with the sync green; deleted row → self-healed on the next unchanged sync with `updated_at` untouched on the other rows
|
||||
- [ ] the mock injection is label-keyed and resets deterministically (a re-run of the suite is green without manual state cleanup)
|
||||
- [ ] regression E2E green in isolation: `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py`
|
||||
- [ ] full test suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] one atomic Conventional Commit, `--no-gpg-sign` (message in `00_phase.md`), phase dir moved to `.agents/phases/complete/`
|
||||
@@ -0,0 +1,104 @@
|
||||
# Phase 97 — The RAG view shows the KB as the drill-down tree the agent sees, with editable folder descriptions
|
||||
|
||||
**Source:** Owner request (chat, 2026-09-11) — "The RAG page shows a list of all files uploaded by the user, but the agent sees a tree structure after the last phase [94]. I want the UI to show what the agent sees — a tree that the user can click through so the user can see descriptions of the directories. The user should be able to edit the descriptions of the directories just like they can edit the summaries of any file."
|
||||
**Story:** n/a (owner request — the RAG view on the `02_story_import_documents` catalog; the folder descriptions + tree concept on `94_ls_tree_drilldown`; the edit affordance pattern on `57_edit_document_summaries`).
|
||||
**Context:** The RAG (Knowledge base) view — `frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (shell view module, phase 76) — renders the flat all-documents table from `GET /api/docs` (`app/api/docs.py`: source | path | title | chunks | indexed, `#docs-tbody`), with KB-wide stat cards, the sync button/poll machinery, and the phase-16 anonymous gate. Phase 94 made the agent's `ls` a drill-down tree over stored sync-time folder descriptions: `folder_summaries` (migration 0017, PK `(source, folder_path)`, `folder_path ""` = source root, rows only for ≥ 2-doc folders), the shared folder concept (`app/rag/folder_summaries.folder_of`; existence rule: a folder exists ⟺ some indexed path starts with `folder + "/"`; counts are the recursive subtree), and the pure level functions `app/rag/agent.py` `ls_top` / `group_folder_listing` / `ls_folder`. Phase 57's edit affordance (admin `PATCH /api/documents/summary` + the `.doc-summary` Edit → inline textarea → Save/Cancel → live-region status in `frontend/assets/document.js`) is the interaction model to mirror. Migration head is `0017_folder_summaries`. The deterministic E2E mock already returns the canned `FOLDER_SUMMARY_MODE` one-liner naming the folder (phase 94).
|
||||
|
||||
## Objective
|
||||
The admin's RAG (Knowledge base) view shows the catalog as the same drill-down tree the agent's `ls` walks — sources at the top, then per level the subfolders with their stored descriptions and the level's files — so the owner can click through the directories, read the descriptions the agent reads, and edit (or clear) any directory's description with the phase-57 inline edit affordance. A manually-edited description persists: the sync-time generator never overwrites and never prunes it.
|
||||
|
||||
## Dependencies
|
||||
- `94_ls_tree_drilldown` (complete) — the `folder_summaries` table, the stored per-folder descriptions, and the shared folder concept (`folder_of`, the existence rule, the recursive counts, `group_folder_listing` for the cross-check).
|
||||
- `57_edit_document_summaries` (complete) — the edit affordance pattern (Edit → inline textarea → Save/Cancel → live-region status) and the admin-gated summary-PATCH precedent.
|
||||
- `96_oneshot_resilience` (todo) — queue order only (numeric); no code dependency (different subsystem).
|
||||
|
||||
## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
|
||||
- **The tree replaces the flat all-documents table in the RAG view** (derived from the request: "I want the UI to show what the agent sees"). KB-wide stat cards stay; the per-level file table keeps the existing 5-column contract (`#docs-table` / `#docs-tbody`) so file rows are unchanged in shape; the flat `GET /api/docs` endpoint is untouched (API surface unchanged). Existing story suites whose catalog assertions assume the flat layout are updated IN THIS PHASE (tasks 07/08) — the asserted behavior (a document is listed, linkable, viewable) is preserved; only navigation gains a drill step.
|
||||
- **Manual edits persist across syncs** (derived from "just like they can edit the summaries of any file" — an edited file summary is never silently rewritten by a re-import): `folder_summaries` gains `manually_edited`; the phase-94 generator SKIPS (never overwrites) and never prunes a manual row. Clearing a description deletes the row — the next KB-changing sync regenerates an AI description for that folder (that IS the reset path; no separate regenerate button in v1).
|
||||
- **The UI tree is a superset of the `ls` top level:** sources list as registered sources in registry order (the `ls()` order, including a registered 0-document source — the phase-70/72 invariant) FOLLOWED by indexed sources that are not registered (path order) — ad-hoc `scripts/import_docs --source ~/X` imports and removed-but-not-yet-pruned sources. The catalog has never hidden an indexed document (the phase-02 contract); the agent's `ls` keeps showing registry sources only (phase 94) — unchanged.
|
||||
- **No re-embedding on edit** (contrast with phase 57): a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line) — the PATCH is a pure DB write, no LLM call.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
|
||||
### The tree endpoint (task 02)
|
||||
`GET /api/docs/tree` (admin, like `/api/docs`) returns the FULL recursive tree in one fetch — the UI drills client-side, zero per-level fetches:
|
||||
|
||||
```json
|
||||
{
|
||||
"sources": [
|
||||
{
|
||||
"name": "alpha", "documents": 5, "summary": "… or null",
|
||||
"children": [
|
||||
{ "kind": "folder", "path": "one", "documents": 2, "summary": "… or null",
|
||||
"children": [
|
||||
{ "kind": "file", "path": "one/a.md", "title": "A", "chunks": 3,
|
||||
"indexed_at": "2026-09-11T08:00:00+00:00" }
|
||||
]
|
||||
},
|
||||
{ "kind": "file", "path": "root-note.md", "title": "Root note", "chunks": 1,
|
||||
"indexed_at": "…" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- **Sources:** registered sources first in `list_source_names` (registry) order, then indexed-only sources (alphabetical) — the superset rule above. `documents` = the source's whole recursive count; `summary` = the stored `(source, "")` row or null. A registered 0-document source still lists (`0`, no children).
|
||||
- **Folder nodes:** `path` source-relative (never `""` — the source node IS the root); direct subfolders only, in path order (the phase-94 existence rule); `documents` = the recursive subtree count; `summary` = the stored row (AI OR manual) or null; `children` = its own subfolders + direct files, same shape.
|
||||
- **File nodes:** direct files only (`folder_of(path) == parent`), in path order (catalog order — the same order `GET /api/docs` serves); `path` source-relative; `title` / `chunks` (content + `is_summary` chunks — the same count `/api/docs` returns) / `indexed_at` verbatim.
|
||||
- **Pure builder** `build_kb_tree(names, doc_rows, summaries)` — module-level in `app/api/docs.py` (unit-testable without a DB): `names` = the registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order; `summaries` = `{(source, folder_path): summary}` over ALL stored rows. It reuses `app.rag.folder_summaries.folder_of` and the phase-94 existence rule — ONE concept end to end: the UI tree is the `ls` tree plus file metadata. **Cross-check property (unit-pinned):** for a single-source dataset the builder's level equals `app.rag.agent.group_folder_listing`'s output (same subfolder paths/counts/summaries, same file paths/titles in order) — "the UI shows what the agent sees" as a test, at the root and one nested level.
|
||||
- `GET /api/docs` stays untouched.
|
||||
|
||||
### Edit persistence (task 01)
|
||||
- Migration `0018_folder_summary_manual_flag` (down `0017_folder_summaries` — confirm with `alembic heads`): `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; tested downgrade drops the column (A13, the `test_migration_0017.py` pattern).
|
||||
- `generate_folder_summaries` (`app/rag/folder_summaries.py`): fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into one `{(source, folder_path): row}` dict). Upsert loop: an existing row with `manually_edited` is SKIPPED (no `lite` call — no burn on owner text) and counted `stats["kept_manual"] += 1`. Prune loop: deletes only NON-manual rows (a manual row for a folder that dropped below 2 documents is kept — owner content persists until cleared). The generator's logger line gains `kept_manual=%d`. **The import summary-line token STAYS 3 fields** (`scripts/import_docs.py` `folder_summaries=<generated>/<failed>/<pruned>` — `tests/integration/test_import_docs_overview.py` pins it).
|
||||
|
||||
### The edit endpoint (task 03)
|
||||
`PATCH /api/folders/summary` (admin) — body `{"source": str, "folder_path": str, "summary": str}`; `folder_path = ""` = the source root. Lives in `app/api/docs.py` next to the phase-57 document-summary PATCH (same `kb` router family).
|
||||
- **Source check:** registered (registry) OR has indexed documents → else 404 `{"detail": "source not found"}`.
|
||||
- **Folder check:** `""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some path starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem).
|
||||
- **Non-empty after strip** → upsert the row with `manually_edited = True` + fresh UTC `updated_at`. **Empty/whitespace** → delete the row if present (clear — the phase-57 analog; the reset path). Commit; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
|
||||
- No LLM/embedding call (the decision above); the docstring says so.
|
||||
|
||||
### The RAG view (tasks 04/05)
|
||||
`frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (the house createElement/textContent contract — never innerHTML with document-derived data):
|
||||
- **New static skeleton** (in `index.html`, after the stat cards, before the existing file `.table-wrap`):
|
||||
- `nav#kb-crumb[aria-label="Catalog location"][hidden]` — JS fills the segments.
|
||||
- `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title` (the level's full source-relative path, e.g. `alpha/two`), `p#kb-level-summary`. (The level Edit button lands in task 05.)
|
||||
- `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table` (visually-hidden caption, the `.docs-table` styling) — `thead`: **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled rows). ONE table for every level: at the top level the rows are the SOURCES themselves (the `ls()` equivalence — sources list exactly like folders: name, count, description).
|
||||
- **File table:** the existing `.table-wrap` / `#docs-table` / `#docs-tbody` is UNCHANGED in columns (Source | Path | Title | Chunks | Indexed) and in row rendering (`makeRow` — the path link still opens `openDocumentModal`, the no-JS href escape hatch intact). It now holds the CURRENT LEVEL's direct files; hidden when the level has none (at the top level it is always hidden — files are seen per source, as with `ls(source)`).
|
||||
- **Navigation state:** module-level `current = {source: string | null, folder: string | null}` (`{null, null}` = top level; `folder ""` = the source root). Source row click → `{source, ""}`; folder link click → `{source, folder}`; breadcrumb segment click → that ancestor. Breadcrumb: hidden at the top level; one link per ancestor (source, then folders), the last segment a `span` with `aria-current="page"`.
|
||||
- **Level rendering:** the level block shows the CURRENT level's stored description (source: the `(source, "")` row; folder: its row) and is hidden when none is stored (the `ls` rule — count only, no placeholder); the folders table = the direct subfolders (count + stored description + row Edit button in task 05); the file table = the direct files. `#folders-wrap` hidden when no subfolders.
|
||||
- **Stat cards:** unchanged KB-wide values — computed by walking the whole in-memory tree (documents, chunks sum, max `indexed_at`) — identical to today's flat walk.
|
||||
- **Empty state:** `#sources-empty` shows ONLY when the tree has zero sources (nothing registered, nothing indexed); a registered 0-document source renders its row (`0 documents`) instead.
|
||||
- **Never-stale reload (PLAN §7.4):** every re-fetch (boot, `bor:view-refresh`, sync-success settle, upload-success settle) re-renders the current level; if the current position no longer exists in the new tree (source unregistered/pruned, folder gone), the view RESETS to the top level — no stale breadcrumb, no stale block. The `loadSeq` race token (phase 79) carries over to the tree load.
|
||||
- **Unchanged:** the sync button/poll/label/banner/error-modal machinery, the anonymous gate (no `/api/docs*` fetch for anonymous — the gate branch never fetches the tree), the shell router wiring, the stat-card markup.
|
||||
- **Edit (task 05):** Edit buttons on `#kb-level` AND on every source/folder row's Description cell (a description can be CREATED where none is stored — a < 2-document folder or the generator's fail-soft miss; the button is always present — the view is admin-only already, the endpoint's `require_admin` is the API-level gate). The phase-57 interaction: swap the description text node for `textarea.kb-summary-editor` (prefilled via `value` — the XSS contract), Save / Cancel, `p.kb-summary-status[role="status"][aria-live="polite"]`. Save → `PATCH /api/folders/summary` `{"source", "folder_path", "summary"}` (folder_path `""` for the source root). 200 → re-render the text (textContent only) + status "Description updated." (empty save → the text goes away — level block hidden / cell emptied — + "Description cleared."); the in-memory tree node's summary updates in place (no re-fetch). Failure → neutral retry copy (the phase-55 convention), the editor stays open with the user's text. Cancel → restore the text node.
|
||||
- **`styles.css`:** `.kb-crumb` (+ link / current segment), `.kb-level` (+ title / summary typography), `.folder-link`, `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem — the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house dark-tech palette, AA contrast (text + color, never color alone), no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, ≥ 24px targets, no CDN.
|
||||
|
||||
## Tasks
|
||||
1. `01_manual_flag.md` — migration 0018 + `FolderSummary.manually_edited` + the generator's skip/keep logic + stats/log
|
||||
2. `02_tree_api.md` — `GET /api/docs/tree` (pure builder + schemas + endpoint)
|
||||
3. `03_folder_summary_api.md` — `PATCH /api/folders/summary` (update / create / clear)
|
||||
4. `04_tree_ui.md` — the RAG view tree: level rendering, drill navigation, breadcrumb, level block, folder table, reload fallback
|
||||
5. `05_folder_edit_ui.md` — the edit affordance (level block + rows) + PATCH wiring + editor + status + styles + source pins
|
||||
6. `06_e2e_kb_tree.md` — the dedicated Playwright suite `tests/e2e/test_kb_tree.py`
|
||||
7. `07_catalog_suites_import_sync.md` — the existing import/sync/upload story suites onto the drill-down catalog
|
||||
8. `08_catalog_suites_viewer_nav.md` — the existing viewer/nav/UX story suites + the regression run
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `build_kb_tree` (multi-source order, indexed-only sources, a 0-document registered source, nested counts, the existence rule, a file/folder name collision, ordering, summaries present/absent, file metadata verbatim) + the `group_folder_listing` cross-check; the generator's keep/keep-out (a manual row survives regeneration AND prune, stats, the logger line); migration 0018 up/down (A13).
|
||||
- Integration: the tree endpoint (shape, ordering, 403 anonymous, empty registry, indexed-only + orphan visibility per the superset rule, the stat-walk equivalence); the PATCH endpoint (update, create where no row exists, source root, clear + double-clear, 404 unknown source/folder/traversal, 403 non-admin) — the `tests/integration/test_docs_api.py` pattern; the `import_docs` line keeps its 3-field `folder_summaries=` token with a manual row present.
|
||||
- Frontend source pins (house style, the `tests/unit/test_save_chat_ui.py` pattern): `sources.js` (the `/api/docs/tree` fetch, the drill-state transitions, the PATCH path + body, the textContent contract, the status copies, the reset-to-top fallback, the three refresh wirings), `styles.css` (the new class names).
|
||||
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` in isolation; the updated story suites green in isolation (the task 07/08 lists), including `test_ls_tree_drilldown.py`, `test_edit_summaries.py`, `test_import_documents.py`.
|
||||
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The RAG view (admin) lists the sources with counts + descriptions; clicking through shows, per level, the subfolders (count + description) and the level's files (existing 5-column table, the document-modal link unchanged)
|
||||
- [ ] The level block shows the current directory's description; Edit → inline textarea → Save round-trips to `PATCH /api/folders/summary` (the new text renders; an empty save clears it — the row is deleted)
|
||||
- [ ] A manual description survives a KB-changing sync (the generator's `kept_manual`), and clearing it resets the folder to AI generation on the next sync
|
||||
- [ ] The agent's `ls` output is byte-identical for the same data (the phase-94 suites green) — the tree reads the same rows; the only schema change is the additive flag column
|
||||
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
|
||||
- [ ] `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the updated story suites green in isolation
|
||||
- [ ] One atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(kb): show the catalog as the drill-down tree the agent sees, with editable folder descriptions`)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 01 — Manual-edit flag: migration 0018 + the generator keeps the owner's descriptions
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
A stored folder description the owner edited can never be overwritten (or pruned) by the sync-time generator — `folder_summaries.manually_edited` (migration 0018) plus the phase-94 generator's skip/keep logic.
|
||||
|
||||
## Work
|
||||
1. `alembic/versions/0018_folder_summary_manual_flag.py` (down revision `0017_folder_summaries` — confirm with `uv run alembic heads`) — add `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; the **tested downgrade** drops the column (A13 — the `tests/integration/test_migration_0017.py` pattern: a new `tests/integration/test_migration_0018.py`).
|
||||
2. `app/models.py` — `FolderSummary.manually_edited: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))` + the docstring: set ONLY by `PATCH /api/folders/summary` (phase 97, task 03); the generator (below) skips a manual row on regeneration and never prunes it — an owner correction is never silently rewritten (the phase-97 `00_phase.md` decision).
|
||||
3. `app/rag/folder_summaries.py` — `generate_folder_summaries`:
|
||||
- Fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into a single `{(source, folder_path): FolderSummary}` dict used by both loops).
|
||||
- Upsert loop: an existing row with `manually_edited` → SKIP (no `llm.chat` call — no `lite` burn on owner text) and `stats["kept_manual"] += 1`.
|
||||
- Prune loop: delete only rows whose `manually_edited` is False (a manual row for a folder that dropped below 2 documents is KEPT — owner content persists until cleared).
|
||||
- The stats dict gains `"kept_manual": 0`; the logger line becomes `folder_summaries: generated=%d failed=%d pruned=%d kept_manual=%d`.
|
||||
- **The import summary-line token STAYS 3 fields** — `scripts/import_docs.py`'s `folder_summaries=<generated>/<failed>/<pruned>` is unchanged (`tests/integration/test_import_docs_overview.py` pins it; `app/api/sync.py` just logs the dict).
|
||||
4. Docstrings: the module docstring's storage paragraph + the `generate_folder_summaries` docstring note the flag's two rules (skip on regenerate, keep on prune) — the house "module docstrings carry the contracts" rule.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/test_folder_summaries.py` extensions): a manual row survives a regeneration pass (skipped — the fake LLM is never called for it — `kept_manual` right, the row's text/stamp untouched); a manual row survives the prune (its folder drops below 2 documents); a non-manual row is still pruned and still regenerated; `skip=True` no-op unchanged; the logger line format (caplog, 4 fields).
|
||||
- Integration: `test_migration_0018.py` (up: column present, default false, existing rows backfilled false; down: column dropped); an `test_import_docs_overview.py`-pattern case with a manually-edited row inserted directly — the gate fires, the token stays `folder_summaries=<g>/<f>/<p>`, and the manual row's text is unchanged afterward.
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] migration up/down green; the flag round-trips through the model
|
||||
- [ ] the generator never overwrites or prunes a manual row; stats + log carry `kept_manual`; the import token is unchanged
|
||||
- [ ] full test suite green, coverage >90%
|
||||
- [ ] no behavior change in completed work (the phase-94 generator + sync suites green)
|
||||
@@ -0,0 +1,26 @@
|
||||
# Task 02 — `GET /api/docs/tree`: the full recursive tree the UI renders
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
One admin endpoint that returns the catalog as the recursive tree the agent's `ls` walks (with the file metadata the rows and stat cards need) — the RAG view's single fetch (the shape contract is in `00_phase.md`, section "The tree endpoint").
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py` — next to `DocList`: `KbTreeFile` (`path`, `title`, `chunks: int (ge=0)`, `indexed_at: str`), `KbTreeFolder` (`path`, `documents: int (ge=0)`, `summary: str | None`, `children` — recursive union of folder/file nodes), `KbTreeSource` (`name`, `documents: int (ge=0)`, `summary: str | None`, `children`), `KbTree` (`sources: list[KbTreeSource]`). The `00_phase.md` JSON shape is the contract (Pydantic v2 handles the recursive union; `kind` discriminators optional — the JSON keys `kind: "folder" | "file"` must appear as spec'd).
|
||||
2. `app/api/docs.py`:
|
||||
- `build_kb_tree(names, doc_rows, summaries) -> list[KbTreeSource]` — PURE, module-level (unit-testable without a DB). Inputs: `names` = registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order (source, path); `summaries` = `{(source, folder_path): str}` over ALL stored rows. Behavior per `00_phase.md`: sources = `names` first (each always present, even 0 documents), then the distinct indexed sources not in `names` (alphabetical — the superset rule); a folder node exists iff some indexed path starts with `folder + "/"` (the phase-94 existence rule — reuse `app.rag.folder_summaries.folder_of`, never re-derive "folder"); `documents` = recursive subtree count; subfolders in path order, direct files (`folder_of(path) == parent`) in input (catalog) order; file nodes carry `title` / `chunks` / `indexed_at` verbatim; `summary` = the stored row (any row — AI or manual) or null.
|
||||
- `GET /docs/tree` endpoint (`Depends(require_admin)`, `response_model=KbTree`) — compose: `names = list_source_names(db)` (import from `app.rag.agent` — `app/api/chat.py` already imports from that module, no cycle), `doc_rows` = the same outerjoin/grouped query `GET /docs` uses (id excluded is fine — the tree has no document ids), `summaries` = all `folder_summaries` rows for the listed sources; call the builder.
|
||||
- Module docstring: add the new endpoint's contract line (the house docstring convention — one line per endpoint).
|
||||
3. `GET /api/docs` is UNCHANGED (same query, same response — its suites stay green untouched).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/` — new `test_kb_tree_builder.py` or extend the existing docs unit file): multi-source — registry order preserved + an indexed-only source appended alphabetically; a registered 0-document source (`documents: 0`, no children); a nested document counts into the source, every ancestor folder, and its own folder; the existence rule (a folder node only with a true descendant — a file path is never a folder); a file/folder NAME COLLISION (a document sharing a directory's name — both appear: the folder node via its descendants + the file node); subfolder path order + file catalog order; `summary` present/absent; file `chunks` / `indexed_at` / `title` verbatim; an indexed document under an UNLISTED source is impossible (every doc source is listed by construction — the superset rule) while the registry order still leads.
|
||||
- **The cross-check property (the "UI shows what the agent sees" pin):** for a single-source dataset, the builder's root level equals `app.rag.agent.group_folder_listing(source, "", (path, title) rows, {folder: summary})` — same subfolder `(path, count, summary)` triples in order AND the same file `(path, title)` pairs in order (uncapped — pass the full file list; the builder has no 50-line cap, the UI is for humans); repeat for one nested level.
|
||||
- Integration (`tests/integration/test_docs_api.py` extensions, the house pattern — real app + real DB): seed documents across two sources (one with nested folders, one flat) + `folder_summaries` rows → the shape/ordering/counts/summaries; 403 anonymous; empty registry + no documents → `{"sources": []}`; an indexed-only source (documents with a source absent from `git_sources`) lists after the registered ones; the stat-walk equivalence — summing the file nodes' `chunks` + their count equals what `GET /api/docs` returns for the same data (the stat-card values are unchanged by the redesign).
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/docs/tree` returns the `00_phase.md` shape (admin; 403 otherwise); `GET /api/docs` byte-identical in behavior
|
||||
- [ ] the pure builder is unit-pinned, including the `group_folder_listing` cross-check at two levels
|
||||
- [ ] full test suite green, coverage >90%, ruff + pyright clean
|
||||
- [ ] no behavior change in completed work (the phase-94 ls suites green)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — `PATCH /api/folders/summary`: the admin folder-description editor endpoint
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The endpoint behind the view's edit affordance: update / create / clear a stored folder description, marking every non-empty save `manually_edited` (task 01's keep/keep-out rules apply from this point on).
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py` — `FolderSummaryUpdate` (`source: str`, `folder_path: str` — `""` = the source root, `summary: str`) + `FolderSummaryResult` (`source`, `folder_path`, `summary: str | None`), next to the phase-57 `SummaryUpdate` / `SummaryResult`.
|
||||
2. `app/api/docs.py` — `PATCH /folders/summary` on the same `kb` router, next to the phase-57 `PATCH /documents/summary` (the house docstring line for it):
|
||||
- `Depends(require_admin)` — the catalog is admin-only (phase 16); the endpoint gate is the API-level defense in depth (the RAG view never renders for anonymous).
|
||||
- **Source check:** the source is registered (`list_source_names(db)`) OR has indexed documents (`SELECT 1 FROM documents WHERE source = … LIMIT 1`) → else 404 `{"detail": "source not found"}`.
|
||||
- **Folder check:** `folder_path == ""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some `Document.path` starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem access).
|
||||
- **Non-empty after `strip()`** → upsert the `folder_summaries` row with `summary = stripped text`, `manually_edited = True`, fresh UTC `updated_at` (insert or update — a manual description can be created where no row exists: a < 2-document folder, or the generator's fail-soft miss).
|
||||
- **Empty/whitespace** → `db.delete(row)` when a row exists (clear — the phase-57 analog; the row may be AI-written or manual, either way it is gone; the next KB-changing sync regenerates an AI row — the reset path). A clear with no row is a 200 no-op.
|
||||
- `db.commit()`; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
|
||||
- **No LLM/embedding call** — a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line); the docstring records the deliberate contrast with phase 57's `is_summary` re-embed.
|
||||
3. Router registration needs no change (same router object as `/docs` / `/documents/summary`).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration (`tests/integration/test_docs_api.py` extensions): update an existing AI-written row (text replaced, `manually_edited` true, `updated_at` advanced — a second save with a different text updates in place); create where no row exists (< 2-document folder); the source root (`folder_path: ""`) round-trips; clear (the row is deleted, the response `summary` null, a second clear is a 200 no-op); 404 unknown source / unknown folder / a traversal folder path / a folder under an unknown source; 403 anonymous AND a live access-token user who is not the admin (`require_admin` — the phase-79 token users exist); the round-trip through the task-02 endpoint — `GET /api/docs/tree` shows the new text after the save and null after the clear.
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the endpoint contract is pinned (update / create / source-root / clear / double-clear / 404s / 403s)
|
||||
- [ ] every non-empty save sets `manually_edited` — the task-01 generator keep/keep-out applies from this point on
|
||||
- [ ] no LLM call on the path — source pin: the PATCH handler never constructs an `LLMClient` (grep pin, the house source-pin pattern)
|
||||
- [ ] full test suite green, coverage >90%, ruff + pyright clean
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 04 — The RAG view renders the tree: drill navigation, breadcrumb, level block, folder table
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The Knowledge base view (admin) shows the catalog as the drill-down tree — sources at the top, then per level the subfolders (count + description) and the level's files — replacing the flat all-documents table; descriptions are read-only here (task 05 adds the edit affordance).
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` `#view-rag` — static skeleton additions (the house no-JS-safe skeleton convention), after `#stat-cards`, before the existing file `.table-wrap`:
|
||||
- `nav#kb-crumb[aria-label="Catalog location"][hidden]` — empty; the JS fills the segments (createElement).
|
||||
- `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title`, `p#kb-level-summary` (the level Edit button lands in task 05 — do NOT add it here).
|
||||
- `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table.kb-folders-table` (a `.visually-hidden` caption, the `.docs-table` styling) — `thead` **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled). ONE table for every level — at the top level the rows are the SOURCES themselves (the `ls()` equivalence: sources list exactly like folders — name, count, description).
|
||||
- The existing `.table-wrap` / `#docs-table` / `#docs-tbody` block, `#sources-empty`, `#stat-cards`, the sync button/result/banner/gate: UNCHANGED.
|
||||
2. `frontend/assets/sources.js` — the catalog load becomes tree-based (the module header docstring gains the phase-97 section, the house convention):
|
||||
- `loadDocs()` → `loadTree()`: `fetch("/api/docs/tree")` in the admin branch only (the anonymous gate branch NEVER fetches — the phase-16 soft rule, unchanged); keep the `loadSeq` race token (phase 79) — only the newest load may touch the DOM; store the parsed tree in a module-level `kbTree`; render the current level.
|
||||
- `current = {source: string | null, folder: string | null}` — module state, initial `{null, null}` = top level (`folder ""` = the source root).
|
||||
- `renderLevel()` from `current` + `kbTree`:
|
||||
- **Top level:** breadcrumb hidden; level block hidden; folders table = the source rows (name in the Folder cell, recursive count, the `(source, "")` description text when stored); the file table HIDDEN (files are seen per source — `ls()` shows no files at the top).
|
||||
- **Inside a source/folder:** breadcrumb segments (the source name, then the folder chain — each an `<a class="kb-crumb-link">`, the last segment a `<span aria-current="page">`); level block = the current level's stored description (title = the full source-relative path, e.g. `alpha/two`) — HIDDEN when none is stored (the `ls` rule: count only, no placeholder); folders table = the direct subfolders (`.folder-link` name, count, stored description text); file table = the direct files — `makeRow` UNCHANGED (the path link still `preventDefault`s + `openDocumentModal`; the no-JS href escape hatch intact); `#folders-wrap` hidden when the level has no subfolders, the file wrap hidden when it has no direct files.
|
||||
- **Stat cards:** walk the WHOLE tree (document count, chunks sum, max `indexed_at`) — the values identical to today's flat walk (`fmtDate` reuse).
|
||||
- **Empty state:** `#sources-empty` visible ONLY when `kbTree.sources` is empty (both wraps hidden); a registered 0-document source renders its row (`0`) instead — the semantic change is deliberate (the `ls` invariant), noted in the module docstring.
|
||||
- **Navigation (client-side, no fetch, no URL change):** source/folder link click → set `current`, re-render; breadcrumb link click → that ancestor; the top-level breadcrumb (if ever rendered) → `{null, null}`.
|
||||
- **Refresh wirings — the `loadDocs` call sites become `loadTree` (same points, same semantics):** the `bor:view-refresh` listener (admin branch), `applySyncSuccess` (the KB just changed), the upload-success branch of `startSyncPolling`. **Never-stale fallback (PLAN §7.4):** after a re-fetch, if `current.source` is no longer in the tree's sources, or `current.folder` no longer exists under it, reset `current = {null, null}` BEFORE rendering — no stale breadcrumb, no stale block.
|
||||
- The sync button / poll / label / banner / error-modal / gate code is otherwise UNTOUCHED.
|
||||
3. `frontend/assets/styles.css` — `.kb-crumb` (+ `.kb-crumb-link`, the `aria-current` segment styling), `.kb-level` (+ title/summary typography), `.kb-folders-table` (reuse the `.docs-table` rules), `.folder-link` (row drill link — hover/focus affordance, ≥ 24px target) — house dark-tech palette, AA contrast, no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, no CDN.
|
||||
|
||||
## Testing & Quality
|
||||
- Frontend source pins (house pattern — new `tests/unit/test_kb_tree_ui.py`, mirroring `tests/unit/test_sync_button.py`'s sources.js pins): the exact `/api/docs/tree` fetch path; the `#kb-crumb` / `#kb-level` / `#folders-tbody` / `#kb-level-title` ids referenced; the drill-state transitions (source click, folder click, breadcrumb up, top reset); the textContent contract (no `innerHTML` with document-derived data — the module's standing rule); the empty-state semantic (`sources` empty → `#sources-empty`); the reset-to-top fallback on a vanished location; `loadTree` wired at exactly the three refresh points (view-refresh, sync success, upload success) — and the anonymous branch still never fetches.
|
||||
- The existing sources.js / sync-button / router pins stay green (the untouched machinery).
|
||||
- Coverage: the pins are the unit gate; the `app/` gate is untouched (no backend in this task).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] an admin RAG view lists the sources (count + description when stored); clicking through drills (the breadcrumb goes up); the level block shows the current directory's description; the level's files render as the existing 5-column rows with the modal link
|
||||
- [ ] the stat cards / empty state / sync machinery behave as today (the untouched pins green); anonymous still gets the gate with zero catalog fetches
|
||||
- [ ] a re-fetch after the current location vanished resets to the top level (pinned)
|
||||
- [ ] `uv run pytest` green (the pins); `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] no backend change; the phase-94 + phase-57 suites green
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 05 — Folder description editor: the phase-57 affordance on the level block + the rows
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The owner edits (or clears) any directory's description with the exact interaction of the file-summary editor — Edit button → inline textarea (prefilled) → Save/Cancel → live-region status — wired to `PATCH /api/folders/summary` (task 03).
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` — inside `#kb-level`: `button.kb-summary-edit#kb-level-edit` (type=button, the phase-57 label "Edit", ≥ 24px target) after `p#kb-level-summary`.
|
||||
2. `frontend/assets/sources.js` — ONE shared editor function (mirrors the phase-57 wiring in `frontend/assets/document.js`; this module never builds HTML from document-derived data — the editor parts are static createElement, the description text is textContent only):
|
||||
- **Edit buttons:** on `#kb-level` (the static button) AND on every source/folder row's Description cell (created with the row in task 04's row builder — `.kb-summary-edit`). Always present — a description can be CREATED where none is stored (a < 2-document folder, or the generator's fail-soft miss; the editor opens prefilled with the empty string). No whoami gate in the view (the RAG view is admin-only already — the phase-16 gate; the endpoint's `require_admin` is the API-level gate).
|
||||
- **On Edit:** swap the description text node (level block `p` content / row cell content) for: `textarea.kb-summary-editor` (current value via `.value` — the XSS contract), `button.kb-summary-save` "Save", `button.kb-summary-cancel` "Cancel", `p.kb-summary-status[role="status"][aria-live="polite"]`; focus the textarea.
|
||||
- **Save** → `PATCH /api/folders/summary` with `{"source", "folder_path", "summary"}` — `folder_path` is `""` for the source root, the source-relative folder path otherwise (both known from `current` / the row data).
|
||||
- 200 → re-render the description text (textContent only) — the level block AND/OR the row cell where the edit happened — + status "Description updated."; an empty save → the text goes away (level block hidden when empty / row cell emptied) + status "Description cleared."; update the in-memory `kbTree` node's `summary` in place (no re-fetch — the tree state stays coherent; the reload fallback is the safety net).
|
||||
- Failure (non-2xx / network) → neutral retry copy (the phase-55 convention — e.g. "Couldn't save the description — try again."), the editor stays open with the user's text, the stored text is untouched.
|
||||
- **Cancel** → restore the text node (and drop the editor + status).
|
||||
3. `frontend/assets/styles.css` — `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem, the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house palette, AA contrast, `:focus-visible` via the global rule, no CDN.
|
||||
4. Module header docstring: the phase-97 edit section (the house per-phase note convention).
|
||||
|
||||
## Testing & Quality
|
||||
- Source pins (`tests/unit/test_kb_tree_ui.py` extensions): the exact PATCH path + body shape (`folder_path ""` for the source root); the editor element construction (textarea `.value` prefill, Save/Cancel, the live-region `role="status"`); the status copies "Description updated." / "Description cleared."; the in-place `kbTree` update on success; the failure path keeps the editor (pinned via the status-copy + open-editor assertions, the phase-55 pin style); textContent re-render (no `innerHTML` on the description); the `styles.css` class names (the five `.kb-summary-*` + `.kb-summary-edit` if distinct).
|
||||
- Coverage: the pins are the unit gate; no backend change.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Edit → Save round-trips: the new text renders in the row and/or level block, the server row is `manually_edited` (the task-03 integration pins cover the flag)
|
||||
- [ ] an empty save clears (the text is gone, "Description cleared."); Cancel restores; a failure keeps the editor + shows neutral copy
|
||||
- [ ] a description can be created where none was stored (the editor opens prefilled empty; the save creates the row — task-03 pin)
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 06 — The dedicated story suite: `tests/e2e/test_kb_tree.py`
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The phase's Playwright gate (A16 — one dedicated E2E file, run in isolation): the drill-down tree, the directory descriptions, and the edit/clear round-trip pinned against the real app + the deterministic mock.
|
||||
|
||||
## Work
|
||||
`tests/e2e/test_kb_tree.py` — module docstring per the house pattern (story mapping, the run command, the fixture description, the mock notes):
|
||||
- **Fixture** (the phase-94 suite's pattern — a host temp-dir tree via `tmp_path_factory`, TWO local sources registered through the authenticated API, the real in-process `POST /api/sync` pipeline; the mock's existing `FOLDER_SUMMARY_MODE` branch stores the canned one-liner naming the folder, so every stored description is deterministic):
|
||||
- `alpha/` — `root-note.md` at the source root, `one/` (2 docs), `two/` (2 docs);
|
||||
- `beta/` — `gamma/` (2 docs).
|
||||
- Total: 7 documents; alpha counts 5 (root + 2 + 2), beta counts 2.
|
||||
- **Tests** (Playwright Mapping Rule — each test is one distinct observable):
|
||||
1. `test_top_level_lists_sources_with_descriptions` — the source rows: `alpha` (5) + `beta` (2) in registry order with the canned source-root descriptions; the top-level file table is HIDDEN (files are per-source — the `ls()` equivalence); the stat cards read 7 documents + the computed chunks total.
|
||||
2. `test_drill_into_source` — click the `alpha` row → breadcrumb `alpha` (aria-current); the level block shows alpha's root description (the canned text); the folder rows `one` / `two` each with count 2 + their canned summaries; the file row `root-note.md` (title + chunks column) in the file table.
|
||||
3. `test_drill_into_folder` — drill to `alpha` → `two` → breadcrumb `alpha` → `two`; the level block shows `two`'s description; the file rows `two-a` / `two-b` (titles, the Source column `alpha`); `#folders-wrap` hidden (no subfolders); the breadcrumb link on `alpha` goes back up to the source level.
|
||||
4. `test_edit_folder_description` — on the `alpha` level, click Edit on the `one/` row (or the level block for `alpha` — the executor's pick, pinned in the test) → the textarea prefilled with the canned text → set the new text → Save → the new text renders in the row/block + status "Description updated."; a SQL assert (the `SessionLocal` house pattern) on `folder_summaries`: the row's `summary` is the new text AND `manually_edited` is true.
|
||||
5. `test_clear_folder_description` — an empty save → the text is gone (row cell emptied / level block hidden) + status "Description cleared."; the SQL assert: no row for the folder.
|
||||
6. `test_manual_description_survives_a_changed_sync` — edit a description (test-4 style); add a new file to the fixture dir; `POST /api/sync` (the mock regenerates the OTHER folders' summaries) → the edited description is UNCHANGED (the catalog re-fetch renders it; the SQL row keeps the manual text + the flag) while the untouched folders show the canned regenerated text (the `kept_manual` path, E2E-pinned).
|
||||
7. `test_reload_falls_back_to_top_level` — drill to `alpha/two`; delete `two/`'s documents + chunks directly (the house DB pattern); re-show the RAG view (the nav link — the `bor:view-refresh` trigger) → the breadcrumb is hidden and the top level renders (the never-stale contract, PLAN §7.4).
|
||||
8. `test_anonymous_sees_the_gate` — the RAG view for an anonymous visitor: the sign-in gate visible, no folders/file table, no Edit affordance, and no `/api/docs/tree` request (the phase-16 soft rule — assert via the response/locator state, the existing suites' pattern).
|
||||
- **MOCK note:** no mock changes needed (the `FOLDER_SUMMARY_MODE` branch is phase 94's). The suite is mock-only in the standard sense — `E2E_REAL_LLM=1` keeps the canned summaries unavailable, so the suite follows the phase-94 convention (its assertions key on the canned text; the module docstring states the requirement).
|
||||
|
||||
## Testing & Quality
|
||||
- Run in isolation (DB up): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov`.
|
||||
- The suite must not modify app/frontend source to pass — it asserts on the shipped behavior of tasks 01–05.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the suite green in isolation; the test → observable mapping documented in the module docstring
|
||||
- [ ] the edit/clear/survive-sync/fallback observables all pinned (tests 4–7)
|
||||
- [ ] no changes to completed-phase code beyond what tasks 01–05 shipped
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 07 — The existing import / sync / upload story suites onto the drill-down catalog
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The story suites that assert the catalog through the old flat table keep asserting the SAME behavior (a document is listed, linkable, viewable; the stat cards hold) — with a drill step where the asserted path lives under a folder or source. No product behavior changes; navigation only.
|
||||
|
||||
## Work
|
||||
- **The recipe** (apply per suite — mechanical, assertion intent unchanged):
|
||||
1. **Nested-path row assertions** (`#docs-tbody tr` has_text a path with `/`): before asserting, drill — click the source row (top level), then each folder link (`#folders-tbody .folder-link`) one per path segment, until the level containing the asserted file renders. A small local drill helper per suite (a few lines: `for seg in path_segments: page.locator(f"#folders-tbody .folder-link:has-text('{seg}')").click()`) is fine — the house style is per-suite helpers, no new shared module unless two suites need the identical one.
|
||||
2. **Total row counts** (`#docs-tbody tr` to_have_count N over a whole KB): the flat total no longer exists in one tbody — re-assert per level (drill to each folder, assert that level's count; the sum is implied) or assert the stat cards (`#stat-docs` still holds the KB total — unchanged).
|
||||
3. **`.first` visible waits** (the catalog-rendered signal): the top-level file table is now always hidden (files are per-source — the `ls()` equivalence) — drill into a source (and folder, if all its files are nested) BEFORE the wait; the source/folder row itself is a valid "the catalog rendered" signal where the intent is just that.
|
||||
4. **Empty-state assertions:** unchanged semantics where the registry is empty (`#sources-empty`); where a source IS registered but the KB is cleared, `#docs-tbody` is still 0 rows (the file table hides; the source row renders `0` — adjust any `#sources-empty` expectation in that state if present).
|
||||
5. Suites that seed by direct `import_sources` (no registry row) work unchanged in principle — their source appears as an indexed-only source row (the `00_phase.md` superset rule); the drill just clicks that row.
|
||||
- **Suites (the known set from a source scan — the executor runs each in isolation and fixes whatever is red; the list is a starting point, not a guarantee):**
|
||||
- `test_import_documents.py` — EXPECTED_ROWS all nested under `homelab/…` + `deployments/…`: drill per folder for each row; the 13-stat + chunk/last-indexed cards; the hidden-junk absence (scoped after a drill); the empty-state case (registry empty — should already pass).
|
||||
- `test_admin_auth.py` — the 13 row count (per-level re-assert or the stat cards).
|
||||
- `test_sticky_navbar.py` — the TOTAL_DOCS count (same).
|
||||
- `test_sync_upload_progress.py` — the N_FILES counts + the `docs/*.md` rows (drill `docs`).
|
||||
- `test_upload_no_scan.py` — the `notes/…` rows + counts (drill `notes`; the `skipme.md` absence).
|
||||
- `test_sync_button.py`, `test_sync_model_down.py` — the FIXTURE_DOC row (drill as needed).
|
||||
- `test_import_extensions_env.py` — the SH_REL / MD_REL rows (drill).
|
||||
- `test_quadlet_jinja_import.py` — the `homelab/quadlet/…` rows (drill).
|
||||
- `test_retrieval_quality.py` — the `.hidden` absence (scoped after a drill).
|
||||
- `test_archive_upload_sources.py` — the cleared-state row count (the empty-state semantic, point 4).
|
||||
- anything else that turns red in the regression run (the full E2E list — the AGENTS.md rule-9 gate).
|
||||
- **Do NOT change the asserted product behavior.** If a suite's assertion cannot be preserved through the drill (a genuine behavior change, not a navigation one), stop and flag it in the phase record — no silent deviation (AGENTS.md rule 3 spirit).
|
||||
|
||||
## Testing & Quality
|
||||
- Each updated suite green in isolation: `uv run pytest tests/e2e/test_<suite>.py -v --no-cov` (DB up).
|
||||
- Coverage: n/a (tests-only task) — the `app/` gate is untouched.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] every suite in the list green in isolation; the asserted document behavior preserved (the drill is the only change)
|
||||
- [ ] no app/frontend source changes from this task (tests only)
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 08 — The existing viewer / nav / UX story suites + the regression run
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The remaining story suites that touch the catalog table (document viewer, navigation, UX) are green on the tree view, and the phase's full validation gate is run end to end.
|
||||
|
||||
## Work
|
||||
- **The same recipe as task 07** (drill before asserting nested rows; per-level counts; drill before `.first` waits; the empty-state semantic) for:
|
||||
- `test_document_viewer.py` — the `gitlab-compose.yaml` / `xss-fixture.md` rows (drill, then the row link opens the modal — the viewer flow itself is unchanged).
|
||||
- `test_document_back_navigation.py` — the `kubernetes.md` row (drill).
|
||||
- `test_edit_summaries.py` — the DOC_PATH row (drill — the phase-57 summary-edit flow on the VIEWER is untouched by this phase; only the catalog navigation to reach the doc changes).
|
||||
- `test_summary_in_viewer.py` — the YAML/MD rows (drill).
|
||||
- `test_markdown_tables.py` — the TABLES_PATH row (drill).
|
||||
- `test_responsive_polish.py` — the `backup_rotation` row + the desktop/mobile `.first` waits (drill).
|
||||
- the `.first`-visible waits against all-nested KBs: `test_chat_persistence.py`, `test_navbar_refresh.py`, `test_nav_consistency.py`, `test_nav_switch_keeps_stream.py`, `test_shared_header.py`, `test_sources_midstream_bug.py` — drill to a level that has a file before the wait (or assert the source/folder row where the intent is just "the catalog rendered").
|
||||
- anything else that turns red in the regression run.
|
||||
- **The regression run (the phase gate, the AGENTS.md rule-9 commands):**
|
||||
1. `uv run pytest --cov=app --cov-report=term-missing` — unit + integration green, TOTAL > 90%.
|
||||
2. `uv run ruff check . && uv run pyright` — clean.
|
||||
3. The catalog-touching E2E list in isolation — at minimum the task 07/08 suites above PLUS the completed phases' suites this phase's area of change can touch: `test_ls_tree_drilldown.py` (the agent's `ls` — must be byte-identical for the same data), `test_kb_overview.py`, `test_edit_summaries.py`, `test_import_documents.py`, `test_sync_button.py`, `test_admin_auth.py`.
|
||||
4. `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` (task 06's suite) green in isolation.
|
||||
5. The full `tests/e2e/` sweep in isolation for any suite the run above did not cover and that references the RAG view — the final pass (the `00_phase.md` completion criteria) is the authority; this task leaves nothing red.
|
||||
|
||||
## Testing & Quality
|
||||
- Each updated suite green in isolation; the gates above green.
|
||||
- Coverage: the `app/` TOTAL > 90% (this task is tests-only — the coverage is the tasks 01–03 code, already gated).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] every suite in the list green in isolation; the asserted behavior preserved (drill = the only change)
|
||||
- [ ] unit + integration green, TOTAL coverage > 90%, ruff + pyright clean
|
||||
- [ ] `test_ls_tree_drilldown.py` green — the agent's tree is byte-identical (the only schema change is the additive flag column)
|
||||
- [ ] no app/frontend source changes from this task (tests only)
|
||||
@@ -0,0 +1,17 @@
|
||||
All checks complete. Final report:
|
||||
|
||||
**Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design)
|
||||
|
||||
- `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff
|
||||
- `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe
|
||||
- `.env.example` comments updated (chat-turn stream + one-shot summary calls)
|
||||
|
||||
**Test / lint / coverage results**
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%)
|
||||
- `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated)
|
||||
- Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed
|
||||
- `uv run ruff check . && uv run pyright` → clean (0 errors)
|
||||
|
||||
**Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files).
|
||||
|
||||
**Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.
|
||||
@@ -0,0 +1,95 @@
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
............................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 181 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 94 0 100%
|
||||
app/api/docs.py 50 0 100%
|
||||
app/api/git_sources.py 229 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 29 0 100%
|
||||
app/api/sync.py 115 0 100%
|
||||
app/api/tokens.py 28 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 140 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 33 0 100%
|
||||
app/db.py 21 0 100%
|
||||
app/main.py 66 0 100%
|
||||
app/models.py 124 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 316 1 99%
|
||||
app/rag/archive_upload.py 128 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/folder_summaries.py 120 0 100%
|
||||
app/rag/git_sources.py 14 0 100%
|
||||
app/rag/importer.py 190 3 98%
|
||||
app/rag/llm.py 243 1 99%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 88 0 100%
|
||||
app/rag/retriever.py 150 3 98%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 24 0 100%
|
||||
app/schemas.py 279 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 3718 14 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,10 @@
|
||||
All checks pass. The diff is confined to `_EmptyContentError` + `_chat_once` + `chat()` in `app/rag/llm.py`, the `.env.example` comments, and the unit test file — the streaming path is byte-identical.
|
||||
|
||||
**Task 01 report**
|
||||
|
||||
- `LLMClient.chat()` now retries empty-content replies (the 2026-09-11 incident class) under the existing `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` policy: single-attempt body extracted verbatim into private `_chat_once()`; empty replies raise internal `_EmptyContentError` (carries `finish_reason`, never escapes the client); `chat()` logs one `WARNING` per retry (model, `finish_reason`, `attempt n/N`) + flat sleep, then raises `LLMError("…returned empty content on all N attempts — refusing to store a silent summary")`; `llm_retries=0` keeps the pre-phase-96 message byte-identical. Choiceless/transport errors still raise immediately (no app-level retry).
|
||||
- `chat()` docstring extended with the D1–D3 retry policy; `.env.example` comments on both knobs now name the phase-67 stream + phase-96 one-shot coverage. No new settings.
|
||||
- Unit pins added in `tests/unit/test_llm_client.py`: empty-then-success (2 attempts, 1 sleep, 1 warning with `finish_reason=length`/`attempt 1/4`), all-empty exhaustion (4 attempts/3 sleeps + exact message; `llm_retries=1` names 2), first-success (1 call, 0 sleeps), no-choices and transport (no retry), `llm_retries=0` legacy message asserted as exact string, flat-delay (never backoff), explicit-model warning, `finish_reason=None` log safety; the two pre-existing empty-content pins moved to the kill-switch setting.
|
||||
- Tests: `uv run pytest tests/unit/test_llm_client.py -v` → 66 passed; full `uv run pytest -v` → **1975 passed**, 0 failed; `uv run pytest --cov=app` → app/ 99% (llm.py 99%, only the unreachable guard line uncovered); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings.
|
||||
- Phase-67 streaming retry suite (12 tests) green; no changes outside `chat()`/`_chat_once` in `app/rag/llm.py`. Changes left uncommitted in the working tree per harness protocol.
|
||||
- Next pending task: `.agents/phases/todo/96_oneshot_resilience/02_gapfill_generator.md`
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 98%]
|
||||
............................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 181 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 94 0 100%
|
||||
app/api/docs.py 50 0 100%
|
||||
app/api/git_sources.py 229 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 29 0 100%
|
||||
app/api/sync.py 110 0 100%
|
||||
app/api/tokens.py 28 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 140 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 33 0 100%
|
||||
app/db.py 21 0 100%
|
||||
app/main.py 66 0 100%
|
||||
app/models.py 124 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 316 1 99%
|
||||
app/rag/archive_upload.py 128 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/folder_summaries.py 110 0 100%
|
||||
app/rag/git_sources.py 14 0 100%
|
||||
app/rag/importer.py 190 3 98%
|
||||
app/rag/llm.py 243 1 99%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 88 0 100%
|
||||
app/rag/retriever.py 150 3 98%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 24 0 100%
|
||||
app/schemas.py 279 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 3703 14 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
All deletions are the intended refactor; nothing else in the module changed. Task 02 is complete and green.
|
||||
|
||||
## Report — Phase 96, Task 02 (gap-fill generator)
|
||||
|
||||
**Implemented**
|
||||
- `app/rag/folder_summaries.py`: new `missing_folder_summaries(db)` — candidate folders (generator's exact catalogue query → `group_by_folder` → ≥`MIN_DOCS_PER_FOLDER`) minus stored keys (one bounded select), sorted `(source, folder_path)`, `[]` on no gap
|
||||
- `generate_folder_summaries(…, only_missing=False)`: restricts iteration to missing keys derived from the same catalogue pass (one stored-key select, no second catalogue query); existing rows byte-identical (text AND `updated_at`), prune pass runs in both modes vs the full candidate set; default path behavior-identical
|
||||
- Private helpers `_catalog_rows` / `_candidates` / `_stored_keys` so gate and fill share one concept; `folder_summary_table_empty` left in place (task 03 removes it); docstrings updated per task contract
|
||||
- `tests/unit/test_folder_summaries.py`: +10 pins — missing-set shapes (fresh/full/one-deleted/empty-KB/single-doc/stale-row) and `only_missing` (exact fill in sorted order with `updated_at` byte-identity, zero-burn no-gap, stale-row prune, fail-soft); all 32 pre-existing tests unmodified
|
||||
|
||||
**Results**
|
||||
- `uv run pytest tests/unit/test_folder_summaries.py -v --no-cov` → 42 passed
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → 1985 passed, TOTAL 99% (>90%); `folder_summaries.py` 100%
|
||||
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
|
||||
|
||||
**Decisions**: prune compares against the full candidate map (not the restricted key set) so `only_missing` can never wrongly delete present rows — empirically verified `db.get` returns `None` for behind-the-session deletes, and sync paths use fresh sessions.
|
||||
|
||||
**Next pending task**: `03_sync_gates.md` (gap gate in `scripts/import_docs.py` + `app/api/sync.py`).
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
......................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 181 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 94 0 100%
|
||||
app/api/docs.py 50 0 100%
|
||||
app/api/git_sources.py 229 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 29 0 100%
|
||||
app/api/sync.py 110 0 100%
|
||||
app/api/tokens.py 28 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 140 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 33 0 100%
|
||||
app/db.py 21 0 100%
|
||||
app/main.py 66 0 100%
|
||||
app/models.py 124 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 316 1 99%
|
||||
app/rag/archive_upload.py 128 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/folder_summaries.py 122 0 100%
|
||||
app/rag/git_sources.py 14 0 100%
|
||||
app/rag/importer.py 190 3 98%
|
||||
app/rag/llm.py 243 1 99%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 88 0 100%
|
||||
app/rag/retriever.py 150 3 98%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 24 0 100%
|
||||
app/schemas.py 279 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 3715 14 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,11 @@
|
||||
All completion criteria met. Final report:
|
||||
|
||||
**Task 03 (sync gates) — complete**
|
||||
|
||||
- **`scripts/import_docs.py`**: unchanged-walk gate now `folders_due = bool(_folder_summaries_gap())` (task 02's `missing_folder_summaries`); gap → `generate_folder_summaries(…, only_missing=True)`; summary line token gains ` (gap-fill)` suffix on the targeted-fill path (changed runs and `skipped` rendering byte-identical to phase 94); `--limit` skip, module + `_run` docstrings updated.
|
||||
- **`app/api/sync.py`**: unchanged resync → gap probe; non-empty → `only_missing=True` fill + `sync: folder_summaries gap-fill missing=… stats=…` log; empty → existing skip log; changed → full regen (unchanged); module + block docstrings updated; `sync_status` shape untouched.
|
||||
- **`app/rag/folder_summaries.py`**: `folder_summary_table_empty` deleted — grep confirms zero remaining references; imports replaced in both sync modules.
|
||||
- **Tests**: new integration pins — script unchanged+gap (exactly 1 `FOLDER_SUMMARY_MODE` call, other rows byte-identical incl. `updated_at`, `(gap-fill)` token), script changed→full (call count == candidate count, all re-stamped), API unchanged+gap (targeted fill + gap-fill log, detail key set pinned); first-run subsumption pins now assert the `(gap-fill)` path; unit gate test moved to `missing_folder_summaries`.
|
||||
- **Defects fixed in completed work**: stale `generate_folder_summaries` fakes in `test_import_docs_git.py` (3 failures), `test_sync_api.py` (`FakeFolderSummaries` + `only_missing_flags`), and `test_sync_button.py` (signature + DB-free `missing_folder_summaries` seam).
|
||||
- **Results**: `uv run pytest --cov=app --cov-report=term-missing` → 1988 passed, TOTAL 99% (`app/api/sync.py` 100%, `app/rag/folder_summaries.py` 100%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. Phase-94/53 contracts (zero-burn, bump gating, `--limit` skip, status shape) verified green.
|
||||
- **Deviations**: none. Next pending task: `04_e2e_oneshot_retry.md`.
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
............................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 181 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 94 0 100%
|
||||
app/api/docs.py 50 0 100%
|
||||
app/api/git_sources.py 229 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 29 0 100%
|
||||
app/api/sync.py 115 0 100%
|
||||
app/api/tokens.py 28 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 140 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 33 0 100%
|
||||
app/db.py 21 0 100%
|
||||
app/main.py 66 0 100%
|
||||
app/models.py 124 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 316 1 99%
|
||||
app/rag/archive_upload.py 128 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/folder_summaries.py 120 0 100%
|
||||
app/rag/git_sources.py 14 0 100%
|
||||
app/rag/importer.py 190 3 98%
|
||||
app/rag/llm.py 243 1 99%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 88 0 100%
|
||||
app/rag/retriever.py 150 3 98%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 24 0 100%
|
||||
app/schemas.py 279 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 3718 14 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
Task 04 complete — all gates green.
|
||||
|
||||
**Implemented**
|
||||
- `tests/e2e/mock_llm.py`: incident-shape injection in the `FOLDER_SUMMARY_MODE` path — `_folder_summary_incident()` keyed by the `Folder: …` label (existing parse); `/e2e_empty_once` → first non-stream POST returns the 2026-09-11 envelope (`content=""`, `finish_reason="length"`), later POSTs normal; `/e2e_empty_always` → every POST empty; per-label counter resets after the success it guards (phase-67 pattern); module docstring + branch comment updated
|
||||
- `tests/e2e/test_oneshot_llm_retry.py` (new, port 8138): 1 local source, 3 ≥2-doc folders; sync #1 → `ls oneshot` drill-down pins retry-recovered `e2e_empty_once` row, normal row, bare `e2e_empty_always` line (no `: …` suffix), sync success; deleted `normal` row → unchanged sync #2 → row healed, `e2e_empty_once` `updated_at` untouched (targeted fill), `e2e_empty_always` still absent, sync green
|
||||
|
||||
**Results**
|
||||
- `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → 2 passed (re-run green — deterministic reset)
|
||||
- Regressions in isolation: `test_ls_tree_drilldown.py` 3 passed, `test_sync_button.py` 3, `test_local_directory_sources.py` 3, `test_llm_retry.py` 4
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → 1988 passed, TOTAL coverage 99% (>90%)
|
||||
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
|
||||
|
||||
**Decisions**: injection lives in the non-stream response path (where the envelope forms) with the label/counter logic beside the `FOLDER_SUMMARY_MODE` branch; sync driven via httpx (house `test_sync_button`/phase-94 pattern), drill turns via Playwright SSE hook (phase-94 lens). No app-code changes needed (tasks 01–03 already in tree). No commit made — left to the harness per pipeline rules.
|
||||
|
||||
**Next pending task**: none in phase 96 (04 was the last); no other phases in `todo/`.
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
............................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 181 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 94 0 100%
|
||||
app/api/docs.py 50 0 100%
|
||||
app/api/git_sources.py 229 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 29 0 100%
|
||||
app/api/sync.py 115 0 100%
|
||||
app/api/tokens.py 28 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 140 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 33 0 100%
|
||||
app/db.py 21 0 100%
|
||||
app/main.py 66 0 100%
|
||||
app/models.py 124 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 316 1 99%
|
||||
app/rag/archive_upload.py 128 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/folder_summaries.py 120 0 100%
|
||||
app/rag/git_sources.py 14 0 100%
|
||||
app/rag/importer.py 190 3 98%
|
||||
app/rag/llm.py 243 1 99%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 88 0 100%
|
||||
app/rag/retriever.py 150 3 98%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 24 0 100%
|
||||
app/schemas.py 279 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 3718 14 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+2
-2
@@ -20,9 +20,9 @@ BOR_DATABASE_URL=postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese
|
||||
BOR_LLM_BASE_URL=https://aipi.reeseapps.com/v1
|
||||
BOR_LLM_API_KEY= # falls back to $AIPI_KEY, then "not-needed"
|
||||
BOR_LLM_CHAT_MODEL=turbo
|
||||
# BOR_LLM_RETRIES=3 # retry a dead LLM request before the first token lands (phase 67); 0 = off
|
||||
# BOR_LLM_RETRIES=3 # LLM retries: chat-turn stream before the first token (phase 67) + empty one-shot summary replies (phase 96); 0 = off
|
||||
# BOR_LLM_TIMEOUT=300 # HTTP timeout for LLM API calls, seconds (default 120)
|
||||
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67)
|
||||
# BOR_LLM_RETRY_DELAY=5 # seconds between LLM retries (phase 67 chat-turn stream; phase 96 one-shot summary calls)
|
||||
BOR_LLM_EMBED_MODEL=embed
|
||||
BOR_LLM_SUMMARY_MODEL=lite # one-shot completions: document summaries (phase 30), KB overview (phase 31)
|
||||
BOR_EMBEDDING_DIM=768 # verified 2026-08 via scripts/llm_probe.py
|
||||
|
||||
+39
-19
@@ -44,13 +44,17 @@ decisions):
|
||||
(phase 31 trigger, best-effort inside) — and ``generate_
|
||||
folder_summaries`` (phase 94, task 02) regenerates the stored folder
|
||||
summaries (the drill-down ``ls``'s per-level descriptions): its gate
|
||||
is the same change trigger **plus** an empty ``folder_summaries``
|
||||
table (the first sync after migration 0017 — the KB may predate the
|
||||
table). It is per-folder fail-soft (a ``lite`` outage keeps the
|
||||
failed folders' previous rows and never flips the run to
|
||||
``failed``) and only flushes — this run's own short-lived session
|
||||
commits (the phase-53 convention), so a folder failure never blocks
|
||||
step 6's bump;
|
||||
is the same change trigger (full regeneration) **plus**, on an
|
||||
unchanged re-sync, a GAP probe — a candidate folder (at least 2
|
||||
docs) with no stored row (``missing_folder_summaries``, phase 96,
|
||||
task 03): a gap fills ONLY the missing rows (``only_missing=True``;
|
||||
the old table-empty first-sync trigger is subsumed exactly — an
|
||||
empty table leaves every candidate missing), a complete table burns
|
||||
zero ``lite`` calls. It is per-folder fail-soft (a ``lite`` outage
|
||||
keeps the failed folders' previous rows — or leaves the row absent
|
||||
— and never flips the run to ``failed``) and only flushes — this
|
||||
run's own short-lived session commits (the phase-53 convention), so
|
||||
a folder failure never blocks step 6's bump;
|
||||
6. when the import changed the KB (added + updated + pruned > 0 — the
|
||||
saved-chat invalidation gate, phase 53 task 02: a pruned document
|
||||
can invalidate a saved answer that cited it, deliberately broader
|
||||
@@ -88,7 +92,7 @@ from app.config import get_settings
|
||||
from app.core.auth import require_admin
|
||||
from app.core.errors import sanitize_error as _sanitize_error
|
||||
from app.db import SessionLocal
|
||||
from app.rag.folder_summaries import folder_summary_table_empty, generate_folder_summaries
|
||||
from app.rag.folder_summaries import generate_folder_summaries, missing_folder_summaries
|
||||
from app.rag.git_sources import effective_sources
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient, check_models
|
||||
@@ -261,22 +265,38 @@ async def _run_sync() -> None:
|
||||
overview = False
|
||||
if summary.added + summary.updated > 0:
|
||||
overview = await regenerate_overview(llm)
|
||||
# Phase 94 (task 02): the folder summaries — the drill-down
|
||||
# ls's per-level descriptions. Same change gate as the overview
|
||||
# (added + updated > 0), plus the table-empty first-run trigger
|
||||
# (the first sync after migration 0017 — the KB may have been
|
||||
# imported by the CLI before the table landed). The generator is
|
||||
# per-folder fail-soft (a lite outage never flips the run to
|
||||
# failed) and only flushes: this run's own short-lived session
|
||||
# commits (the phase-53 convention), so the step-6 bump stays
|
||||
# change-gated on the KB, not on the summaries. No status-surface
|
||||
# change: the stats are log-only (the detail shape is untouched).
|
||||
# Phase 94 (task 02), phase 96 (task 03): the folder
|
||||
# summaries — the drill-down ls's per-level descriptions. A
|
||||
# changed KB (added + updated > 0) is a full regeneration
|
||||
# (today's behavior, byte-identical); an unchanged re-sync
|
||||
# takes the GAP probe instead of the old table-empty one — a
|
||||
# candidate folder (at least 2 docs) with no stored row: a gap
|
||||
# fills ONLY the missing rows (only_missing=True — the
|
||||
# subsumed table-empty first-sync trigger included, where
|
||||
# every candidate is missing), a complete table burns zero
|
||||
# lite calls. The generator is per-folder fail-soft (a lite
|
||||
# outage never flips the run to failed) and only flushes: this
|
||||
# run's own short-lived session commits (the phase-53
|
||||
# convention), so the step-6 bump stays change-gated on the KB,
|
||||
# not on the summaries. No status-surface change: the stats
|
||||
# are log-only (the detail shape is untouched).
|
||||
fs_db = SessionLocal()
|
||||
try:
|
||||
if summary.added + summary.updated > 0 or folder_summary_table_empty(fs_db):
|
||||
if summary.added + summary.updated > 0:
|
||||
folder_stats = await generate_folder_summaries(fs_db, llm)
|
||||
fs_db.commit()
|
||||
logger.info("sync: folder_summaries stats=%s", folder_stats)
|
||||
else:
|
||||
missing = missing_folder_summaries(fs_db)
|
||||
if missing:
|
||||
folder_stats = await generate_folder_summaries(
|
||||
fs_db, llm, only_missing=True
|
||||
)
|
||||
fs_db.commit()
|
||||
logger.info(
|
||||
"sync: folder_summaries gap-fill missing=%d stats=%s",
|
||||
len(missing), folder_stats,
|
||||
)
|
||||
else:
|
||||
logger.info("sync: folder_summaries skipped (KB unchanged)")
|
||||
finally:
|
||||
|
||||
+106
-28
@@ -33,6 +33,14 @@ Chat turns never generate folder summaries — the agent's ``ls`` output
|
||||
caller's job at sync time (phase 94, task 02), and :func:`generate_
|
||||
folder_summaries` only flushes — the sync path owns the transaction
|
||||
(the phase-53 ``bump_sources_version`` convention).
|
||||
|
||||
Self-heal (phase 96): an exhausted one-shot retry can still leave a
|
||||
candidate folder without a row. :func:`missing_folder_summaries`
|
||||
names those gaps (the generator's exact candidate computation minus
|
||||
the stored keys — one concept, as with :func:`group_by_folder`), and
|
||||
:func:`generate_folder_summaries`'s ``only_missing=True`` fills
|
||||
EXACTLY those on the next sync — existing rows stay
|
||||
byte-identical (text AND ``updated_at``), the prune pass still runs.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -293,22 +301,85 @@ def _upsert(db: Session, source: str, folder_path: str, summary: str) -> None:
|
||||
row.updated_at = now
|
||||
|
||||
|
||||
def folder_summary_table_empty(db: Session) -> bool:
|
||||
"""Whether ``folder_summaries`` holds no rows (the sync-path gate).
|
||||
def _catalog_rows(db: Session) -> list[DocRow]:
|
||||
"""The document catalogue in catalogue order (one query).
|
||||
|
||||
Phase 94 (task 02): the ``_overview_row_exists`` pattern
|
||||
(``scripts.import_docs``) extended to a table-emptiness check —
|
||||
after an unchanged re-sync, an EMPTY table (the first full sync
|
||||
after migration 0017, or after a ``--limit`` debug walk that
|
||||
skipped generation) still gets a fresh batch, while a populated
|
||||
table is left untouched until the KB actually changes. One
|
||||
bounded ``LIMIT 1`` probe, never a count scan.
|
||||
``(source, path, title, summary)`` ordered by ``(source, path)`` —
|
||||
the EXACT query :func:`generate_folder_summaries` runs, so the gap
|
||||
detector and the fill can never disagree about what the catalogue
|
||||
contains (phase 96, task 02: one concept, as with
|
||||
:func:`group_by_folder`).
|
||||
"""
|
||||
return db.execute(select(FolderSummary.source).limit(1)).first() is None
|
||||
result = db.execute(
|
||||
select(Document.source, Document.path, Document.title, Document.summary)
|
||||
.order_by(Document.source, Document.path)
|
||||
).all()
|
||||
return [
|
||||
(source, path, title, summary)
|
||||
for source, path, title, summary in result
|
||||
]
|
||||
|
||||
|
||||
def _candidates(rows: Sequence[DocRow]) -> dict[tuple[str, str], list[DocRow]]:
|
||||
"""The generator's candidate map (recursive subtree ≥ 2 docs).
|
||||
|
||||
The :func:`group_by_folder` groups filtered by
|
||||
:data:`MIN_DOCS_PER_FOLDER` — the EXACT set a full regeneration
|
||||
would cover, so a "missing" folder can never disagree with what a
|
||||
regeneration would (re)generate.
|
||||
"""
|
||||
groups = group_by_folder(rows)
|
||||
return {
|
||||
key: docs for key, docs in groups.items() if len(docs) >= MIN_DOCS_PER_FOLDER
|
||||
}
|
||||
|
||||
|
||||
def _stored_keys(db: Session) -> set[tuple[str, str]]:
|
||||
"""The ``(source, folder_path)`` keys that already hold a row.
|
||||
|
||||
One bounded key select — no count scan (the phase-94
|
||||
table-emptiness probe's house pattern).
|
||||
"""
|
||||
return {
|
||||
(source, folder_path)
|
||||
for source, folder_path in db.execute(
|
||||
select(FolderSummary.source, FolderSummary.folder_path)
|
||||
).all()
|
||||
}
|
||||
|
||||
|
||||
def missing_folder_summaries(db: Session) -> list[tuple[str, str]]:
|
||||
"""The candidate folders whose stored row is ABSENT (phase 96, 02).
|
||||
|
||||
The unchanged-sync self-heal gate: a one-shot reply can still
|
||||
exhaust its retries and leave a candidate folder without a row, and
|
||||
the phase-94 change gate only regenerated on a KB change — so the
|
||||
gap persisted until the next KB change. This function names the gap:
|
||||
the generator's candidate folders (the EXACT candidate computation
|
||||
— one catalogue query in catalogue order, :func:`group_by_folder`,
|
||||
recursive subtree ≥ :data:`MIN_DOCS_PER_FOLDER`) minus the stored
|
||||
keys (one bounded select). The gate probe and the
|
||||
:func:`generate_folder_summaries` fill both key off this one
|
||||
concept.
|
||||
|
||||
Returns the missing keys sorted by ``(source, folder_path)``;
|
||||
``[]`` when there is no gap — including an empty catalogue over an
|
||||
empty table (no candidates, no gap). A single-document folder is
|
||||
never listed (it is not a candidate), and a stored row for a folder
|
||||
that dropped below the minimum is NOT missing (it is stale — the
|
||||
prune pass owns it).
|
||||
"""
|
||||
return sorted(
|
||||
key for key in _candidates(_catalog_rows(db)) if key not in _stored_keys(db)
|
||||
)
|
||||
|
||||
|
||||
async def generate_folder_summaries(
|
||||
db: Session, llm: FolderSummaryLLM, *, skip: bool = False
|
||||
db: Session,
|
||||
llm: FolderSummaryLLM,
|
||||
*,
|
||||
skip: bool = False,
|
||||
only_missing: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Regenerate the stored folder summaries for the current catalogue.
|
||||
|
||||
@@ -328,38 +399,45 @@ async def generate_folder_summaries(
|
||||
FOLDER: one folder's :class:`LLMError` is logged and counted,
|
||||
its previous row (if any) is kept, and the remaining folders
|
||||
still land (a ``lite`` outage must never fail the sync — the KB
|
||||
is the product, the summaries are auxiliary).
|
||||
is the product, the summaries are auxiliary). With
|
||||
``only_missing=True`` the iteration is restricted to the
|
||||
candidates that have NO stored row (the same gap
|
||||
:func:`missing_folder_summaries` reports, derived from the SAME
|
||||
catalogue pass — one bounded stored-key select, no second
|
||||
catalogue query): existing rows stay byte-identical (summary
|
||||
text AND ``updated_at`` — never re-stamped, even a stale-looking
|
||||
one; staleness is the changed-KB regeneration's job) and no
|
||||
``lite`` call is burned for a folder that already has a summary
|
||||
— the unchanged-sync self-heal fill (phase 96, task 02).
|
||||
4. DELETE rows whose folder no longer has ≥ 2 documents — a
|
||||
pruned/renamed folder's summary goes stale and is dropped.
|
||||
Rows for folders that still qualify persist (regenerated in
|
||||
step 3 — an unchanged folder's summary is still true).
|
||||
step 3 — an unchanged folder's summary is still true). The
|
||||
prune pass runs in BOTH modes: under ``only_missing`` on an
|
||||
unchanged catalogue it is a no-op (the invariant kept), and it
|
||||
still drops rows whose folder fell below the minimum.
|
||||
|
||||
Only flushes — the CALLER commits (the phase-53
|
||||
``bump_sources_version`` convention: the sync path owns the
|
||||
transaction, so a failed sync rolls the summaries back with it).
|
||||
|
||||
Returns the small stats dict ``{"generated", "failed", "pruned"}``
|
||||
for the caller's summary-line logging (PLAN §9 ample logging).
|
||||
for the caller's summary-line logging (PLAN §9 ample logging) —
|
||||
the caller logs the mode, not the generator.
|
||||
"""
|
||||
stats = {"generated": 0, "failed": 0, "pruned": 0}
|
||||
if skip:
|
||||
return stats
|
||||
|
||||
result = db.execute(
|
||||
select(Document.source, Document.path, Document.title, Document.summary)
|
||||
.order_by(Document.source, Document.path)
|
||||
).all()
|
||||
rows: list[DocRow] = [
|
||||
(source, path, title, summary)
|
||||
for source, path, title, summary in result
|
||||
]
|
||||
groups = group_by_folder(rows)
|
||||
candidates = {
|
||||
key: docs for key, docs in groups.items() if len(docs) >= MIN_DOCS_PER_FOLDER
|
||||
}
|
||||
candidates = _candidates(_catalog_rows(db))
|
||||
keys = sorted(candidates)
|
||||
if only_missing:
|
||||
stored = _stored_keys(db)
|
||||
keys = [key for key in keys if key not in stored]
|
||||
|
||||
for source, folder_path in sorted(candidates):
|
||||
docs = candidates[(source, folder_path)]
|
||||
for key in keys:
|
||||
source, folder_path = key
|
||||
docs = candidates[key]
|
||||
try:
|
||||
summary = await summarize_folder(source, folder_path, docs, llm)
|
||||
except LLMError as e:
|
||||
|
||||
+96
-17
@@ -207,6 +207,23 @@ class _TooLarge(RuntimeError):
|
||||
"""Internal: the endpoint rejected the request's input size."""
|
||||
|
||||
|
||||
class _EmptyContentError(RuntimeError):
|
||||
"""Internal: the one-shot reply parsed but carries no usable content.
|
||||
|
||||
Raised by :meth:`LLMClient._chat_once` for the retryable failure
|
||||
class (phase 96: ``content`` None or whitespace). Carries the
|
||||
reply's ``finish_reason`` (``None`` when the provider omits it) so
|
||||
:meth:`LLMClient.chat` can log the greppable retry line. Never
|
||||
escapes the client — ``chat()`` converts it to the public
|
||||
:class:`LLMError` (D3). Not an :class:`LLMError` on purpose: callers
|
||||
catching :class:`LLMError` must only ever see final, public errors.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, finish_reason: str | None) -> None:
|
||||
super().__init__(message)
|
||||
self.finish_reason = finish_reason
|
||||
|
||||
|
||||
class LLMClient:
|
||||
"""Thin async wrapper over the aipi OpenAI-compatible API."""
|
||||
|
||||
@@ -332,24 +349,19 @@ class LLMClient:
|
||||
(vec,) = await self.embed([text])
|
||||
return vec
|
||||
|
||||
async def chat(
|
||||
self, messages: list[dict[str, Any]], model: str | None = None
|
||||
) -> str:
|
||||
"""One-shot (non-streaming) completion (A5 extended, phase 30).
|
||||
async def _chat_once(self, messages: list[dict[str, Any]], model: str) -> str:
|
||||
"""One non-streaming completion attempt (phase 96, private).
|
||||
|
||||
Short, low-temperature request (``temperature=0.2``, 2048-token
|
||||
cap — summaries and outlines are small, so a fixed budget is
|
||||
enough) against ``BOR_LLM_SUMMARY_MODEL`` (default ``lite``)
|
||||
unless *model* names another. Used by the document summarizer
|
||||
(phase 30) and the KB overview generator (phase 31).
|
||||
|
||||
Any transport/HTTP/malformed failure, a choiceless reply, or an
|
||||
empty/missing ``content`` field raises :class:`LLMError` — a
|
||||
silent empty summary must never be stored.
|
||||
The single-attempt body of :meth:`chat` — the transport wrap
|
||||
(→ :class:`LLMError` with the sanitized base URL), the choiceless
|
||||
check, and the empty-content check. Raises
|
||||
:class:`_EmptyContentError` (carrying the reply's
|
||||
``finish_reason``) for an empty reply — the one failure class
|
||||
:meth:`chat` retries — and :class:`LLMError` for everything else.
|
||||
"""
|
||||
try:
|
||||
resp = await self._client.chat.completions.create(
|
||||
model=model or self.settings.llm_summary_model,
|
||||
model=model,
|
||||
messages=cast("list[ChatCompletionMessageParam]", messages),
|
||||
temperature=0.2,
|
||||
max_tokens=2048,
|
||||
@@ -368,14 +380,81 @@ class LLMClient:
|
||||
f"chat completion from {sanitize_error(self.settings.llm_base_url)} "
|
||||
"returned no choices"
|
||||
)
|
||||
content = resp.choices[0].message.content
|
||||
choice = resp.choices[0]
|
||||
content = choice.message.content
|
||||
if content is None or not content.strip():
|
||||
raise LLMError(
|
||||
raise _EmptyContentError(
|
||||
f"chat completion from {sanitize_error(self.settings.llm_base_url)} "
|
||||
"returned empty content — refusing to store a silent summary"
|
||||
"returned empty content — refusing to store a silent summary",
|
||||
choice.finish_reason,
|
||||
)
|
||||
return content.strip()
|
||||
|
||||
async def chat(
|
||||
self, messages: list[dict[str, Any]], model: str | None = None
|
||||
) -> str:
|
||||
"""One-shot (non-streaming) completion (A5 extended, phase 30).
|
||||
|
||||
Short, low-temperature request (``temperature=0.2``, 2048-token
|
||||
cap — summaries and outlines are small, so a fixed budget is
|
||||
enough) against ``BOR_LLM_SUMMARY_MODEL`` (default ``lite``)
|
||||
unless *model* names another. Used by the document summarizer
|
||||
(phase 30), the KB overview generator (phase 31), folder
|
||||
summaries (phase 94), and the pre-sync probe
|
||||
(``check_models``, phase 41).
|
||||
|
||||
Any transport/HTTP/malformed failure, a choiceless reply, or an
|
||||
empty/missing ``content`` field raises :class:`LLMError` — a
|
||||
silent empty summary must never be stored.
|
||||
|
||||
Empty-reply retry (phase 96, house LLM-retry policy D1–D3): the
|
||||
empty-content failure class — the model answered but said
|
||||
nothing (``content`` None or whitespace) — is retried up to
|
||||
``settings.llm_retries`` times (``BOR_LLM_RETRIES``, default 3;
|
||||
``0`` = off) with a flat ``settings.llm_retry_delay``
|
||||
(``BOR_LLM_RETRY_DELAY``) between attempts, one WARNING per
|
||||
retry naming the model, the reply's ``finish_reason`` (``None``
|
||||
when the provider omits it), and the attempt count. After
|
||||
``1 + llm_retries`` empty attempts it raises
|
||||
:class:`LLMError` naming the attempts; with ``llm_retries=0``
|
||||
the single-attempt message stays byte-identical to the
|
||||
pre-phase-96 behavior. NO other failure retries at the app level:
|
||||
a choiceless reply and transport failures raise immediately (the
|
||||
openai SDK's own ``max_retries=2`` already re-POSTs wire-level
|
||||
failures — an app-level transport retry would stack on top of
|
||||
it).
|
||||
"""
|
||||
total = 1 + self.settings.llm_retries
|
||||
chosen = model or self.settings.llm_summary_model
|
||||
for attempt in range(1, total + 1):
|
||||
try:
|
||||
return await self._chat_once(messages, chosen)
|
||||
except _EmptyContentError as e:
|
||||
if attempt >= total:
|
||||
# Retries exhausted — refuse to store the silent
|
||||
# summary. total == 1 (BOR_LLM_RETRIES=0 kill switch)
|
||||
# keeps the pre-phase-96 message byte-identical.
|
||||
if total == 1:
|
||||
raise LLMError(str(e)) from e
|
||||
raise LLMError(
|
||||
f"chat completion from "
|
||||
f"{sanitize_error(self.settings.llm_base_url)} "
|
||||
f"returned empty content on all {total} attempts — "
|
||||
"refusing to store a silent summary"
|
||||
) from e
|
||||
logger.warning(
|
||||
"one-shot LLM reply returned empty content (model=%s, "
|
||||
"finish_reason=%s, attempt %d/%d) — retrying in %.1fs",
|
||||
chosen,
|
||||
e.finish_reason,
|
||||
attempt,
|
||||
total,
|
||||
self.settings.llm_retry_delay,
|
||||
)
|
||||
# Flat delay — the phase-67 convention, no backoff.
|
||||
await asyncio.sleep(self.settings.llm_retry_delay)
|
||||
raise AssertionError("unreachable: every attempt returned or raised")
|
||||
|
||||
async def chat_stream(
|
||||
self,
|
||||
messages: list[dict[str, Any]],
|
||||
|
||||
+83
-37
@@ -56,14 +56,22 @@ than none).
|
||||
|
||||
The stored folder summaries (phase 94 — the drill-down ``ls``'s
|
||||
per-level descriptions, the ``folder_summaries`` table) regenerate in
|
||||
the same run under the same gate: a changed KB (added + updated > 0),
|
||||
or an empty table after an unchanged walk (the first full run after
|
||||
migration 0017, or after a ``--limit`` first walk that skipped them).
|
||||
Same contract — **best-effort, per-folder fail-soft**: a ``lite``
|
||||
failure keeps the failed folders' previous rows and only counts into
|
||||
the stats; ``--limit`` debug runs skip them entirely (never burning a
|
||||
``lite`` call). The stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>`` (or
|
||||
the same run under the same gate: a changed KB (added + updated > 0 —
|
||||
full regeneration), or, after an unchanged walk, a GAP — a candidate
|
||||
folder (≥ 2 docs) with no stored row (phase 96: this subsumes the old
|
||||
table-empty trigger exactly — an empty table leaves EVERY candidate
|
||||
missing, as after the first full run after migration 0017 or a
|
||||
``--limit`` first walk that skipped them — and catches the single row
|
||||
an exhausted one-shot retry lost mid-run). A gap after an unchanged
|
||||
walk fills ONLY the missing rows (``only_missing`` — every other row
|
||||
stays byte-identical, summary text AND ``updated_at``), and the run's
|
||||
stats token carries `` (gap-fill)`` behind the numbers. Same contract
|
||||
— **best-effort, per-folder fail-soft**: a ``lite`` failure keeps the
|
||||
failed folders' previous rows (or leaves the row absent) and only
|
||||
counts into the stats; ``--limit`` debug runs skip them entirely
|
||||
(never burning a ``lite`` call). The stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>`` (`` (gap-fill)``
|
||||
appended after the stats when the run took the targeted-fill path;
|
||||
``folder_summaries=skipped`` when the gate did not fire), and the rows
|
||||
commit in the run's own short-lived session (the phase-53 convention —
|
||||
a failed commit rolls them back with it).
|
||||
@@ -92,7 +100,7 @@ from app.core.debugging import configure_debugging
|
||||
from app.core.logging import configure_logging
|
||||
from app.db import SessionLocal
|
||||
from app.models import KbOverview
|
||||
from app.rag.folder_summaries import folder_summary_table_empty, generate_folder_summaries
|
||||
from app.rag.folder_summaries import generate_folder_summaries, missing_folder_summaries
|
||||
from app.rag.git_sources import effective_sources
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient
|
||||
@@ -231,18 +239,20 @@ def _overview_row_exists() -> bool:
|
||||
return session.get(KbOverview, 1) is not None
|
||||
|
||||
|
||||
def _folder_summaries_table_empty() -> bool:
|
||||
"""Whether the ``folder_summaries`` table holds any row (phase 94).
|
||||
def _folder_summaries_gap() -> list[tuple[str, str]]:
|
||||
"""The folder-summary gaps (phase 96, task 03): the candidate
|
||||
folders (≥ 2 docs) with no stored row, sorted.
|
||||
|
||||
The ``_overview_row_exists`` pattern extended to a table-emptiness
|
||||
check (one bounded ``LIMIT 1`` probe): an empty table after an
|
||||
unchanged re-import — e.g. the first full run after migration 0017,
|
||||
or after a ``--limit`` first walk that skipped generation — still
|
||||
gets a fresh batch of folder summaries, while a populated table is
|
||||
left untouched until the KB actually changes.
|
||||
The unchanged-walk self-heal trigger, replacing the phase-94
|
||||
table-emptiness probe (which the gap subsumes exactly: an empty
|
||||
table leaves every candidate missing, so the targeted fill over
|
||||
all candidates IS a full generation — the first full run after
|
||||
migration 0017, or after a ``--limit`` first walk that skipped
|
||||
generation, still generates — and a single row an exhausted
|
||||
one-shot retry lost mid-run is healed on the next sync).
|
||||
"""
|
||||
with SessionLocal() as session:
|
||||
return folder_summary_table_empty(session)
|
||||
return missing_folder_summaries(session)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
@@ -275,7 +285,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
|
||||
llm = LLMClient()
|
||||
|
||||
async def _run() -> tuple[ImportSummary, str, str, dict[str, int] | None]:
|
||||
async def _run(
|
||||
) -> tuple[ImportSummary, str, str, dict[str, int] | None, bool]:
|
||||
"""Import, then (change-gated) advance the sources version,
|
||||
refresh the stored KB overview, and regenerate the stored folder
|
||||
summaries.
|
||||
@@ -299,16 +310,24 @@ def main(argv: list[str] | None = None) -> int:
|
||||
runs and unchanged re-runs never bump. The returned token is
|
||||
the new version, or ``"skipped"``.
|
||||
|
||||
The folder summaries (phase 94, task 02) follow the same gate
|
||||
— a changed KB, or an empty ``folder_summaries`` table after an
|
||||
unchanged walk (the first full run after migration 0017, or
|
||||
after a ``--limit`` first walk) — with per-folder fail-soft
|
||||
The folder summaries (phase 94, task 02; phase 96, task 03)
|
||||
follow the same gate — a changed KB (full regeneration), or,
|
||||
after an unchanged walk, a GAP: a candidate folder (≥ 2 docs)
|
||||
with no stored row (the subsumed table-empty trigger — an
|
||||
empty table leaves every candidate missing — plus a row an
|
||||
exhausted one-shot retry lost) — with per-folder fail-soft
|
||||
inside the generator (a ``lite`` failure keeps the failed
|
||||
folders' previous rows). The generator only flushes: this
|
||||
folders' previous rows or leaves the row absent). The gap path
|
||||
passes ``only_missing=True`` (existing rows stay
|
||||
byte-identical) and its stats token gains the `` (gap-fill)``
|
||||
suffix on the summary line. The generator only flushes: this
|
||||
run's own short-lived session commits (the phase-53
|
||||
convention), and the stats land on the summary line as
|
||||
``folder_summaries=<generated>/<failed>/<pruned>``
|
||||
(``None`` — rendered ``skipped`` — when the gate did not fire).
|
||||
(``None`` — rendered ``skipped`` — when the gate did not
|
||||
fire). The returned fifth element names the mode the stats
|
||||
were taken in (the `` (gap-fill)`` suffix trigger — ``True``
|
||||
only when the unchanged-walk gap fired the targeted fill).
|
||||
"""
|
||||
summary = await import_sources(
|
||||
sources, llm, prune=args.prune, limit=args.limit,
|
||||
@@ -339,24 +358,30 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# folder summaries (the --limit skip, mirrored above for the
|
||||
# sources version).
|
||||
logger.info("overview: skipped (--limit)")
|
||||
return summary, "skipped", sources_version, None
|
||||
return summary, "skipped", sources_version, None, False
|
||||
changed = summary.added + summary.updated > 0
|
||||
overview_due = changed
|
||||
folders_due = changed
|
||||
folder_gap_fill = False
|
||||
if not changed:
|
||||
if summary.files == 0:
|
||||
logger.info("overview: skipped (nothing imported)")
|
||||
return summary, "skipped", sources_version, None
|
||||
# The unchanged-walk first-run triggers: the overview when
|
||||
# no row exists yet (the first run after migration 0005),
|
||||
# the folder summaries when the table is empty (the first
|
||||
# full run after migration 0017, or after a --limit first
|
||||
# walk that skipped them).
|
||||
return summary, "skipped", sources_version, None, False
|
||||
# The unchanged-walk triggers: the overview when no row
|
||||
# exists yet (the first run after migration 0005), the
|
||||
# folder summaries when the table has a GAP — a candidate
|
||||
# folder (>= 2 docs) with no stored row (phase 96, task
|
||||
# 03; the old table-empty trigger is the special case
|
||||
# where every candidate is missing). The gap path is
|
||||
# ALWAYS the targeted fill: only the missing rows
|
||||
# regenerate (only_missing=True), every other row stays
|
||||
# byte-identical.
|
||||
overview_due = not _overview_row_exists()
|
||||
folders_due = _folder_summaries_table_empty()
|
||||
folders_due = bool(_folder_summaries_gap())
|
||||
folder_gap_fill = folders_due
|
||||
if not overview_due and not folders_due:
|
||||
logger.info("overview: skipped (KB unchanged)")
|
||||
return summary, "skipped", sources_version, None
|
||||
return summary, "skipped", sources_version, None, False
|
||||
overview_status = "skipped"
|
||||
if overview_due:
|
||||
ok = await regenerate_overview(llm)
|
||||
@@ -371,20 +396,41 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# summaries back with it.
|
||||
folder_stats: dict[str, int] | None = None
|
||||
if folders_due:
|
||||
# Phase 96 (task 03): a changed-KB run is a full
|
||||
# regeneration (today's behavior, byte-identical); the
|
||||
# unchanged-walk gap run is the targeted fill (only the
|
||||
# missing candidates burn a lite call).
|
||||
session = SessionLocal()
|
||||
try:
|
||||
folder_stats = await generate_folder_summaries(session, llm)
|
||||
folder_stats = await generate_folder_summaries(
|
||||
session, llm, only_missing=folder_gap_fill
|
||||
)
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
return summary, overview_status, sources_version, folder_stats
|
||||
return (
|
||||
summary, overview_status, sources_version, folder_stats,
|
||||
folder_gap_fill,
|
||||
)
|
||||
|
||||
summary, overview_status, sources_version, folder_stats = asyncio.run(_run())
|
||||
(
|
||||
summary,
|
||||
overview_status,
|
||||
sources_version,
|
||||
folder_stats,
|
||||
folder_gap_fill,
|
||||
) = asyncio.run(_run())
|
||||
folder_token = (
|
||||
"skipped"
|
||||
if folder_stats is None
|
||||
else f"{folder_stats['generated']}/{folder_stats['failed']}/{folder_stats['pruned']}"
|
||||
)
|
||||
if folder_stats is not None and folder_gap_fill:
|
||||
# PLAN §9 greppable-cron-safe line — the line-extension house
|
||||
# rule: the targeted fill (phase 96, task 03) is named on the
|
||||
# summary line; the full-regeneration token stays
|
||||
# byte-identical to phase 94.
|
||||
folder_token += " (gap-fill)"
|
||||
print(
|
||||
f"import_docs: files={summary.files} added={summary.added} "
|
||||
f"updated={summary.updated} unchanged={summary.unchanged} "
|
||||
|
||||
+131
-1
@@ -385,7 +385,38 @@ test_llm_retry.py``). The mock is single-conversation per e2e server, so
|
||||
SDK-level retries), so one POST per attempt: the counter is per
|
||||
POST here, unlike the chat counter above.
|
||||
Non-streaming requests (document summaries, KB overview) never 500 —
|
||||
the retry scope is the chat turn only (owner-locked A1).
|
||||
the retry scope is the chat turn only (owner-locked A1). The one
|
||||
non-streaming injection is phase 96's incident shape below (it
|
||||
answers 200 with EMPTY content — the semantic failure class, not a
|
||||
dead endpoint).
|
||||
|
||||
Failure injection (phase 96, one-shot resilience, task 04) — the
|
||||
2026-09-11 incident shape for the folder-summary one-shot path
|
||||
(``tests/e2e/test_oneshot_llm_retry.py``): a NON-stream
|
||||
``chat/completions`` request whose system prompt carries
|
||||
``FOLDER_SUMMARY_MODE`` (the folder-summary marker — ``chat()`` is
|
||||
the mock's only non-streaming consumer of it) and whose user
|
||||
message's ``Folder: …`` header (the branch's existing parse, the
|
||||
``FOLDER_HEADER_PREFIX`` tail) labels this suite's own fixture
|
||||
folders:
|
||||
- the label ends with ``/e2e_empty_once``: the FIRST non-stream POST
|
||||
for that label answers the incident envelope — the mock's normal
|
||||
OpenAI chat-completion shape with ``choices[0].message.content =
|
||||
""`` and ``choices[0].finish_reason = "length"`` (the exact wire
|
||||
shape of the empty ``lite`` reply: the budget spent in
|
||||
``reasoning_content``) — and every later POST returns the normal
|
||||
``Fixture folder summary for <folder>.`` line (the one-shot retry,
|
||||
phase 96 task 01, recovers the row).
|
||||
- the label ends with ``/e2e_empty_always``: EVERY non-stream POST
|
||||
for that label answers the empty envelope (the 1 +
|
||||
``BOR_LLM_RETRIES`` exhaustion → per-folder fail-soft → the row
|
||||
stays absent while the sync stays green, phase 94 contract).
|
||||
The once-sequence is driven by a module-level per-label counter that
|
||||
resets after the success it guards (the phase-67 ``_fail_posts``
|
||||
pattern — the mock is single-conversation per e2e server), so a
|
||||
second sync re-drives the sequence deterministically. The trigger
|
||||
strings are this suite's own folder names, so no other E2E can hit
|
||||
them (they seed different trees).
|
||||
|
||||
``max_tokens`` is honored deterministically (token ≈ whitespace word),
|
||||
like a real endpoint: an answer longer than the cap is truncated. This
|
||||
@@ -1496,6 +1527,70 @@ def _history_echo(body: dict[str, Any]) -> str:
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 96 (task 04, one-shot resilience): the incident-shape injection
|
||||
# for the folder-summary one-shot path — see the module docstring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The folder-label triggers (phase 96, task 04): the ``FOLDER_SUMMARY_MODE``
|
||||
#: branch's folder label (the user message's ``Folder: …`` tail — the
|
||||
#: ``FOLDER_HEADER_PREFIX`` parse) ending with these suffixes is THIS
|
||||
#: SUITE'S own fixture folder (``tests/e2e/test_oneshot_llm_retry.py``
|
||||
#: seeds the names — no other E2E can hit them; they seed different
|
||||
#: trees, and the triggers carry the E2E prefix).
|
||||
ONESHOT_EMPTY_ONCE_SUFFIX = "/e2e_empty_once"
|
||||
ONESHOT_EMPTY_ALWAYS_SUFFIX = "/e2e_empty_always"
|
||||
|
||||
#: Module-level per-label incident counter — the mock is single-
|
||||
#: conversation per e2e server (the phase-67 ``_fail_posts``
|
||||
#: convention). Counts the non-stream folder-summary POSTs served per
|
||||
#: trigger label; the once-sequence resets after the success it guards
|
||||
#: (the first normal reply), so a second sync re-drives the sequence
|
||||
#: deterministically.
|
||||
_empty_once_posts: dict[str, int] = {}
|
||||
|
||||
|
||||
def _folder_summary_incident(body: dict[str, Any]) -> bool:
|
||||
"""Should this NON-stream folder-summary POST answer with the
|
||||
2026-09-11 incident envelope (``content=""`` +
|
||||
``finish_reason="length"`` — the exact wire shape of the empty
|
||||
``lite`` reply phase 96's one-shot retry targets)?
|
||||
|
||||
* the system prompt lacks ``FOLDER_SUMMARY_MODE`` → never (only the
|
||||
folder-summary one-shot carries the marker — ``chat()`` is the
|
||||
mock's only non-streaming consumer of it, so the counter counts
|
||||
exactly the one-shot POSTs the app's retry policy drives);
|
||||
* the label ends with ``/e2e_empty_always`` → EVERY POST (the
|
||||
exhaustion path — 1 + ``BOR_LLM_RETRIES`` empty attempts, the
|
||||
per-folder fail-soft leaves the row absent);
|
||||
* the label ends with ``/e2e_empty_once`` → the FIRST non-stream
|
||||
POST for that label only — every later POST returns the normal
|
||||
line (the retry recovers the row), and the counter resets on
|
||||
that first normal reply (the phase-67 pattern).
|
||||
|
||||
The folder label is the ``FOLDER_SUMMARY_MODE`` branch's existing
|
||||
parse (the user message's first line, the ``FOLDER_HEADER_PREFIX``
|
||||
tail) — the injection is a pure function of the request plus the
|
||||
per-label counter (the house marker-flow convention).
|
||||
"""
|
||||
if "FOLDER_SUMMARY_MODE" not in _system(body):
|
||||
return False
|
||||
user = _user(body)
|
||||
header = user.splitlines()[0] if user else ""
|
||||
if not header.startswith(FOLDER_HEADER_PREFIX):
|
||||
return False
|
||||
label = header.removeprefix(FOLDER_HEADER_PREFIX).strip()
|
||||
if label.endswith(ONESHOT_EMPTY_ALWAYS_SUFFIX):
|
||||
return True
|
||||
if label.endswith(ONESHOT_EMPTY_ONCE_SUFFIX):
|
||||
n = _empty_once_posts.get(label, 0) + 1
|
||||
_empty_once_posts[label] = n
|
||||
if n == 1:
|
||||
return True # the first POST: the incident envelope
|
||||
_empty_once_posts[label] = 0 # the retry went out — restart
|
||||
return False
|
||||
|
||||
|
||||
def compose_answer(body: dict[str, Any]) -> str:
|
||||
system = _system(body)
|
||||
user = _user(body)
|
||||
@@ -1516,6 +1611,12 @@ def compose_answer(body: dict[str, Any]) -> str:
|
||||
# shadow every folder-summary call. Checked BEFORE the
|
||||
# DEFLECT_MODE branch, like the other lite-mode markers (a
|
||||
# deflection prompt never carries one).
|
||||
# Phase 96 (task 04): the incident-shape injection keys on THIS
|
||||
# branch's label (``_folder_summary_incident`` — the module
|
||||
# docstring) and overrides the NON-STREAM response envelope in
|
||||
# ``chat_completions`` (``content=""`` +
|
||||
# ``finish_reason="length"``); the line below is what the later
|
||||
# / normal POSTs return.
|
||||
header = user.splitlines()[0] if user else ""
|
||||
folder = (
|
||||
header.removeprefix(FOLDER_HEADER_PREFIX).strip()
|
||||
@@ -2167,6 +2268,35 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
)
|
||||
|
||||
if not body.get("stream"):
|
||||
# Phase 96 (task 04): the incident-shape injection (the module
|
||||
# docstring) — the trigger-labelled folder summary answers the
|
||||
# exact 2026-09-11 envelope: the mock's normal OpenAI
|
||||
# chat-completion shape with ``content=""`` and
|
||||
# ``finish_reason="length"`` (the budget spent in
|
||||
# ``reasoning_content``). ``chat()`` is the mock's only
|
||||
# non-streaming folder-summary consumer, so this is the one-shot
|
||||
# retry path (``app.rag.llm.LLMClient.chat``, phase 96 task 01)
|
||||
# and nothing else — every other non-stream response is
|
||||
# byte-identical to pre-phase-96.
|
||||
if _folder_summary_incident(body):
|
||||
return {
|
||||
"id": f"chatcmpl-{uuid.uuid4()}",
|
||||
"object": "chat.completion",
|
||||
"created": int(time.time()),
|
||||
"model": body.get("model", "turbo"),
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": ""},
|
||||
"finish_reason": "length",
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 2048,
|
||||
"total_tokens": 2148,
|
||||
},
|
||||
}
|
||||
message: dict[str, Any] = {"role": "assistant", "content": answer}
|
||||
if thinking:
|
||||
# Harmless future-proofing: the app only uses streaming, but a
|
||||
|
||||
@@ -0,0 +1,633 @@
|
||||
"""Phase 96 task 04 E2E (Playwright, mock-only): the one-shot LLM
|
||||
resilience — the 2026-09-11 incident shape, retried and healed.
|
||||
|
||||
The dedicated story suite for ``96_oneshot_resilience`` (A16 — one
|
||||
Playwright file per phase, run in isolation): a folder whose FIRST
|
||||
one-shot summary reply arrives in the exact incident shape
|
||||
(``content=""`` + ``finish_reason="length"``) still ends up with its
|
||||
stored summary (the task-01 retry recovered it, visible in the
|
||||
``ls`` drill-down), a folder whose replies are ALWAYS empty stays
|
||||
absent without failing the sync (the task-01 exhaustion + the phase-94
|
||||
per-folder fail-soft contract), and a row deleted behind the app's back
|
||||
is self-healed by the next UNCHANGED sync with the other rows untouched
|
||||
(tasks 02/03 targeted fill).
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
|
||||
deterministic incident-shape injection in ``tests/e2e/mock_llm.py``
|
||||
(phase 96, task 04): a NON-stream ``chat/completions`` request whose
|
||||
system prompt carries ``FOLDER_SUMMARY_MODE`` and whose ``Folder: …``
|
||||
label ends with ``/e2e_empty_once`` answers the incident envelope
|
||||
(``content=""`` + ``finish_reason="length"`` — the mock's normal OpenAI
|
||||
shape) on its FIRST non-stream POST only, and a label ending with
|
||||
``/e2e_empty_always`` answers it on EVERY non-stream POST. The
|
||||
per-label counter resets after the success it guards (the phase-67
|
||||
``_fail_posts`` pattern), so a re-run of the suite is green without
|
||||
manual state cleanup. The trigger strings are this suite's own folder
|
||||
names, so no other E2E can hit them.
|
||||
|
||||
The e2e app server boots with ``BOR_LLM_RETRY_DELAY=0`` (the conftest
|
||||
pattern) so the 4-attempt exhaustion paths are instant;
|
||||
``BOR_LLM_RETRIES`` is forced to the code default (3 — the conftest
|
||||
leak-guard pattern), so ``e2e_empty_always`` costs exactly 4 POSTs per
|
||||
sync that attempts it and the suite stays fast.
|
||||
|
||||
KB fixture — a host temp dir tree (``tmp_path_factory``; the app runs
|
||||
on the same host) with ONE registered local source (the
|
||||
``test_local_directory_sources.py`` registration + real-Sync pattern;
|
||||
no git anywhere): ``oneshot/`` with three ≥ 2-doc folders —
|
||||
``e2e_empty_once/`` (2 docs), ``e2e_empty_always/`` (2), ``normal/``
|
||||
(2). Every fixture doc carries the words ``drill down the tree`` in
|
||||
its body, so the scripted ``ls`` drill-down questions (the phase-94
|
||||
``DRILL_TRIGGER`` echo pattern — the mock echoes the received tool
|
||||
result into its grounded answer, the E2E's only lens on the LLM's
|
||||
context) FTS-match at least one chunk and run grounded.
|
||||
|
||||
Test → phase mapping (Playwright Mapping Rule):
|
||||
1. ``test_incident_reply_retried_and_always_empty_stays_absent`` —
|
||||
after the changed sync #1 (the module fixture pins the stored rows:
|
||||
the ``e2e_empty_once`` row EXISTS — the retry recovered it, without
|
||||
task 01 it would be absent — and the ``e2e_empty_always`` row does
|
||||
NOT — all 4 attempts empty → ``LLMError`` → per-folder fail-soft),
|
||||
a scripted ``ls oneshot`` turn asserts the drill-down listing: the
|
||||
``e2e_empty_once/`` line carries
|
||||
``: Fixture folder summary for oneshot/e2e_empty_once.`` (the retry
|
||||
recovered the row), the ``normal/`` line carries its summary, and
|
||||
the ``e2e_empty_always/`` line is ``e2e_empty_always/ — 2
|
||||
documents`` with NO ``: …`` suffix — while the sync reported
|
||||
success (a folder-summary exhaustion never flips the run, phase 94).
|
||||
2. ``test_deleted_row_self_heals_on_unchanged_sync`` — the ``normal``
|
||||
stored row is deleted directly (simulating a historical failure),
|
||||
``e2e_empty_once``'s row ``updated_at`` is captured, and the
|
||||
UNCHANGED sync #2 runs the gap gate (task 03): a second scripted
|
||||
``ls oneshot`` turn shows ``normal/`` healed
|
||||
(``: Fixture folder summary for oneshot/normal.`` again) and
|
||||
``e2e_empty_always/`` still absent (the gap-fill attempted it,
|
||||
exhausted, stayed absent — the sync still succeeded), and the DB
|
||||
pins the targeted fill: the ``normal`` row is back with its
|
||||
deterministic text and the ``e2e_empty_once`` row's ``updated_at``
|
||||
is UNCHANGED (a full regeneration would have re-stamped it).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from playwright.sync_api import Locator, Page, expect
|
||||
from sqlalchemy import select, text
|
||||
|
||||
from app.config import Settings as _Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import FolderSummary
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import (
|
||||
ADMIN_PASSWORD,
|
||||
SESSION_SECRET,
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
# Phase 79 (task 04, full inventory): the conftest session app owns its
|
||||
# port in a combined run — this module app binds its own port instead
|
||||
# (a same-port second uvicorn dies on bind and would drive the wrong
|
||||
# server). Env-overridable.
|
||||
APP_PORT = int(os.environ.get("E2E_APP_PORT_ONESHOT", "8138"))
|
||||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fixture documents + the pinned drill-down listing (deterministic)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: The local source — the temp directory's basename (``kind=local`` →
|
||||
#: the directory's basename is the source name, phase 38).
|
||||
SOURCE = "oneshot"
|
||||
|
||||
#: The three ≥ 2-doc folders (the mock's trigger labels are the folder
|
||||
#: NAMES — the seeded KB carries them, the house marker-flow
|
||||
#: convention). Path order (the ``ls`` subfolder order):
|
||||
#: e2e_empty_always < e2e_empty_once < normal.
|
||||
FOLDER_ONCE = "e2e_empty_once"
|
||||
FOLDER_ALWAYS = "e2e_empty_always"
|
||||
FOLDER_NORMAL = "normal"
|
||||
|
||||
TOTAL_DOCS = 6 # three folders x 2 docs each
|
||||
|
||||
#: Every fixture body carries ``drill down the tree`` (the
|
||||
#: ``DRILL_TRIGGER`` phrase's words): every scripted question
|
||||
#: FTS-matches at least one chunk → HIGH gate → the ``<tools>`` section
|
||||
#: the drill-down flow keys on (the phase-94 seed convention).
|
||||
DRILL_LEAD = "The drill down the tree fixture note"
|
||||
|
||||
|
||||
def _md(title: str, body: str) -> str:
|
||||
return f"# {title}\n\n{body}\n"
|
||||
|
||||
|
||||
#: The mock's byte-stable ``FOLDER_SUMMARY_MODE`` lines for this
|
||||
#: fixture (the phase-94 template — the label is the
|
||||
#: ``FOLDER_HEADER_PREFIX`` tail: ``<source>`` for the root,
|
||||
#: ``<source>/<folder>`` for a folder).
|
||||
SUM_ROOT = f"Fixture folder summary for {SOURCE}."
|
||||
SUM_ONCE = f"Fixture folder summary for {SOURCE}/{FOLDER_ONCE}."
|
||||
SUM_ALWAYS = f"Fixture folder summary for {SOURCE}/{FOLDER_ALWAYS}."
|
||||
SUM_NORMAL = f"Fixture folder summary for {SOURCE}/{FOLDER_NORMAL}."
|
||||
|
||||
# --- the pinned ``ls oneshot`` level (app.rag.agent's phase-94 template)
|
||||
|
||||
#: The source root: 0 direct files, 3 subfolders (path order).
|
||||
LS_HEADER = f"{SOURCE} — 0 documents, 3 folders:"
|
||||
|
||||
#: The subfolder lines — the ``: {summary}`` suffix appended ONLY when
|
||||
#: the subfolder's summary is stored (``render_folder_listing``; the
|
||||
#: renderer's 2-space indent is whitespace-normalized away by the
|
||||
#: ``to_contain_text`` match — the phase-94 pin convention).
|
||||
LINE_ALWAYS = f"{FOLDER_ALWAYS}/ — 2 documents"
|
||||
LINE_ALWAYS_WITH_COLON = f"{FOLDER_ALWAYS}/ — 2 documents: "
|
||||
LINE_ONCE = f"{FOLDER_ONCE}/ — 2 documents: {SUM_ONCE}"
|
||||
LINE_NORMAL = f"{FOLDER_NORMAL}/ — 2 documents: {SUM_NORMAL}"
|
||||
|
||||
# --- the scripted drill-down turns (the phase-94 ``DRILL_TRIGGER``
|
||||
# questions — the mock echoes the listing verbatim into the answer) ---
|
||||
|
||||
LS_QUESTION_1 = f"Drill down the tree: ls {SOURCE} — what's in source {SOURCE}?"
|
||||
LS_QUESTION_2 = f"Drill down the tree: ls {SOURCE} — list the source folders again"
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def oneshot_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||
"""The one-source temp tree (see the module docstring): the app
|
||||
server runs on the same host, so the paths are visible to it."""
|
||||
root = tmp_path_factory.mktemp("bor_oneshot")
|
||||
src = root / SOURCE
|
||||
for folder, prefix in (
|
||||
(FOLDER_ONCE, "once"),
|
||||
(FOLDER_ALWAYS, "always"),
|
||||
(FOLDER_NORMAL, "normal"),
|
||||
):
|
||||
(src / folder).mkdir(parents=True)
|
||||
for letter, topic in (("a", "A"), ("b", "B")):
|
||||
(src / folder / f"{prefix}-{letter}.md").write_text(
|
||||
_md(
|
||||
f"Oneshot {prefix.title()} {letter.upper()}",
|
||||
f"{DRILL_LEAD} for {SOURCE} {folder} "
|
||||
f"{letter}: this document covers topic {topic} "
|
||||
f"of the {SOURCE} source tree.",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
assert len(list(src.rglob("*.md"))) == TOTAL_DOCS
|
||||
return src
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_server(mock_llm: int, oneshot_dir: Path) -> Iterator[str]:
|
||||
"""The real app under test — per-module app (the conftest pattern,
|
||||
cf. ``test_local_directory_sources.py``): NO ``BOR_GIT_SOURCES``
|
||||
(the env fallback is git-only — the source here is a DB-registered
|
||||
local directory), the mock LLM, the mock-calibrated threshold, and
|
||||
the leak-guarded code defaults. ``BOR_LLM_RETRY_DELAY=0`` + the
|
||||
code-default ``BOR_LLM_RETRIES`` (the phase-67 conftest pattern):
|
||||
the one-shot retry waits are instant and the exhaustion budget is
|
||||
the REAL one (4 attempts). The session app is never started in this
|
||||
isolated run, so no port clash."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||
env["BOR_LLM_BASE_URL"] = (
|
||||
"https://aipi.reeseapps.com/v1"
|
||||
if USE_REAL_LLM
|
||||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||||
)
|
||||
# Mock-calibrated threshold (conftest pattern): every scripted
|
||||
# question FTS-matches the fixture docs (the ``drill down the
|
||||
# tree`` words), so the gate is HIGH either way.
|
||||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||
# Phase 67 / phase 96: instant retry waits + the code-default budget
|
||||
# (the conftest leak-guard pattern) — the one-shot retry and the
|
||||
# 4-attempt exhaustion run in real time at zero delay.
|
||||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||||
env.setdefault(
|
||||
"BOR_DATABASE_URL",
|
||||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||
)
|
||||
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
|
||||
# reads it from cwd) — override it with an EMPTY value (the env var
|
||||
# beats the .env file): the registry must hold EXACTLY the one local
|
||||
# directory this suite registers.
|
||||
env["BOR_GIT_SOURCES"] = ""
|
||||
# Leak guards (conftest pattern): an operator's local (gitignored)
|
||||
# .env cannot leak corpus-specific settings into the app under test.
|
||||
env["BOR_DOCS_REPO"] = ""
|
||||
env["BOR_SUGGESTIONS"] = json.dumps(
|
||||
_Settings.model_fields["suggestions"].default
|
||||
)
|
||||
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
|
||||
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||||
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
||||
cwd=REPO,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
_wait_http(f"{APP_URL}/api/health")
|
||||
yield APP_URL
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url(app_server: str) -> str:
|
||||
return app_server
|
||||
|
||||
|
||||
def _truncate_all() -> None:
|
||||
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
|
||||
share one Postgres, so a leftover source or document would pollute
|
||||
the ``ls`` listing the drill answers assert on byte-exactly."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview, git_sources, folder_summaries"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _run_sync_http(base_url: str, timeout_s: float = 180.0) -> dict[str, Any]:
|
||||
"""Login + ``POST /api/sync`` + poll the status endpoint until the
|
||||
run reaches a terminal state (the ``test_sync_button.py`` /
|
||||
``test_local_directory_sources.py`` pattern, over plain httpx)."""
|
||||
with httpx.Client(base_url=base_url, timeout=30.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, r.text
|
||||
r = client.post("/api/sync")
|
||||
assert r.status_code == 202, r.text
|
||||
deadline = time.monotonic() + timeout_s
|
||||
body: dict[str, Any] = {}
|
||||
while time.monotonic() < deadline:
|
||||
r = client.get("/api/sync/status")
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
if body["state"] in ("success", "failed"):
|
||||
return body
|
||||
time.sleep(0.5)
|
||||
raise AssertionError(f"sync did not reach a terminal state: {body}")
|
||||
|
||||
|
||||
def _folder_rows() -> dict[str, tuple[str, datetime]]:
|
||||
"""The source's stored folder summaries
|
||||
``{folder_path: (summary, updated_at)}`` (``""`` = the source
|
||||
root) — the test process's direct DB access, the E2E's other
|
||||
established lens."""
|
||||
with SessionLocal() as db:
|
||||
rows = db.execute(
|
||||
select(
|
||||
FolderSummary.folder_path,
|
||||
FolderSummary.summary,
|
||||
FolderSummary.updated_at,
|
||||
).where(FolderSummary.source == SOURCE)
|
||||
).all()
|
||||
return {folder: (summary, updated_at) for folder, summary, updated_at in rows}
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def synced_kb(app_server: str, oneshot_dir: Path) -> None:
|
||||
"""The story's precondition: the KB synced under the deterministic
|
||||
mock's incident-shape injection.
|
||||
|
||||
Registers the temp directory through the authenticated API (the
|
||||
``test_local_directory_sources.py`` pattern) and runs the REAL
|
||||
in-process sync #1 (``POST /api/sync`` — walk → chunk → embed →
|
||||
overview → folder summaries → version bump). The sync changed the
|
||||
KB → FULL folder regeneration, and the mock's injection drives the
|
||||
incident shapes:
|
||||
|
||||
* ``oneshot`` (the root) + ``oneshot/normal`` — normal replies,
|
||||
one POST each;
|
||||
* ``oneshot/e2e_empty_once`` — the FIRST non-stream POST answers
|
||||
the incident envelope (``content=""`` + ``finish_reason=
|
||||
"length"``), the retry (task 01) recovers the row on the second
|
||||
POST;
|
||||
* ``oneshot/e2e_empty_always`` — EVERY non-stream POST answers the
|
||||
empty envelope: 1 + ``BOR_LLM_RETRIES`` = 4 attempts exhausted →
|
||||
``LLMError`` → the phase-94 per-folder fail-soft leaves the row
|
||||
ABSENT and never flips the run.
|
||||
|
||||
The fixture pins all of that in the DB: the ``e2e_empty_once`` row
|
||||
EXISTS (without task 01 it would be absent — the incident's data
|
||||
loss), the ``e2e_empty_always`` row does NOT, and the sync still
|
||||
reported ``success``.
|
||||
"""
|
||||
_truncate_all()
|
||||
with httpx.Client(base_url=app_server, timeout=30.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, r.text
|
||||
r = client.post(
|
||||
"/api/git-sources", json={"kind": "local", "path": str(oneshot_dir)}
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
body = _run_sync_http(app_server)
|
||||
assert body["state"] == "success", body
|
||||
detail = body["detail"]
|
||||
assert detail["added"] == TOTAL_DOCS, detail
|
||||
assert detail["updated"] == 0, detail
|
||||
assert detail["pruned"] == 0, detail
|
||||
assert detail["overview"] is True, detail
|
||||
# The full regeneration under the injection: the retried row
|
||||
# EXISTS (the incident, healed by task 01's one-shot retry), the
|
||||
# exhausted row is ABSENT (the per-folder fail-soft — the sync
|
||||
# above still reported success), the normal rows landed.
|
||||
rows = _folder_rows()
|
||||
assert set(rows) == {"", FOLDER_ONCE, FOLDER_NORMAL}, rows
|
||||
assert rows[""][0] == SUM_ROOT, rows
|
||||
assert rows[FOLDER_ONCE][0] == SUM_ONCE, rows
|
||||
assert rows[FOLDER_NORMAL][0] == SUM_NORMAL, rows
|
||||
assert FOLDER_ALWAYS not in rows, rows # all 4 attempts empty → no row
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean(db_ready: None) -> Iterator[None]:
|
||||
"""Per-test query_log isolation (the KB itself is module-scoped —
|
||||
the drill turns never change it, so the folder summaries and the
|
||||
registry persist across the tests of this module)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE query_log"))
|
||||
db.commit()
|
||||
yield
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE query_log"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Page helpers (the phase-94 drill-down house pattern)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
|
||||
#: (a response clone read in the background) — wire-level assertions
|
||||
#: for the ``tool`` frames, independent of the UI rendering.
|
||||
SSE_HOOK = """
|
||||
() => {
|
||||
if (window.__sseInstalled) return;
|
||||
window.__sseInstalled = true;
|
||||
window.__sseFrames = [];
|
||||
const origFetch = window.fetch;
|
||||
window.fetch = async function (...args) {
|
||||
const res = await origFetch.apply(this, args);
|
||||
try {
|
||||
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
|
||||
if (url.includes('/api/chat')) {
|
||||
res.clone().text().then((bodyText) => {
|
||||
for (const block of bodyText.split('\\n\\n')) {
|
||||
const line = block.trim();
|
||||
if (line.startsWith('data: ')) {
|
||||
window.__sseFrames.push(line.slice(6));
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (e) { /* non-clonable responses: ignored */ }
|
||||
return res;
|
||||
};
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _install_page_hooks(page: Page) -> None:
|
||||
page.evaluate(SSE_HOOK)
|
||||
|
||||
|
||||
def _frames(page: Page) -> list[dict]:
|
||||
"""The SSE frames captured since the last submit (``_submit``
|
||||
clears the buffer), once the hook's background read settles."""
|
||||
deadline = time.monotonic() + 30.0
|
||||
while True:
|
||||
raw = page.evaluate("() => window.__sseFrames || []")
|
||||
parsed = [json.loads(line) for line in raw if line]
|
||||
if any(f.get("type") == "done" for f in parsed):
|
||||
return parsed
|
||||
if time.monotonic() > deadline:
|
||||
raise AssertionError(
|
||||
f"SSE hook captured no `done` frame (frames so far: "
|
||||
f"{len(parsed)}) — hook install failed?"
|
||||
)
|
||||
time.sleep(0.05)
|
||||
|
||||
|
||||
def _tool_frames(frames: list[dict]) -> list[dict]:
|
||||
return [f for f in frames if f.get("type") == "tool"]
|
||||
|
||||
|
||||
def _submit(page: Page, question: str) -> None:
|
||||
page.evaluate("window.__sseFrames = []")
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
# The user bubble lands synchronously with the submit handler.
|
||||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||||
|
||||
|
||||
def _wait_settled(page: Page) -> None:
|
||||
"""The turn is complete: answer text in the bubble, button recovered
|
||||
(the phase-48 settle wait)."""
|
||||
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=60_000)
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=60_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=60_000)
|
||||
|
||||
|
||||
def _last_brain(page: Page) -> Locator:
|
||||
return page.locator(".msg.brain").last
|
||||
|
||||
|
||||
def _assert_drill_turn(
|
||||
page: Page,
|
||||
expected_tool: dict[str, Any],
|
||||
expected_lines: list[str],
|
||||
forbidden: list[str] | None = None,
|
||||
) -> None:
|
||||
"""One scripted drill turn, fully asserted: the wire carries exactly
|
||||
the expected ``tool`` frame (ahead of the first ``delta``), the
|
||||
bubble carries the expected listing lines (the mock's echo of the
|
||||
tool result the model received), the forbidden substrings are ABSENT
|
||||
(the no-suffix assertions), and the turn was grounded (the ``done``
|
||||
frame is not deflected)."""
|
||||
frames = _frames(page)
|
||||
assert _tool_frames(frames) == [expected_tool], _tool_frames(frames)
|
||||
first_delta = next(
|
||||
i for i, f in enumerate(frames) if f.get("type") == "delta"
|
||||
)
|
||||
assert all(
|
||||
i < first_delta for i, f in enumerate(frames) if f.get("type") == "tool"
|
||||
)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["deflected"] is False, done
|
||||
bubble = _last_brain(page).locator(".bubble")
|
||||
for line in expected_lines:
|
||||
expect(bubble).to_contain_text(line)
|
||||
text = bubble.text_content() or ""
|
||||
for needle in forbidden or []:
|
||||
assert needle not in text, text
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 1. Sync #1: the incident shape is retried (the row lands) and the
|
||||
# always-empty folder stays absent with the sync green
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_incident_reply_retried_and_always_empty_stays_absent(
|
||||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||||
) -> None:
|
||||
"""The ``ls oneshot`` drill-down shows the stored folder summaries
|
||||
the sync #1 produced under the incident-shape injection: the
|
||||
``e2e_empty_once/`` line carries its summary (the retry recovered
|
||||
the row — without task 01 the line would have NO ``: …`` suffix),
|
||||
the ``normal/`` line does too, and the ``e2e_empty_always/`` line
|
||||
is bare (exhausted → fail-soft → absent) — while the sync above
|
||||
reported success (the folder-stats failure never flips the run,
|
||||
phase 94)."""
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
|
||||
_submit(page, LS_QUESTION_1)
|
||||
_wait_settled(page)
|
||||
lines = _last_brain(page).locator(".tool-call")
|
||||
expect(lines).to_have_count(1)
|
||||
expect(lines.nth(0)).to_contain_text(f"Listing documents in {SOURCE}")
|
||||
_assert_drill_turn(
|
||||
page,
|
||||
{"type": "tool", "name": "ls", "argument": SOURCE},
|
||||
[
|
||||
LS_HEADER,
|
||||
# The bare line (the row is absent — no stored summary to
|
||||
# append) …
|
||||
LINE_ALWAYS,
|
||||
# …and the colon-suffixed lines (the rows the retry + the
|
||||
# normal path stored):
|
||||
LINE_ONCE,
|
||||
LINE_NORMAL,
|
||||
],
|
||||
# The ``e2e_empty_always/`` line must NOT carry a ``: …``
|
||||
# suffix — the bare line above is a prefix of the suffixed
|
||||
# shape, so the absence is pinned here (the all-4-attempts
|
||||
# exhaustion left no row to quote).
|
||||
forbidden=[LINE_ALWAYS_WITH_COLON],
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 2. The gap-fill: a deleted row self-heals on the next UNCHANGED sync,
|
||||
# the other rows untouched
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_deleted_row_self_heals_on_unchanged_sync(
|
||||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||||
) -> None:
|
||||
"""Delete the ``normal`` stored row behind the app's back (a
|
||||
historical failure), run the UNCHANGED sync #2, and assert the
|
||||
task-02/03 gap gate: ``missing_folder_summaries`` names the gap,
|
||||
``only_missing=True`` fills EXACTLY the missing rows, and every
|
||||
other row stays byte-identical (text AND ``updated_at``).
|
||||
|
||||
* ``normal`` — healed: the second ``ls oneshot`` turn carries
|
||||
``: Fixture folder summary for oneshot/normal.`` again, and the
|
||||
DB row is back with its deterministic text;
|
||||
* ``e2e_empty_once`` — the gap-fill NEVER calls it (the row
|
||||
exists): its ``updated_at`` is UNCHANGED (a full regeneration
|
||||
would have re-stamped it);
|
||||
* ``e2e_empty_always`` — the gap-fill attempted it, exhausted
|
||||
(4 empty POSTs by design), stayed ABSENT — and the sync still
|
||||
succeeded (the fail-soft never flips the run)."""
|
||||
page.set_default_timeout(30_000)
|
||||
# The gap: delete the ``normal`` row directly (the test process has
|
||||
# DB access via the conftest engine — the same connection the app
|
||||
# uses), capturing the other row's stamp for the targeted-fill
|
||||
# assertion. No KB change anywhere.
|
||||
before = _folder_rows()
|
||||
assert FOLDER_NORMAL in before # the row existed (sync #1 stored it)
|
||||
updated_at_once = before[FOLDER_ONCE][1]
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"DELETE FROM folder_summaries "
|
||||
"WHERE source = :s AND folder_path = :f"
|
||||
),
|
||||
{"s": SOURCE, "f": FOLDER_NORMAL},
|
||||
)
|
||||
db.commit()
|
||||
assert FOLDER_NORMAL not in _folder_rows()
|
||||
|
||||
# Sync #2 — the KB is UNCHANGED (nothing in the temp tree moved),
|
||||
# so the gate takes the gap probe: two candidates missing
|
||||
# (``e2e_empty_always`` + ``normal``) → the targeted fill.
|
||||
body = _run_sync_http(app_url)
|
||||
assert body["state"] == "success", body # exhaustion never flips the run
|
||||
detail = body["detail"]
|
||||
assert detail["added"] == 0, detail
|
||||
assert detail["updated"] == 0, detail
|
||||
assert detail["pruned"] == 0, detail
|
||||
assert detail["overview"] is False, detail # unchanged → no overview burn
|
||||
|
||||
# The second scripted drill turn: the healed + the surviving lines,
|
||||
# the always folder still bare.
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
_submit(page, LS_QUESTION_2)
|
||||
_wait_settled(page)
|
||||
lines = _last_brain(page).locator(".tool-call")
|
||||
expect(lines).to_have_count(1)
|
||||
expect(lines.nth(0)).to_contain_text(f"Listing documents in {SOURCE}")
|
||||
_assert_drill_turn(
|
||||
page,
|
||||
{"type": "tool", "name": "ls", "argument": SOURCE},
|
||||
[
|
||||
LS_HEADER,
|
||||
LINE_ALWAYS,
|
||||
LINE_ONCE,
|
||||
LINE_NORMAL, # healed — the suffix is back
|
||||
],
|
||||
forbidden=[LINE_ALWAYS_WITH_COLON],
|
||||
)
|
||||
|
||||
# The DB pins the targeted fill (the E2E's other established lens):
|
||||
rows = _folder_rows()
|
||||
assert set(rows) == {"", FOLDER_ONCE, FOLDER_NORMAL}, rows
|
||||
assert rows[FOLDER_NORMAL][0] == SUM_NORMAL, rows # deterministic text
|
||||
# The targeted fill never touched the other rows — a FULL
|
||||
# regeneration would have re-stamped this one (the ``_upsert``
|
||||
# fresh-UTC-stamp rule).
|
||||
assert rows[FOLDER_ONCE][1] == updated_at_once, (
|
||||
rows[FOLDER_ONCE][1],
|
||||
updated_at_once,
|
||||
)
|
||||
assert FOLDER_ALWAYS not in rows, rows # exhausted again → still absent
|
||||
@@ -111,8 +111,10 @@ def _stub_folder_summaries(monkeypatch: pytest.MonkeyPatch) -> list[dict]:
|
||||
the call record; the canned stats are the zero dict."""
|
||||
calls: list[dict] = []
|
||||
|
||||
async def fake_generate(db: object, llm: object, *, skip: bool = False) -> dict[str, int]:
|
||||
calls.append({"skip": skip})
|
||||
async def fake_generate(
|
||||
db: object, llm: object, *, skip: bool = False, only_missing: bool = False
|
||||
) -> dict[str, int]:
|
||||
calls.append({"skip": skip, "only_missing": only_missing})
|
||||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||||
|
||||
monkeypatch.setattr(import_docs, "generate_folder_summaries", fake_generate)
|
||||
|
||||
@@ -303,7 +303,12 @@ class FakeFolderSummaries:
|
||||
layer boundary — the real generator would read the global
|
||||
``documents`` table and call the (real) ``LLMClient`` over the
|
||||
network. The generator only flushes, so the fake honours the
|
||||
``skip`` flag the same way (the zero stats, no side effects)."""
|
||||
``skip`` flag the same way (the zero stats, no side effects).
|
||||
Phase 96 (task 03): the unchanged-walk gap path calls the
|
||||
generator with ``only_missing=True`` — the fake records the flag
|
||||
the same way it records ``skip`` (the real gap probe,
|
||||
``missing_folder_summaries``, runs against the real tables).
|
||||
"""
|
||||
|
||||
ZERO = {"generated": 0, "failed": 0, "pruned": 0}
|
||||
|
||||
@@ -312,11 +317,18 @@ class FakeFolderSummaries:
|
||||
self.llms: list[LLMClient] = []
|
||||
self.sessions: list[Session] = []
|
||||
self.skip_flags: list[bool] = []
|
||||
self.only_missing_flags: list[bool] = []
|
||||
|
||||
async def __call__(
|
||||
self, db: Session, llm: LLMClient, *, skip: bool = False
|
||||
self,
|
||||
db: Session,
|
||||
llm: LLMClient,
|
||||
*,
|
||||
skip: bool = False,
|
||||
only_missing: bool = False,
|
||||
) -> dict[str, int]:
|
||||
self.skip_flags.append(skip)
|
||||
self.only_missing_flags.append(only_missing)
|
||||
if skip:
|
||||
return dict(self.ZERO)
|
||||
self.llms.append(llm)
|
||||
|
||||
@@ -14,9 +14,19 @@ DB, explicit ``--source``):
|
||||
- a KB-changing import → one row per ≥ 2-doc subtree (the source root
|
||||
+ the 2-doc folder; the 1-doc folder gets none), committed in the
|
||||
run's transaction, the summary line ending
|
||||
``folder_summaries=<generated>/<failed>/<pruned>``;
|
||||
- an unchanged re-import → zero ``lite`` calls,
|
||||
``folder_summaries=skipped``, rows untouched;
|
||||
``folder_summaries=<generated>/<failed>/<pruned>`` (unchanged by
|
||||
phase 96 — no gap-fill suffix on a full regeneration);
|
||||
- an unchanged re-import with a COMPLETE table → zero ``lite`` calls,
|
||||
``folder_summaries=skipped``, rows untouched (the phase-94
|
||||
zero-burn invariant);
|
||||
- an unchanged re-import with a GAP (one stored row deleted) →
|
||||
exactly one ``FOLDER_SUMMARY_MODE`` call (the missing folder only),
|
||||
the row back with the deterministic fake text, every other row
|
||||
byte-identical (summary AND ``updated_at``), the line ending
|
||||
``folder_summaries=1/0/0 (gap-fill)`` (phase 96, task 03 — the
|
||||
failed folder summary self-heals on the next sync);
|
||||
- a KB change on a second run → still a FULL regeneration (call count
|
||||
== candidate count, every row re-stamped, no gap-fill suffix);
|
||||
- a subtree dropping below 2 docs after a changed re-walk → its row
|
||||
pruned;
|
||||
- one folder's ``lite`` failure → its previous row kept, the other
|
||||
@@ -24,7 +34,9 @@ DB, explicit ``--source``):
|
||||
- a ``--limit`` debug run → no generation, no rows,
|
||||
``folder_summaries=skipped``;
|
||||
- a fresh (empty) table after a ``--limit`` first walk → an unchanged
|
||||
full walk generates (the table-empty first-run trigger).
|
||||
full walk generates via the gap-fill path (the subsumed table-empty
|
||||
first-run trigger — every candidate is missing), the line carrying
|
||||
`` (gap-fill)``.
|
||||
|
||||
API path (``POST /api/sync`` end to end, real import over a host temp
|
||||
local dir, deterministic ``FakeEmbedder``):
|
||||
@@ -32,14 +44,20 @@ local dir, deterministic ``FakeEmbedder``):
|
||||
- a KB-changing sync → the rows land (visible via the test's own
|
||||
session) and the status detail keeps its exact pre-phase key set
|
||||
(no folder-summary surface — the stats are log-only);
|
||||
- an unchanged re-sync → zero ``FOLDER_SUMMARY_MODE`` calls;
|
||||
- an unchanged re-sync (complete table) → zero ``FOLDER_SUMMARY_MODE``
|
||||
calls (the phase-94 zero-burn invariant);
|
||||
- an unchanged re-sync with a GAP (one stored row deleted) → targeted
|
||||
fill of exactly that row (one ``FOLDER_SUMMARY_MODE`` call, the
|
||||
``gap-fill`` log line), every other row byte-identical, the status
|
||||
detail shape untouched (phase 96, task 03);
|
||||
- a ``lite`` outage (one folder failing) → the failed folder's row is
|
||||
kept, the run reports ``success`` (never ``failed``), and the
|
||||
sources-version bump still lands (the bump is change-gated on the
|
||||
KB, not on the summaries);
|
||||
- an empty table after a populated sync (the migration-0017 scenario)
|
||||
→ an unchanged walk regenerates (the overview's API gate, purely
|
||||
change-gated, does not).
|
||||
→ an unchanged walk regenerates via the gap probe (the subsumed
|
||||
table-empty trigger; the overview's API gate, purely change-gated,
|
||||
does not fire).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -290,6 +308,101 @@ def test_unchanged_reimport_burns_zero_folder_calls(
|
||||
assert _rows(db) == rows # rows byte-identical
|
||||
|
||||
|
||||
def test_unchanged_reimport_with_gap_fills_only_the_missing_row(
|
||||
db: Session,
|
||||
src: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""Phase 96 (task 03): an unchanged walk with ONE deleted stored
|
||||
row → exactly one ``FOLDER_SUMMARY_MODE`` call (the deleted
|
||||
folder only), the row back with the deterministic fake text, every
|
||||
OTHER row byte-identical (summary AND ``updated_at``), the summary
|
||||
line ending ``folder_summaries=1/0/0 (gap-fill)`` — the failed
|
||||
folder summary self-heals on the next sync instead of persisting
|
||||
until a KB change."""
|
||||
llm1 = FakeEmbedder()
|
||||
rc, out = _run_main(monkeypatch, llm1, ["--source", str(src)], capsys)
|
||||
assert rc == 0
|
||||
rows_before = _rows(db)
|
||||
assert set(rows_before) == {("MyDocs", ""), ("MyDocs", "a")}
|
||||
root_stamp_before = _updated_at(db, "MyDocs", "")
|
||||
assert root_stamp_before is not None
|
||||
|
||||
# Simulate the phase-96 incident: a lost row (an exhausted
|
||||
# one-shot retry leaves a candidate without its row).
|
||||
db.execute(
|
||||
text(
|
||||
"DELETE FROM folder_summaries "
|
||||
"WHERE source = 'MyDocs' AND folder_path = 'a'"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
llm2 = FakeEmbedder()
|
||||
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
|
||||
assert rc == 0
|
||||
assert "unchanged=3" in out
|
||||
assert out.rstrip().endswith(
|
||||
"overview=skipped sources_version=skipped "
|
||||
"folder_summaries=1/0/0 (gap-fill)"
|
||||
)
|
||||
# Exactly ONE new folder call — the deleted row's folder only.
|
||||
calls = _folder_calls(llm2)
|
||||
assert len(calls) == 1
|
||||
assert calls[0][1]["content"].splitlines()[0] == "Folder: MyDocs/a"
|
||||
assert len(llm2.chat_calls) == 1 # no other lite traffic at all
|
||||
# The row is back with the deterministic fake text ...
|
||||
rows_after = _rows(db)
|
||||
assert rows_after == rows_before
|
||||
# ... and every OTHER row byte-identical (the root was never
|
||||
# re-stamped by the targeted fill).
|
||||
assert _updated_at(db, "MyDocs", "") == root_stamp_before
|
||||
|
||||
|
||||
def test_changed_reimport_is_a_full_regeneration(
|
||||
db: Session,
|
||||
src: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""Phase 96 (task 03): a KB change is STILL a full regeneration —
|
||||
call count == candidate count, every row re-stamped, and NO
|
||||
`` (gap-fill)`` suffix (byte-identical to today's behavior)."""
|
||||
llm1 = FakeEmbedder()
|
||||
rc, _ = _run_main(monkeypatch, llm1, ["--source", str(src)], capsys)
|
||||
assert rc == 0
|
||||
rows_before = _rows(db)
|
||||
root_stamp_before = _updated_at(db, "MyDocs", "")
|
||||
a_stamp_before = _updated_at(db, "MyDocs", "a")
|
||||
assert root_stamp_before is not None and a_stamp_before is not None
|
||||
|
||||
# A KB change (one doc edited) — the gate is the change, not the
|
||||
# gap.
|
||||
(src / "a" / "one.md").write_text("# A One\nChanged content.\n", encoding="utf-8")
|
||||
llm2 = FakeEmbedder()
|
||||
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
|
||||
assert rc == 0
|
||||
assert "updated=1" in out
|
||||
# Full-regeneration token — the stats without the gap-fill suffix.
|
||||
assert out.rstrip().endswith(
|
||||
"overview=updated sources_version=2 folder_summaries=2/0/0"
|
||||
)
|
||||
# Call count == candidate count — BOTH folders, not a targeted fill.
|
||||
calls = _folder_calls(llm2)
|
||||
assert [c[1]["content"].splitlines()[0] for c in calls] == [
|
||||
"Folder: MyDocs",
|
||||
"Folder: MyDocs/a",
|
||||
]
|
||||
# All rows re-stamped (the full regeneration re-writes every
|
||||
# candidate, even the unchanging one).
|
||||
root_stamp_after = _updated_at(db, "MyDocs", "")
|
||||
a_stamp_after = _updated_at(db, "MyDocs", "a")
|
||||
assert root_stamp_after is not None and root_stamp_after > root_stamp_before
|
||||
assert a_stamp_after is not None and a_stamp_after > a_stamp_before
|
||||
assert _rows(db) == rows_before # deterministic fake → same texts
|
||||
|
||||
|
||||
def test_subtree_dropping_below_two_docs_is_pruned(
|
||||
db: Session,
|
||||
src: Path,
|
||||
@@ -382,10 +495,11 @@ def test_empty_table_generates_on_unchanged_walk(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""The table-empty first-run trigger: after a ``--limit`` first
|
||||
walk (populated KB, empty table), an unchanged full walk generates
|
||||
— for the folder summaries AND the missing outline, still never
|
||||
bumping the version."""
|
||||
"""The subsumed table-empty first-run trigger (phase 96, task
|
||||
03): after a ``--limit`` first walk (populated KB, empty table),
|
||||
an unchanged full walk generates — for the folder summaries (now
|
||||
via the gap-fill path — every candidate is missing) AND the
|
||||
missing outline, still never bumping the version."""
|
||||
llm1 = FakeEmbedder()
|
||||
rc, out = _run_main(monkeypatch, llm1, ["--source", str(src), "--limit", "3"], capsys)
|
||||
assert rc == 0
|
||||
@@ -399,8 +513,13 @@ def test_empty_table_generates_on_unchanged_walk(
|
||||
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
|
||||
assert rc == 0
|
||||
assert "unchanged=3" in out
|
||||
# Phase 96 (task 03): the old table-empty trigger is now the
|
||||
# subsumed gap case — every candidate is missing, so the unchanged
|
||||
# walk takes the targeted-fill path and the token carries
|
||||
# `` (gap-fill)`` (the generated set is the full candidate set).
|
||||
assert out.rstrip().endswith(
|
||||
"overview=updated sources_version=skipped folder_summaries=2/0/0"
|
||||
"overview=updated sources_version=skipped "
|
||||
"folder_summaries=2/0/0 (gap-fill)"
|
||||
)
|
||||
assert set(_rows(db)) == {("MyDocs", ""), ("MyDocs", "a")}
|
||||
assert len(_folder_calls(llm2)) == 2
|
||||
@@ -576,6 +695,86 @@ def test_api_unchanged_resync_burns_zero_folder_calls(
|
||||
assert current_sources_version(db) == 1
|
||||
|
||||
|
||||
def test_api_unchanged_resync_with_gap_fills_only_the_missing_row(
|
||||
sync_client: TestClient,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
db: Session,
|
||||
local_dir: Path,
|
||||
) -> None:
|
||||
"""Phase 96 (task 03), API path: an unchanged re-sync with ONE
|
||||
deleted stored row → targeted fill of exactly that row (one
|
||||
``FOLDER_SUMMARY_MODE`` call), the ``gap-fill`` log line, every
|
||||
other row byte-identical (summary AND ``updated_at``), status
|
||||
``success``, and the status detail shape untouched (the stats stay
|
||||
log-only — the phase-94 contract)."""
|
||||
_seed_local(db, local_dir)
|
||||
_stub_env(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
sync_api,
|
||||
"get_settings",
|
||||
lambda: _settings(str(local_dir.parent / "bor")),
|
||||
)
|
||||
clients = _capture_llm(monkeypatch)
|
||||
|
||||
_login(sync_client)
|
||||
assert sync_client.post("/api/sync").status_code == 202
|
||||
_poll(sync_client, "success")
|
||||
rows_before = _rows(db)
|
||||
assert set(rows_before) == {("LocalDocs", ""), ("LocalDocs", "a")}
|
||||
root_stamp_before = _updated_at(db, "LocalDocs", "")
|
||||
assert root_stamp_before is not None
|
||||
|
||||
# The phase-96 incident shape: a lost row, deleted directly.
|
||||
db.execute(
|
||||
text(
|
||||
"DELETE FROM folder_summaries "
|
||||
"WHERE source = 'LocalDocs' AND folder_path = 'a'"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
records: list[logging.LogRecord] = []
|
||||
|
||||
class _Sink(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
records.append(record)
|
||||
|
||||
sync_logger = logging.getLogger("app.api.sync")
|
||||
sink = _Sink()
|
||||
sync_logger.addHandler(sink)
|
||||
sync_logger.setLevel(logging.INFO)
|
||||
try:
|
||||
assert sync_client.post("/api/sync").status_code == 202
|
||||
body = _poll(sync_client, "success")
|
||||
finally:
|
||||
sync_logger.removeHandler(sink)
|
||||
|
||||
assert body["error"] is None
|
||||
assert body["detail"]["overview"] is False # unchanged → no overview
|
||||
assert body["detail"]["sources_version"] == 1 # unchanged → no bump
|
||||
# No new sync-status surface: the detail keeps its exact key set.
|
||||
assert set(body["detail"]) == {
|
||||
"files", "added", "updated", "unchanged", "pruned", "errors",
|
||||
"chunks", "summaries", "summary_errors", "overview",
|
||||
"sources_version",
|
||||
}
|
||||
assert len(clients) == 2
|
||||
# Targeted fill — exactly ONE folder call, the missing folder only
|
||||
# (plus the phase-41 probe's ping on the same client).
|
||||
calls = _folder_calls(clients[1])
|
||||
assert len(calls) == 1
|
||||
assert calls[0][1]["content"].splitlines()[0] == "Folder: LocalDocs/a"
|
||||
assert len(clients[1].chat_calls) == 2 # ping + the one fill
|
||||
# The row is back with the deterministic fake text ...
|
||||
assert _rows(db) == rows_before
|
||||
# ... every other row byte-identical (the root was never re-stamped
|
||||
# by the targeted fill).
|
||||
assert _updated_at(db, "LocalDocs", "") == root_stamp_before
|
||||
# The gap-fill log line (PLAN §9 ample logging).
|
||||
assert any(
|
||||
"sync: folder_summaries gap-fill" in r.getMessage() for r in records
|
||||
)
|
||||
|
||||
|
||||
def test_api_folder_lite_failure_keeps_rows_stays_green_and_bumps(
|
||||
sync_client: TestClient,
|
||||
@@ -634,10 +833,12 @@ def test_api_empty_table_first_sync_regenerates(
|
||||
db: Session,
|
||||
local_dir: Path,
|
||||
) -> None:
|
||||
"""The migration-0017 scenario: the KB predates the table — wipe
|
||||
the rows and re-sync an unchanged KB: the empty-table trigger
|
||||
fires for the folder summaries (the overview's API gate, purely
|
||||
change-gated, does not)."""
|
||||
"""The migration-0017 scenario (the subsumed table-empty trigger —
|
||||
phase 96, task 03): the KB predates the table — wipe the rows and
|
||||
re-sync an unchanged KB: the gap probe fires (every candidate is
|
||||
missing) and the targeted fill regenerates the full candidate set
|
||||
for the folder summaries (the overview's API gate, purely
|
||||
change-gated, does not fire)."""
|
||||
_seed_local(db, local_dir)
|
||||
_stub_env(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
|
||||
@@ -31,9 +31,9 @@ from app.rag.folder_summaries import (
|
||||
SYSTEM_PROMPT,
|
||||
build_folder_summary_prompt,
|
||||
folder_of,
|
||||
folder_summary_table_empty,
|
||||
generate_folder_summaries,
|
||||
group_by_folder,
|
||||
missing_folder_summaries,
|
||||
summarize_folder,
|
||||
)
|
||||
from app.rag.llm import LLMError
|
||||
@@ -613,16 +613,198 @@ def test_generate_only_flushes_caller_commits(db: Session, clean_tables) -> None
|
||||
assert MIN_DOCS_PER_FOLDER == 2 # the ≥ 2 scope rule, pinned by name
|
||||
|
||||
|
||||
def test_folder_summary_table_empty_gate(db: Session, clean_tables) -> None:
|
||||
"""The sync-path gate probe (phase 94, task 02): empty → True
|
||||
(the first full sync after migration 0017 must still generate),
|
||||
one row → False (a populated table waits for a KB change)."""
|
||||
assert folder_summary_table_empty(db) is True # the truncated table
|
||||
_add_doc(db, "FSU", "a/one.md", "One")
|
||||
_add_doc(db, "FSU", "a/two.md", "Two")
|
||||
# ---------- missing_folder_summaries (phase 96, task 02) ----------
|
||||
|
||||
|
||||
def test_missing_fresh_table_is_exactly_the_candidate_set(
|
||||
db: Session, clean_tables
|
||||
) -> None:
|
||||
"""No stored rows → every candidate folder is a gap, sorted by
|
||||
``(source, folder_path)``; the single-doc FSU-solo root is not a
|
||||
candidate and can never be a gap."""
|
||||
_seed_catalogue(db)
|
||||
assert missing_folder_summaries(db) == [
|
||||
("FSU", ""),
|
||||
("FSU", "a"),
|
||||
("FSU", "a/b"),
|
||||
]
|
||||
assert ("FSU-solo", "") not in missing_folder_summaries(db)
|
||||
|
||||
|
||||
def test_missing_fully_populated_table_is_empty(db: Session, clean_tables) -> None:
|
||||
"""Every candidate row present → no gap (the zero-burn gate case)."""
|
||||
_seed_catalogue(db)
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
assert folder_summary_table_empty(db) is False # rows landed
|
||||
assert missing_folder_summaries(db) == []
|
||||
|
||||
|
||||
def test_missing_one_deleted_row_is_that_folder(db: Session, clean_tables) -> None:
|
||||
_seed_catalogue(db)
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
db.execute(
|
||||
text(
|
||||
"DELETE FROM folder_summaries "
|
||||
"WHERE source = 'FSU' AND folder_path = 'a'"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
assert missing_folder_summaries(db) == [("FSU", "a")]
|
||||
|
||||
|
||||
def test_missing_empty_kb_empty_table_is_no_gap(db: Session, clean_tables) -> None:
|
||||
"""No catalogue → no candidates → ``[]`` — an empty table over an
|
||||
empty KB is not a gap (there is nothing to fill)."""
|
||||
assert missing_folder_summaries(db) == []
|
||||
|
||||
|
||||
def test_missing_single_doc_folder_is_never_listed(db: Session, clean_tables) -> None:
|
||||
"""A below-minimum folder without a row is NOT a gap — it is not a
|
||||
candidate (its one file line IS its summary)."""
|
||||
_add_doc(db, "FSU", "solo/one.md", "One")
|
||||
assert missing_folder_summaries(db) == []
|
||||
|
||||
|
||||
def test_missing_stale_row_is_not_a_gap(db: Session, clean_tables) -> None:
|
||||
"""A stored row for a folder that dropped below 2 docs is stale,
|
||||
not missing — the prune pass owns it, the gap detector ignores it."""
|
||||
_seed_catalogue(db)
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
db.add(FolderSummary(source="FSU", folder_path="gone/old", summary="stale"))
|
||||
db.commit()
|
||||
assert missing_folder_summaries(db) == []
|
||||
|
||||
|
||||
# ---------- generate_folder_summaries(only_missing=…) (phase 96, 02) ----------
|
||||
|
||||
|
||||
def _updated_at(db: Session, source: str, folder_path: str) -> object:
|
||||
"""The stored row's ``updated_at`` (raw SQL — bypasses the ORM
|
||||
identity map, so the before/after byte-identity comparison is
|
||||
honest)."""
|
||||
return db.execute(
|
||||
text(
|
||||
"SELECT updated_at FROM folder_summaries "
|
||||
"WHERE source = :s AND folder_path = :f"
|
||||
),
|
||||
{"s": source, "f": folder_path},
|
||||
).scalar_one()
|
||||
|
||||
|
||||
def test_only_missing_fills_exactly_the_missing_keys(
|
||||
db: Session, clean_tables
|
||||
) -> None:
|
||||
"""Two missing + two present → exactly the missing keys are
|
||||
generated (sorted order, one lite call each); the present rows are
|
||||
byte-identical after (text AND ``updated_at``); stats right."""
|
||||
_seed_catalogue(db)
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
full = _rows(db)
|
||||
a_stamp = _updated_at(db, "FSU", "a")
|
||||
db.execute(
|
||||
text(
|
||||
"DELETE FROM folder_summaries "
|
||||
"WHERE (source, folder_path) IN (('FSU', ''), ('FSU', 'a/b'))"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
assert missing_folder_summaries(db) == [("FSU", ""), ("FSU", "a/b")]
|
||||
|
||||
llm = _FakeLLM()
|
||||
stats = asyncio.run(generate_folder_summaries(db, llm, only_missing=True))
|
||||
assert stats == {"generated": 2, "failed": 0, "pruned": 0}
|
||||
assert llm.calls == 2, "one call per MISSING key — zero for present rows"
|
||||
assert [user.splitlines()[0] for _s, user in llm.requests] == [
|
||||
"Folder: FSU",
|
||||
"Folder: FSU/a/b",
|
||||
], "the missing keys in sorted (source, folder_path) order"
|
||||
|
||||
assert _rows(db) == full, "the fill restores exactly the full candidate set"
|
||||
assert _updated_at(db, "FSU", "a") == a_stamp, (
|
||||
"the present row is byte-identical — never re-stamped by the fill"
|
||||
)
|
||||
|
||||
|
||||
def test_only_missing_no_gap_burns_zero_calls(db: Session, clean_tables) -> None:
|
||||
"""No gap → zero lite calls, zero rows touched, zero stats (the
|
||||
zero-burn invariant the unchanged-sync gate relies on)."""
|
||||
_seed_catalogue(db)
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
before = _rows(db)
|
||||
stamps = {f: _updated_at(db, "FSU", f) for f in ("", "a", "a/b")}
|
||||
llm = _FakeLLM()
|
||||
stats = asyncio.run(generate_folder_summaries(db, llm, only_missing=True))
|
||||
assert stats == {"generated": 0, "failed": 0, "pruned": 0}
|
||||
assert llm.calls == 0, "zero-burn: no gap, no lite call"
|
||||
assert _rows(db) == before
|
||||
for folder, stamp in stamps.items():
|
||||
assert _updated_at(db, "FSU", folder) == stamp, "no row re-stamped"
|
||||
|
||||
|
||||
def test_only_missing_still_prunes_stale_rows(db: Session, clean_tables) -> None:
|
||||
"""The prune pass runs in BOTH modes: the manually seeded stale row
|
||||
(folder gone from the catalogue) is pruned while the genuine
|
||||
missing folders are filled, and the present row stays untouched."""
|
||||
_seed_catalogue(db)
|
||||
db.add(FolderSummary(source="FSU", folder_path="a", summary="keep me"))
|
||||
db.add(FolderSummary(source="FSU", folder_path="gone/old", summary="stale"))
|
||||
db.commit()
|
||||
a_stamp = _updated_at(db, "FSU", "a")
|
||||
assert missing_folder_summaries(db) == [("FSU", ""), ("FSU", "a/b")]
|
||||
|
||||
llm = _FakeLLM()
|
||||
stats = asyncio.run(generate_folder_summaries(db, llm, only_missing=True))
|
||||
assert stats == {"generated": 2, "failed": 0, "pruned": 1}
|
||||
assert llm.calls == 2
|
||||
|
||||
stored = _rows(db)
|
||||
assert ("FSU", "gone/old") not in stored, (
|
||||
"the stale row is pruned even under only_missing"
|
||||
)
|
||||
assert stored[("FSU", "a")] == "keep me"
|
||||
assert _updated_at(db, "FSU", "a") == a_stamp
|
||||
assert stored[("FSU", "")] == REPLY and stored[("FSU", "a/b")] == REPLY
|
||||
|
||||
|
||||
def test_only_missing_fail_soft_keeps_prior_and_lands_others(
|
||||
db: Session, clean_tables
|
||||
) -> None:
|
||||
"""Per-folder fail-soft applies under ``only_missing`` too: the
|
||||
failing missing folder is counted and stays absent; the other
|
||||
missing folders still land; the present row is untouched."""
|
||||
_seed_catalogue(db)
|
||||
db.add(FolderSummary(source="FSU", folder_path="a", summary="keep me"))
|
||||
db.commit()
|
||||
llm = _FakeLLM(fail_folders=("FSU/a/b",))
|
||||
stats = asyncio.run(generate_folder_summaries(db, llm, only_missing=True))
|
||||
assert stats == {"generated": 1, "failed": 1, "pruned": 0}
|
||||
assert llm.calls == 2 # both missing folders were attempted
|
||||
|
||||
stored = _rows(db)
|
||||
assert stored[("FSU", "")] == REPLY, "the other missing folder still lands"
|
||||
assert ("FSU", "a/b") not in stored, "the failed folder stays absent"
|
||||
assert stored[("FSU", "a")] == "keep me", "the present row is untouched"
|
||||
|
||||
|
||||
def test_gap_probe_subsumes_the_table_empty_gate(db: Session, clean_tables) -> None:
|
||||
"""The deleted phase-94 table-empty gate probe, re-expressed through
|
||||
``missing_folder_summaries`` (phase 96, task 03 — the probe's unit
|
||||
coverage moved here): an empty table over a populated catalogue
|
||||
means EVERY candidate is missing (the targeted fill over all
|
||||
candidates IS a full generation — the first full sync after
|
||||
migration 0017 must still generate), a populated table means no
|
||||
gap (a populated table waits for a KB change or a gap)."""
|
||||
assert missing_folder_summaries(db) == [] # the truncated table, empty KB
|
||||
_add_doc(db, "FSU", "a/one.md", "One")
|
||||
_add_doc(db, "FSU", "a/two.md", "Two")
|
||||
assert missing_folder_summaries(db) == [("FSU", ""), ("FSU", "a")]
|
||||
asyncio.run(generate_folder_summaries(db, _FakeLLM()))
|
||||
db.commit()
|
||||
assert missing_folder_summaries(db) == [] # rows landed → no gap
|
||||
db.execute(text("DELETE FROM folder_summaries"))
|
||||
db.commit()
|
||||
assert folder_summary_table_empty(db) is True # emptied again
|
||||
assert missing_folder_summaries(db) == [("FSU", ""), ("FSU", "a")] # emptied again
|
||||
|
||||
@@ -10,6 +10,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import AsyncGenerator
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, cast
|
||||
@@ -346,14 +347,27 @@ class _FakeCompletion:
|
||||
"""One fake non-streaming ChatCompletion (``choices[].message`` shape).
|
||||
|
||||
``content=None`` mirrors the real wire where the field can be absent or
|
||||
empty (reasoning-only replies, provider quirks).
|
||||
empty (reasoning-only replies, provider quirks). ``finish_reason``
|
||||
(phase 96) defaults to ``None`` — the provider omitting it — and the
|
||||
incident signature is ``"length"`` (the whole ``max_tokens`` budget
|
||||
spent in ``reasoning_content``).
|
||||
"""
|
||||
|
||||
def __init__(self, content: str | None, empty_choices: bool = False) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
content: str | None,
|
||||
empty_choices: bool = False,
|
||||
finish_reason: str | None = None,
|
||||
) -> None:
|
||||
if empty_choices:
|
||||
self.choices = []
|
||||
else:
|
||||
self.choices = [SimpleNamespace(message=SimpleNamespace(content=content))]
|
||||
self.choices = [
|
||||
SimpleNamespace(
|
||||
message=SimpleNamespace(content=content),
|
||||
finish_reason=finish_reason,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
class _FakeCompletions:
|
||||
@@ -362,12 +376,25 @@ class _FakeCompletions:
|
||||
chunks: list | None = None,
|
||||
fail: Exception | None = None,
|
||||
completion: _FakeCompletion | None = None,
|
||||
completion_seq: list[_FakeCompletion] | None = None,
|
||||
) -> None:
|
||||
self.chunks = chunks or []
|
||||
self.fail = fail
|
||||
self.completion = completion
|
||||
#: Phase 96: a scripted per-``create()`` reply sequence (the retry
|
||||
#: matrix) — popped one per non-streaming call, in order.
|
||||
self.completion_seq = (
|
||||
list(completion_seq) if completion_seq is not None else None
|
||||
)
|
||||
self.kwargs: dict | None = None
|
||||
self.chat_kwargs: dict | None = None
|
||||
#: Every non-streaming ``create()`` call's kwargs (the attempt
|
||||
#: counter for the phase-96 retry matrix).
|
||||
self.chat_calls: list[dict] = []
|
||||
#: Every ``create()`` call (streaming + non-streaming), incl.
|
||||
#: calls that raised (``fail``) — the attempt counter when the
|
||||
#: failure happens inside the SDK call itself.
|
||||
self.create_calls: int = 0
|
||||
#: Every SDK-shaped stream handed out — teardown tests assert the
|
||||
#: phase-48 ``close()`` on them (phase 71 task 02: with/without
|
||||
#: a filter, the teardown path is the same object).
|
||||
@@ -375,6 +402,7 @@ class _FakeCompletions:
|
||||
|
||||
async def create(self, **kwargs) -> _FakeChatStream | _FakeCompletion:
|
||||
self.kwargs = kwargs
|
||||
self.create_calls += 1
|
||||
if self.fail is not None:
|
||||
raise self.fail
|
||||
if kwargs.get("stream"):
|
||||
@@ -382,6 +410,11 @@ class _FakeCompletions:
|
||||
self.streams.append(stream)
|
||||
return stream
|
||||
self.chat_kwargs = kwargs
|
||||
self.chat_calls.append(dict(kwargs))
|
||||
if self.completion_seq is not None:
|
||||
if not self.completion_seq:
|
||||
raise AssertionError("completion script exhausted")
|
||||
return self.completion_seq.pop(0)
|
||||
assert self.completion is not None
|
||||
return self.completion
|
||||
|
||||
@@ -904,9 +937,12 @@ def test_chat_stream_abandon_with_filter_closes_stream() -> None:
|
||||
def _make_chat_client(
|
||||
completion: _FakeCompletion | None = None,
|
||||
fail: Exception | None = None,
|
||||
completion_seq: list[_FakeCompletion] | None = None,
|
||||
**settings_kwargs: Any,
|
||||
) -> tuple[LLMClient, _FakeCompletions]:
|
||||
completions = _FakeCompletions(fail=fail, completion=completion)
|
||||
completions = _FakeCompletions(
|
||||
fail=fail, completion=completion, completion_seq=completion_seq
|
||||
)
|
||||
fake_openai = SimpleNamespace(chat=SimpleNamespace(completions=completions))
|
||||
llm = LLMClient(_settings(**settings_kwargs))
|
||||
llm._client = fake_openai # pyright: ignore[reportAttributeAccessIssue]
|
||||
@@ -984,18 +1020,196 @@ def test_chat_empty_choices_raises_llm_error() -> None:
|
||||
|
||||
|
||||
def test_chat_missing_content_raises_llm_error() -> None:
|
||||
"""A silent empty summary must never be stored — None content fails."""
|
||||
llm, _ = _make_chat_client(_FakeCompletion(None))
|
||||
"""A silent empty summary must never be stored — None content fails.
|
||||
Phase 96: pinned with the kill switch (``llm_retries=0``) so the
|
||||
pre-phase-96 single-attempt behavior and message are asserted
|
||||
verbatim (the retry matrix below pins the retried contract)."""
|
||||
llm, _ = _make_chat_client(_FakeCompletion(None), llm_retries=0)
|
||||
with pytest.raises(LLMError, match="empty content"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
|
||||
|
||||
def test_chat_whitespace_only_content_raises_llm_error() -> None:
|
||||
llm, _ = _make_chat_client(_FakeCompletion(" \n\t "))
|
||||
"""Whitespace-only content is empty (phase 96 kill-switch pin)."""
|
||||
llm, _ = _make_chat_client(_FakeCompletion(" \n\t "), llm_retries=0)
|
||||
with pytest.raises(LLMError, match="empty content"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
|
||||
|
||||
# ---------- one-shot empty-reply retry (phase 96, task 01) ----------
|
||||
|
||||
_DEFAULT_BASE = "https://aipi.reeseapps.com/v1"
|
||||
|
||||
|
||||
def _empty(finish_reason: str | None = "length") -> _FakeCompletion:
|
||||
"""An incident-shaped empty reply (``content=None``; ``finish_reason``
|
||||
defaults to ``"length"`` — the 2026-09-11 signature)."""
|
||||
return _FakeCompletion(None, finish_reason=finish_reason)
|
||||
|
||||
|
||||
def test_chat_empty_then_success_retries_and_recovers(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""First reply empty (the incident shape), second reply has content →
|
||||
exactly 2 attempts, ONE flat sleep of ``llm_retry_delay`` (default
|
||||
5.0), the trimmed second reply is returned, and ONE warning fired
|
||||
naming the model, the empty reply's ``finish_reason``, and the
|
||||
attempt count."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
caplog.set_level(logging.WARNING, logger="app.llm")
|
||||
llm, completions = _make_chat_client(
|
||||
completion_seq=[
|
||||
_FakeCompletion(None, finish_reason="length"),
|
||||
_FakeCompletion(" Recovered.\n"),
|
||||
]
|
||||
)
|
||||
out = asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert out == "Recovered."
|
||||
assert completions.create_calls == 2
|
||||
assert len(completions.chat_calls) == 2
|
||||
# Both attempts are byte-identical (same request).
|
||||
assert completions.chat_calls[0] == completions.chat_calls[1]
|
||||
assert sleeps == [5.0] # one flat BOR_LLM_RETRY_DELAY (default)
|
||||
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
|
||||
assert len(warnings) == 1
|
||||
line = warnings[0].getMessage()
|
||||
assert "lite" in line # the summary model (default)
|
||||
assert "finish_reason=length" in line # the incident signature
|
||||
assert "attempt 1/4" in line # failed attempt 1 of 1 + 3 retries
|
||||
|
||||
|
||||
def test_chat_explicit_model_named_in_the_retry_warning(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""The warning names the model actually requested (an explicit
|
||||
*model* overrides the default)."""
|
||||
_record_sleeps(monkeypatch)
|
||||
caplog.set_level(logging.WARNING, logger="app.llm")
|
||||
llm, _ = _make_chat_client(
|
||||
completion_seq=[_empty(), _FakeCompletion("ok")],
|
||||
llm_summary_model="tiny",
|
||||
)
|
||||
out = asyncio.run(llm.chat([{"role": "user", "content": "q"}], model="special"))
|
||||
assert out == "ok"
|
||||
line = [r.getMessage() for r in caplog.records if r.levelno == logging.WARNING][0]
|
||||
assert "model=special" in line
|
||||
assert "tiny" not in line
|
||||
|
||||
|
||||
def test_chat_all_empty_exhausts_after_1_plus_retries_attempts(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""Default ``llm_retries=3`` → exactly 4 attempts, 3 sleeps, then
|
||||
``LLMError`` naming the attempts. One empty reply omits
|
||||
``finish_reason`` (provider quirk) — the log line still formats
|
||||
(``finish_reason=None``) and never crashes the diagnostic path."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
caplog.set_level(logging.WARNING, logger="app.llm")
|
||||
llm, completions = _make_chat_client(
|
||||
completion_seq=[_empty(), _empty(), _empty(None), _empty()]
|
||||
)
|
||||
with pytest.raises(LLMError) as exc:
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert str(exc.value) == (
|
||||
f"chat completion from {_DEFAULT_BASE} returned empty content on all "
|
||||
"4 attempts — refusing to store a silent summary"
|
||||
)
|
||||
assert completions.create_calls == 4
|
||||
assert sleeps == [5.0, 5.0, 5.0] # no sleep after the last attempt
|
||||
lines = [r.getMessage() for r in caplog.records if r.levelno == logging.WARNING]
|
||||
assert "attempt 1/4" in lines[0]
|
||||
assert "attempt 2/4" in lines[1]
|
||||
assert "attempt 3/4" in lines[2]
|
||||
assert "finish_reason=None" in lines[2] # the omitted-finish_reason reply
|
||||
|
||||
|
||||
def test_chat_all_empty_custom_retry_count_names_the_attempts(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""``llm_retries=1`` → exactly 2 attempts, 1 sleep, the exhaustion
|
||||
message names 2."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, completions = _make_chat_client(
|
||||
completion_seq=[_empty(), _empty()], llm_retries=1, llm_retry_delay=0.5
|
||||
)
|
||||
with pytest.raises(LLMError, match="all 2 attempts"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert completions.create_calls == 2
|
||||
assert sleeps == [0.5]
|
||||
|
||||
|
||||
def test_chat_first_success_never_retries(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Happy path untouched: exactly 1 ``create()`` call, ZERO sleeps,
|
||||
the trimmed content is returned byte-identically."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, completions = _make_chat_client(_FakeCompletion(" Summary text.\n"))
|
||||
out = asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert out == "Summary text."
|
||||
assert completions.create_calls == 1
|
||||
assert sleeps == []
|
||||
|
||||
|
||||
def test_chat_no_choices_reply_is_not_retried(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""D2: a choiceless reply raises immediately — 1 attempt, no sleep,
|
||||
no retry (only empty content is the retryable class)."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, completions = _make_chat_client(
|
||||
_FakeCompletion(None, empty_choices=True), llm_retries=3
|
||||
)
|
||||
with pytest.raises(LLMError, match="no choices"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert completions.create_calls == 1
|
||||
assert sleeps == []
|
||||
|
||||
|
||||
def test_chat_transport_failure_is_not_retried(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""D2: a transport failure raises immediately — 1 attempt, no sleep,
|
||||
no app-level retry (the openai SDK's own ``max_retries=2`` covers
|
||||
wire-level failures)."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, completions = _make_chat_client(fail=RuntimeError("HTTP 502 Bad Gateway"))
|
||||
with pytest.raises(LLMError, match="HTTP 502"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert completions.create_calls == 1
|
||||
assert completions.chat_calls == [] # the SDK call itself raised
|
||||
assert sleeps == []
|
||||
|
||||
|
||||
def test_chat_zero_retries_raises_legacy_message_byte_identical(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The kill switch (``llm_retries=0``): one attempt, zero sleeps, the
|
||||
PRE-phase-96 message byte-identically (asserted as the exact
|
||||
string, not a pattern)."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, completions = _make_chat_client(_FakeCompletion(None), llm_retries=0)
|
||||
with pytest.raises(LLMError) as exc:
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert str(exc.value) == (
|
||||
f"chat completion from {_DEFAULT_BASE} returned empty content — "
|
||||
"refusing to store a silent summary"
|
||||
)
|
||||
assert completions.create_calls == 1
|
||||
assert sleeps == []
|
||||
|
||||
|
||||
def test_chat_retry_delay_is_flat_never_backoff(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The recorded sleeps are the flat ``llm_retry_delay`` each time —
|
||||
never a growing backoff (the phase-67 convention)."""
|
||||
sleeps = _record_sleeps(monkeypatch)
|
||||
llm, _ = _make_chat_client(
|
||||
completion_seq=[_empty(), _empty(), _empty(), _empty()],
|
||||
llm_retries=3,
|
||||
llm_retry_delay=1.25,
|
||||
)
|
||||
with pytest.raises(LLMError, match="all 4 attempts"):
|
||||
asyncio.run(llm.chat([{"role": "user", "content": "q"}]))
|
||||
assert sleeps == [1.25, 1.25, 1.25]
|
||||
|
||||
|
||||
# ---------- chat_stream_retried (phase 67, task 01) ----------
|
||||
|
||||
_RETRY_MSGS: list[dict[str, str]] = [{"role": "user", "content": "q"}]
|
||||
|
||||
@@ -785,11 +785,19 @@ def _patch_sync_seams(
|
||||
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
|
||||
|
||||
async def fake_folder_summaries(
|
||||
db: object, llm: object, *, skip: bool = False
|
||||
db: object,
|
||||
llm: object,
|
||||
*,
|
||||
skip: bool = False,
|
||||
only_missing: bool = False,
|
||||
) -> dict[str, int]:
|
||||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||||
|
||||
monkeypatch.setattr(sync_api, "generate_folder_summaries", fake_folder_summaries)
|
||||
# Phase 96 (task 03): the unchanged-walk gap probe is DB-free in
|
||||
# these state-machine tests — no gap, so the folder step stays
|
||||
# skipped exactly as before the gate change.
|
||||
monkeypatch.setattr(sync_api, "missing_folder_summaries", lambda session: [])
|
||||
|
||||
class _DummySession:
|
||||
def close(self) -> None:
|
||||
|
||||
Reference in New Issue
Block a user