phase: 98_sync_summary_visibility
Build and Push Containers / build-and-push-app (push) Successful in 1m51s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All verification complete. Final report:

**Phase 98 — Sync summary visibility: final verification pass** (all 5 tasks already complete; implementation verified against the design, no defects found, no code changes needed)

- **Implementation checked:** `SyncStatus` phase machine (4 new keys, terminal-keep counts), `on_progress` hook in `generate_folder_summaries`, `summary_pending` on `KbTreeSource`/`KbTreeFolder` + D3 rule in `build_kb_tree`, phase-aware sync labels + pending UI in `sources.js`, `.kb-summary-pending` CSS — all match decisions D1–D5.
- **Unit + integration:** `uv run pytest` → 2184 tests, 0 failed/errors (exit 0)
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **99%** on `app/` (criterion >90% ✓; `app/api/sync.py` and `app/rag/folder_summaries.py` at 100%)
- **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
- **Phase E2E (isolation):** `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` → **3 passed** (phase machine, live label, pending markers + gap-fill self-heal)
- **Regression suites (each isolated, `--no-cov`):** test_kb_tree ✓, test_ls_tree_drilldown 3 ✓, test_sync_button 3 ✓, test_sync_upload_progress 4 ✓, test_oneshot_llm_retry 2 ✓, test_local_directory_sources 3 ✓
- **Completion criteria:** all 7 verified green — status phase fields + terminal semantics; `Writing KB overview…`/`Summarizing folders… (n/m)` labels (title + aria-live); pending set == `missing_folder_summaries` (integration cross-check pinned at `test_docs_api.py:428`); CLI/`ls` byte-identity (no changes to those paths, pins green); suite/coverage/lint gates; dedicated + regression E2E. Commit left to the harness per protocol (no `git add`/`commit` run).
- **Decisions/deviations:** none — no fixes were required this pass.
- **Next pending phase:** `99_kb_tree_table_and_back_nav`.
This commit is contained in:
2026-09-13 00:23:05 -04:00
parent 909c96c7bc
commit f665a83b1a
39 changed files with 3265 additions and 112 deletions
+31 -3
View File
@@ -40,7 +40,7 @@ from app.core.auth import require_admin, require_user
from app.db import get_db
from app.models import Chunk, Document, FolderSummary
from app.rag.agent import list_source_names
from app.rag.folder_summaries import folder_of
from app.rag.folder_summaries import MIN_DOCS_PER_FOLDER, folder_of
from app.rag.importer import match_extension
from app.rag.llm import EmbeddingError, LLMClient
from app.schemas import (
@@ -376,6 +376,11 @@ def _level_children(
:func:`app.rag.agent.group_folder_listing` level-for-level; the
file list is NOT capped (the ``ls`` 50-line cap is a model-context
budget — the UI is for humans). Recurses one level per call.
Each folder node also carries the phase-98 D3 ``summary_pending``
flag: the recursive count ≥ :data:`MIN_DOCS_PER_FOLDER` AND no
stored ``folder_summaries`` row for ``(source, sub)`` — the same
rule the source node applies (see :func:`build_kb_tree`).
"""
children: list[KbTreeFolder | KbTreeFile] = []
for sub in sorted(g for g in folders if folder_of(g) == folder):
@@ -384,6 +389,8 @@ def _level_children(
path=sub,
documents=counts[sub],
summary=summaries.get((source, sub)),
summary_pending=counts[sub] >= MIN_DOCS_PER_FOLDER
and (source, sub) not in summaries,
children=_level_children(source, sub, folders, counts, rows, summaries),
)
)
@@ -431,7 +438,22 @@ def build_kb_tree(
shape.
* **File nodes** — direct files only, in input (catalog) order;
``path`` source-relative; ``title`` / ``chunks`` / ``indexed_at``
verbatim from the catalogue row.
verbatim from the catalogue row. File nodes carry NO pending
flag (the file table has no description column).
* **Pending** — ``summary_pending`` on the SOURCE and every FOLDER
node (phase 98, decision D3 — ONE concept): true iff the node's
recursive ``documents`` count ≥
:data:`app.rag.folder_summaries.MIN_DOCS_PER_FOLDER` (2) AND it
has NO stored ``folder_summaries`` row (AI or manual — any row;
the builder sees stored rows only). That is EXACTLY
:func:`app.rag.folder_summaries.missing_folder_summaries`'s
candidate set (phase 96's gap-fill regenerates precisely those
keys on the next sync — the marker is honest: "waiting to
generate", and the integration cross-check pins the tree's
pending set to that function so the marker can never drift from
the gap-fill). A < 2-document folder is NEVER pending (it never
gets a summary — its one file line IS its description), and a
registered 0-document source never is.
ONE concept end to end: the builder reuses
:func:`app.rag.folder_summaries.folder_of` and the phase-94
@@ -471,13 +493,19 @@ def _source_node(
count (every one of its documents, the set its stored
``(source, "")`` summary describes). A source with no rows lists
``documents: 0`` and no children (the registered 0-document source
— the phase-70/72 invariant, extended by the superset rule).
— the phase-70/72 invariant, extended by the superset rule) and
is never ``summary_pending`` (0 < the minimum).
``summary_pending`` (phase 98, D3): the whole-source count ≥
:data:`MIN_DOCS_PER_FOLDER` AND no stored ``(source, "")`` row —
the source-root arm of the rule :func:`build_kb_tree` documents.
"""
folders, counts = _folder_counts(rows)
return KbTreeSource(
name=source,
documents=len(rows),
summary=summaries.get((source, "")),
summary_pending=len(rows) >= MIN_DOCS_PER_FOLDER and (source, "") not in summaries,
children=_level_children(source, "", folders, counts, rows, summaries),
)