phase: 98_sync_summary_visibility
Build and Push Containers / build-and-push-app (push) Successful in 1m51s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All verification complete. Final report:

**Phase 98 — Sync summary visibility: final verification pass** (all 5 tasks already complete; implementation verified against the design, no defects found, no code changes needed)

- **Implementation checked:** `SyncStatus` phase machine (4 new keys, terminal-keep counts), `on_progress` hook in `generate_folder_summaries`, `summary_pending` on `KbTreeSource`/`KbTreeFolder` + D3 rule in `build_kb_tree`, phase-aware sync labels + pending UI in `sources.js`, `.kb-summary-pending` CSS — all match decisions D1–D5.
- **Unit + integration:** `uv run pytest` → 2184 tests, 0 failed/errors (exit 0)
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **99%** on `app/` (criterion >90% ✓; `app/api/sync.py` and `app/rag/folder_summaries.py` at 100%)
- **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings
- **Phase E2E (isolation):** `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` → **3 passed** (phase machine, live label, pending markers + gap-fill self-heal)
- **Regression suites (each isolated, `--no-cov`):** test_kb_tree ✓, test_ls_tree_drilldown 3 ✓, test_sync_button 3 ✓, test_sync_upload_progress 4 ✓, test_oneshot_llm_retry 2 ✓, test_local_directory_sources 3 ✓
- **Completion criteria:** all 7 verified green — status phase fields + terminal semantics; `Writing KB overview…`/`Summarizing folders… (n/m)` labels (title + aria-live); pending set == `missing_folder_summaries` (integration cross-check pinned at `test_docs_api.py:428`); CLI/`ls` byte-identity (no changes to those paths, pins green); suite/coverage/lint gates; dedicated + regression E2E. Commit left to the harness per protocol (no `git add`/`commit` run).
- **Decisions/deviations:** none — no fixes were required this pass.
- **Next pending phase:** `99_kb_tree_table_and_back_nav`.
This commit is contained in:
2026-09-13 00:23:05 -04:00
parent 909c96c7bc
commit f665a83b1a
39 changed files with 3265 additions and 112 deletions
+92 -5
View File
@@ -75,6 +75,23 @@ carries the phase-64 per-file progress — ``current_file`` (the
(clone/pull reports no file yet) and in terminal states, which clear
``current_file`` but keep the run's final counts.
Phase 98 (task 01) adds the run's PHASE machine — ``phase`` is the
pipeline stage the run is in: the model-check + clone/pull prelude
reports ``null`` (the bare "Syncing…" label stays), then ``"import"``
(set immediately before ``import_sources``), ``"overview"`` (set
before ``regenerate_overview`` — a changed KB only), and
``"summaries"`` (set before a folder-summary generation — the
changed-KB full regeneration or the unchanged-walk gap-fill; the
no-gap skip sets NO phase and the run stays ``"import"``). While in
``"summaries"`` the generator's ``on_progress`` hook fills
``current_summary`` (the ``source`` / ``source/folder_path`` being
summarized — the bare source name for the source-root row) plus
``summaries_done`` / ``summaries_total`` — the per-folder position
through the long summary span where the file count sits still.
Terminal states (success AND failed) clear ``phase`` +
``current_summary`` but keep the run's final ``summaries_done`` /
``summaries_total`` (the phase-64 keep-final-counts convention).
The ``failed`` state's ``error`` string is masked by the shared
sanitizer — the ``user:pass@`` masker now lives in :mod:`app.core.errors`
(imported here under the private name ``_sanitize_error``).
@@ -127,6 +144,16 @@ class SyncStatus:
after); ``files_done`` / ``files_total`` carry the hook's
done/total position and survive a terminal state (the run's last
position is useful context next to the error).
Phase 98 (task 01) phase fields: ``phase`` is the pipeline stage
(null in the model-check + clone/pull prelude and in terminal
states — the module docstring's phase machine); while in the
``"summaries"`` phase, ``current_summary`` is the source /
``source/folder_path`` the generator is summarizing right now and
``summaries_done`` / ``summaries_total`` carry the generator's
progress hook's position. Terminal states clear ``phase`` +
``current_summary`` but keep the run's final summary counts (the
phase-64 keep-final-counts convention).
"""
state: Literal["idle", "running", "success", "failed"] = "idle"
@@ -139,6 +166,13 @@ class SyncStatus:
current_file: str | None = None
files_done: int = 0
files_total: int = 0
# Phase 98 (task 01): the phase machine — the pipeline stage the
# run is in and, while in the folder-summary phase, the folder
# being summarized plus the hook's done/total position.
phase: Literal["import", "overview", "summaries"] | None = None
current_summary: str | None = None
summaries_done: int = 0
summaries_total: int = 0
_status = SyncStatus()
@@ -153,7 +187,11 @@ def sync_status() -> dict[str, Any]:
``current_file`` (phase 64) is the ``source/relative/path`` the
import is processing right now — null during the clone/pull phase
and in terminal states; ``files_done`` / ``files_total`` carry the
hook's position (0/0 idle).
hook's position (0/0 idle). ``phase`` (phase 98) is the pipeline
stage — null in the prelude and terminal states; while in
``"summaries"``, ``current_summary`` + ``summaries_done`` /
``summaries_total`` carry the generator's position (the terminal
keeps the final summary counts).
"""
return {
"state": _status.state,
@@ -164,6 +202,13 @@ def sync_status() -> dict[str, Any]:
"current_file": _status.current_file,
"files_done": _status.files_done,
"files_total": _status.files_total,
# Phase 98 (task 01): the phase machine — null/0/0 idle (the
# dataclass defaults) and in terminal states (which clear
# phase + current_summary but keep the final summary counts).
"phase": _status.phase,
"current_summary": _status.current_summary,
"summaries_done": _status.summaries_done,
"summaries_total": _status.summaries_total,
}
@@ -202,6 +247,13 @@ async def _run_sync() -> None:
_status.current_file = None
_status.files_done = 0
_status.files_total = 0
# Phase 98 (task 01): the phase fields reset with the run — no
# phase until the import starts (the model-check + clone/pull
# prelude reports null), no folder until the summary span starts.
_status.phase = None
_status.current_summary = None
_status.summaries_done = 0
_status.summaries_total = 0
try:
settings = get_settings()
# Step 1 (phase 41): fail fast — verify both models the sync
@@ -270,12 +322,17 @@ async def _run_sync() -> None:
_status.files_done = done
_status.files_total = total
# Phase 98 (task 01): the phase machine — the import phase
# starts NOW (the model-check + clone/pull prelude above
# reported ``phase: null``).
_status.phase = "import"
summary: ImportSummary = await import_sources(
sources, llm, prune=True, progress=_hook, ignore_by_root=ignore_by_root,
include_hidden_by_root=include_hidden_by_root,
)
overview = False
if summary.added + summary.updated > 0:
_status.phase = "overview" # phase 98 (task 01)
overview = await regenerate_overview(llm)
# Phase 94 (task 02), phase 96 (task 03): the folder
# summaries — the drill-down ls's per-level descriptions. A
@@ -290,19 +347,42 @@ async def _run_sync() -> None:
# outage never flips the run to failed) and only flushes: this
# run's own short-lived session commits (the phase-53
# convention), so the step-6 bump stays change-gated on the KB,
# not on the summaries. No status-surface change: the stats
# are log-only (the detail shape is untouched).
# not on the summaries. The status surface is the PHASE
# machine (phase 98, task 01): the stats dict itself stays
# log-only (the detail shape is untouched).
# Phase 98 (task 01): the folder-summary progress hook — the
# closure-captures-``_status`` convention the file's ``_hook``
# above already uses; the generator's ``total`` (the loop-start
# candidate count) and the bare source name for the root row
# come straight from the hook's arguments (D5's shape).
def _summary_hook(done: int, total: int, source: str, folder_path: str) -> None:
_status.current_summary = (
source if folder_path == "" else f"{source}/{folder_path}"
)
_status.summaries_done = done
_status.summaries_total = total
fs_db = SessionLocal()
try:
if summary.added + summary.updated > 0:
folder_stats = await generate_folder_summaries(fs_db, llm)
# Changed KB: full regeneration — the summaries phase
# starts now, and the hook reports the per-folder
# position through the (long) span.
_status.phase = "summaries"
folder_stats = await generate_folder_summaries(
fs_db, llm, on_progress=_summary_hook
)
fs_db.commit()
logger.info("sync: folder_summaries stats=%s", folder_stats)
else:
missing = missing_folder_summaries(fs_db)
if missing:
# Unchanged-walk gap-fill: the SAME summaries
# phase + hook (the fill's total is the missing
# count — the hook's loop-start ``total``).
_status.phase = "summaries"
folder_stats = await generate_folder_summaries(
fs_db, llm, only_missing=True
fs_db, llm, only_missing=True, on_progress=_summary_hook
)
fs_db.commit()
logger.info(
@@ -310,6 +390,9 @@ async def _run_sync() -> None:
len(missing), folder_stats,
)
else:
# The no-gap skip sets NO phase — the run stays
# "import" through to the terminal (the phase-64
# bare-label pin holds for this shape).
logger.info("sync: folder_summaries skipped (KB unchanged)")
finally:
fs_db.close()
@@ -336,6 +419,8 @@ async def _run_sync() -> None:
_status.state = "success"
_status.finished_at = datetime.now(UTC)
_status.current_file = None # phase 64: keep the final counts
_status.phase = None # phase 98: keep the final summary counts
_status.current_summary = None
_status.detail = {
"files": summary.files,
"added": summary.added,
@@ -356,3 +441,5 @@ async def _run_sync() -> None:
_status.finished_at = datetime.now(UTC)
_status.error = _sanitize_error(str(e))
_status.current_file = None # phase 64: keep the final counts
_status.phase = None # phase 98: keep the final summary counts
_status.current_summary = None