phase: 98_sync_summary_visibility

All verification complete. Final report: **Phase 98 — Sync summary visibility: final verification pass** (all 5 tasks already complete; implementation verified against the design, no defects found, no code changes needed) - **Implementation checked:** `SyncStatus` phase machine (4 new keys, terminal-keep counts), `on_progress` hook in `generate_folder_summaries`, `summary_pending` on `KbTreeSource`/`KbTreeFolder` + D3 rule in `build_kb_tree`, phase-aware sync labels + pending UI in `sources.js`, `.kb-summary-pending` CSS — all match decisions D1–D5. - **Unit + integration:** `uv run pytest` → 2184 tests, 0 failed/errors (exit 0) - **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **99%** on `app/` (criterion >90% ✓; `app/api/sync.py` and `app/rag/folder_summaries.py` at 100%) - **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings - **Phase E2E (isolation):** `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` → **3 passed** (phase machine, live label, pending markers + gap-fill self-heal) - **Regression suites (each isolated, `--no-cov`):** test_kb_tree ✓, test_ls_tree_drilldown 3 ✓, test_sync_button 3 ✓, test_sync_upload_progress 4 ✓, test_oneshot_llm_retry 2 ✓, test_local_directory_sources 3 ✓ - **Completion criteria:** all 7 verified green — status phase fields + terminal semantics; `Writing KB overview…`/`Summarizing folders… (n/m)` labels (title + aria-live); pending set == `missing_folder_summaries` (integration cross-check pinned at `test_docs_api.py:428`); CLI/`ls` byte-identity (no changes to those paths, pins green); suite/coverage/lint gates; dedicated + regression E2E. Commit left to the harness per protocol (no `git add`/`commit` run). - **Decisions/deviations:** none — no fixes were required this pass. - **Next pending phase:** `99_kb_tree_table_and_back_nav`.
2026-09-13 00:23:05 -04:00
parent 909c96c7bc
commit f665a83b1a
39 changed files with 3265 additions and 112 deletions
@@ -129,7 +129,11 @@
 *     summary null) → the text goes away (level block hidden / row
 *     cell emptied) + "Description cleared."; the in-memory kbTree
 *     node's summary is updated IN PLACE (no re-fetch — the tree state
- *     stays coherent; the re-fetch is the safety net). Failure
+ *     stays coherent; the re-fetch is the safety net) — and, since
+ *     Phase 98 (task 04), its summary_pending flag is cleared IN
+ *     PLACE on the same success path (a created/updated description
+ *     is no longer pending — the D4 marker clears where the edit
+ *     happened, no re-fetch). Failure
 *     (non-2xx / network) → neutral retry copy (the phase-55
 *     convention), the editor stays open with the user's text, the
 *     stored text untouched. Cancel → restore the text node.
@@ -141,6 +145,64 @@
 *   • every editor part is static createElement; the description text
 *     is a text node (textContent / .value) — this module never builds
 *     HTML from document-derived data (the house rule, unchanged).
+ *
+ * Phase 98 (task 02) — the sync button's phase-aware labels: the
+ * post-import span of a sync (the KB overview + the long
+ * folder-summary span) no longer reads as a stuck file count. The
+ * SYNC job's running label is now phase-aware (fmtSyncPhaseLabel —
+ * the section header below carries the decision tree): "overview" →
+ * "Writing KB overview…", "summaries" → "Summarizing folders…
+ * <source/folder> (n/m)" (the folder part omitted while
+ * current_summary is null — the phase's first poll), anything else
+ * (the null prelude, "import") → today's "Syncing… <file> (n/m)"
+ * byte-identical. The UPLOAD job (its status has no phase) keeps the
+ * bare label. The untruncated label rides the button title (now set
+ * for EVERY running label, not just file labels) + #sync-result as
+ * before (A4).
+ *
+ * Phase 98 (task 04) — the "Summary pending" markers: the catalog
+ * tells the owner a summary is WAITING instead of showing an empty
+ * cell that reads as "missed". A source/folder node is
+ * summary_pending when its summary is due (recursive count >= 2 —
+ * the D3 rule, exactly the next sync's gap-fill set) but no stored
+ * description exists yet. The marker has TWO surfaces, both driven
+ * by the same node flag:
+ *
+ *   • the row's Description cell (makeDescCell) — when the stored
+ *     summary is empty AND node.summary_pending, the text span
+ *     carries the class kb-summary-pending (the muted ink-soft pair,
+ *     AA on --surface — text + color, never color alone; the row
+ *     cell's font-size/line-height apply — no row-height change) +
+ *     the text "Summary pending" + the D4 title, verbatim:
+ *     "No stored description yet — the next sync will generate one."
+ *     A stored summary
+ *     shows the stored text (NEVER the marker); neither stored nor
+ *     pending shows the empty cell (the ls rule, unchanged). The
+ *     Edit button is UNCHANGED — always present (a manual save
+ *     creates the row).
+ *   • the level block (renderLevel) — #kb-level shows when the
+ *     current level has a stored description (as today) OR is
+ *     pending: title as today (the full source-relative path), and
+ *     #kb-level-summary shows the stored text, or — when pending —
+ *     the D4 pending note, verbatim:
+ *     "No description stored yet — the next sync will generate one. (You can write one yourself.)"
+ *     Neither stored nor pending stays hidden (the ls rule,
+ *     unchanged). The block's
+ *     Edit button (write one manually now) already ships in the
+ *     static markup (phase 97, task 05).
+ *   • the in-place clear: wireDescriptionEdit's success path sets
+ *     node.summary_pending = false right after
+ *     node.summary = data.summary — a created/updated description is
+ *     no longer pending; the marker clears IN PLACE in the surface
+ *     where the edit happened, no re-fetch (the re-fetch stays the
+ *     safety net). closeEditor re-renders the display state from the
+ *     node with the SAME three-state rule the surfaces use: the
+ *     pending marker's muted class + tooltip CANNOT survive a save
+ *     (a stale "next sync" tooltip under a just-created description),
+ *     and a cancel restores the surface's pending display (each
+ *     surface passes its pending copy via pendingText — the row's
+ *     "Summary pending" marker, the level's D4 note — and its
+ *     tooltip via pendingTitle, the row only).
 */

 import { fetchIsAdmin } from "./header.js";
@@ -187,7 +249,15 @@ function mkBtn(cls, label) {
 * itself (the ls rule); a row cell just goes empty (the always-present
 * button stays). Returns a handle whose reset() tears down an OPEN
 * editor before a re-render (PLAN §7.4 — never a stale open editor). */
-function wireDescriptionEdit({ editBtn, textEl, container, getTarget, onCleared }) {
+function wireDescriptionEdit({
+    editBtn,
+    textEl,
+    container,
+    getTarget,
+    onCleared,
+    pendingText,
+    pendingTitle,
+  }) {
  const actions = document.createElement("div");
  actions.className = "kb-summary-actions";
  const saveBtn = mkBtn("kb-summary-save", "Save");
@@ -200,15 +270,32 @@ function wireDescriptionEdit({ editBtn, textEl, container, getTarget, onCleared
  let editor = null;
  let isOpen = false;

-  /* Back to the display state: the text element re-rendered from the
-   * node (the CURRENT stored description, textContent only), the Edit
-   * button available again. A cleared node empties the text and, for
-   * the level block, hides the whole block via onCleared. */
+  /* Back to the display state: the display state re-rendered from the
+   * node with the SAME three-state rule the surfaces use (makeDescCell
+   * / renderLevel — textContent/class/title only, the house rule):
+   * a stored summary → the stored text with NO marker (the in-place
+   * clear — the success path cleared node.summary_pending, so the
+   * muted pending style + the stale "next sync" tooltip cannot survive
+   * a saved description); no stored text but PENDING → the surface's
+   * pending display (pendingText — the row's "Summary pending" marker
+   * + its D4 tooltip, the level's D4 note — a cancel restores exactly
+   * the pre-edit state, including the marker); neither → the empty
+   * text. The Edit button is available again. A cleared node empties
+   * the text and, for the level block, hides the whole block via
+   * onCleared. */
  function closeEditor(message) {
    const target = getTarget();
    const node = target ? target.node : null;
-    const value = node && typeof node.summary === "string" ? node.summary : "";
-    textEl.textContent = value; // text node — the CURRENT stored description
+    const stored = node && typeof node.summary === "string" ? node.summary : "";
+    const pending = node !== null && stored === "" && node.summary_pending;
+    const value = pending && pendingText ? pendingText : stored;
+    textEl.className = pending ? "kb-summary-pending" : "";
+    if (pending) {
+      if (pendingTitle) textEl.title = pendingTitle;
+    } else {
+      textEl.removeAttribute("title");
+    }
+    textEl.textContent = value; // text node — the CURRENT display state
    editBtn.hidden = false;
    status.textContent = message;
    container.replaceChildren(textEl, editBtn, status);
@@ -252,6 +339,7 @@ function wireDescriptionEdit({ editBtn, textEl, container, getTarget, onCleared
      }
      const data = await res.json();
      node.summary = data.summary; // in-place kbTree update (no re-fetch)
+      node.summary_pending = false; // phase 98 (D4): a created/updated description is no longer pending — the marker clears in place (no re-fetch)
      closeEditor(data.summary === null ? "Description cleared." : "Description updated.");
    } catch {
      // Network failure: the reachable? copy; the editor stays open.
@@ -292,11 +380,27 @@ function wireDescriptionEdit({ editBtn, textEl, container, getTarget, onCleared
 * miss), so the button is added unconditionally (the view is
 * admin-only already — the endpoint's require_admin is the API gate).
 * The shared editor is wired with a CONSTANT target (the row's node).
- * `label` is the human name for the button's aria-label. */
+ * `label` is the human name for the button's aria-label.
+ * Phase 98 (task 04, D4): the cell has THREE states — a stored summary
+ * (the stored text, textContent only), PENDING (no stored summary but
+ * node.summary_pending — the marker: the kb-summary-pending class +
+ * the "Summary pending" text + the D4 title; text + color, never color
+ * alone), and the empty cell (neither — the ls rule, unchanged). The
+ * Edit button is UNCHANGED in all three (a manual save creates the row
+ * and clears the marker in place — the editor's success path). */
 function makeDescCell(node, source, folder, label) {
  const td = document.createElement("td");
  const text = document.createElement("span");
-  text.textContent = (node && node.summary) || ""; // text node — never innerHTML
+  if (node && node.summary) {
+    text.textContent = node.summary; // stored description — text node, never innerHTML
+  } else if (node && node.summary_pending) {
+    // Phase 98 (D4): the summary is due but not stored yet — the marker.
+    text.className = "kb-summary-pending";
+    text.textContent = "Summary pending";
+    text.title = "No stored description yet — the next sync will generate one.";
+  } else {
+    text.textContent = ""; // neither stored nor pending — the empty cell (the ls rule)
+  }
  const btn = document.createElement("button");
  btn.type = "button";
  btn.className = "kb-summary-edit";
@@ -308,6 +412,10 @@ function makeDescCell(node, source, folder, label) {
    textEl: text,
    container: td,
    getTarget: () => ({ node, source, folder }),
+    // Phase 98 (D4): the row's pending display (closeEditor restores
+    // it on a cancel; a save clears the marker in place).
+    pendingText: "Summary pending",
+    pendingTitle: "No stored description yet — the next sync will generate one.",
  });
  return td;
 }
@@ -328,8 +436,16 @@ export async function mount(root) {
   * status endpoints — GET /api/sync/status +
   * GET /api/git-sources/upload/status — and applies this decision
   * tree, in order (startSyncPolling):
-   *   1. sync running   → "Syncing… <file> (n/m)" — bare "Syncing…" until
-   *                       the import's first file (clone/pull, A4);
+   *   1. sync running   → the phase-aware label (Phase 98 task 02, D2)
+   *                       — the null-phase prelude + "import" →
+   *                       "Syncing… <file> (n/m)" — bare "Syncing…"
+   *                       until the import's first file (clone/pull,
+   *                       A4, byte-identical); "overview" → "Writing KB
+   *                       overview…"; "summaries" → "Summarizing
+   *                       folders… <source/folder> (n/m)" (the folder
+   *                       part omitted while current_summary is null —
+   *                       the phase's first poll) — the post-import span
+   *                       where the file count sits still;
   *   2. upload running → BARE "Importing…" — the background upload
   *                       RUN (phase 90: unpack + register only, no
   *                       scan — its status never carries a file or
@@ -346,12 +462,16 @@ export async function mount(root) {
   *                       Sources page's error banner, never this page's (A3);
   *   7. both idle      → retry-ready idle.
   * The live label is the status endpoint's full source/relative/path
-   * (A4): CSS ellipsizes #sync-label; the full untruncated path also
-   * rides the button title (hover) and #sync-result (the aria-live
+   * (A4) — or, in the post-import phases, the phase's own copy (Phase
+   * 98 task 02): CSS ellipsizes #sync-label; the full untruncated
+   * label also rides the button title (hover — set for EVERY running
+   * label, not just file labels) and #sync-result (the aria-live
   * announcer — screen readers hear it). The load-time re-attach
-   * (initSyncButton) re-enters a RUNNING upload the same way; a terminal
-   * upload is a no-op there (the boot-time loadTree() already shows the
-   * current catalog).
+   * (initSyncButton) re-enters a RUNNING sync the same way — with
+   * whatever phase the status reports (a mid-summaries reload shows
+   * the summaries label — the never-stale contract) — and a RUNNING
+   * upload the same way; a terminal upload is a no-op there (the
+   * boot-time loadTree() already shows the current catalog).
   *
   * Elements: #sync-btn (the button), #sync-label (the text),
   * #sync-icon (the spinner icon), #sync-result (aria-live result
@@ -397,7 +517,11 @@ export async function mount(root) {
   * "Importing" label is always the bare one. The counts appear only
   * once the import has started (total > 0). CSS ellipsizes the button
   * label; the same untruncated text goes to the button title +
-   * #sync-result (the aria-live announcer). */
+   * #sync-result (the aria-live announcer). Phase 98 (task 02): the
+   * SYNC job's post-import phases (overview / summaries) take their
+   * own label via fmtSyncPhaseLabel (below) — this builder keeps that
+   * prelude/import fall-through byte-identical and serves the UPLOAD
+   * job unchanged (its status has no phase — D2). */
  function fmtSyncLabel(kind, currentFile, done, total) {
    const prefix = kind === "upload" ? "Importing" : "Syncing…";
    let label = currentFile ? `${prefix} ${currentFile}` : prefix;
@@ -405,6 +529,29 @@ export async function mount(root) {
    return label;
  }

+  /* Phase 98 (task 02, D2): the SYNC job's phase-aware running label
+   * — the post-import span where the file count sits still (the
+   * owner's "the number pauses for a really long time"): "overview"
+   * → "Writing KB overview…"; "summaries" → "Summarizing folders…
+   * <source/folder> (n/m)" — the folder part is omitted while
+   * current_summary is null (the phase's first poll); anything else
+   * (the null prelude, "import") → the phase-64 fmtSyncLabel
+   * fall-through, byte-identical (same builder, same fields, same
+   * order). The UPLOAD job never passes a status (its endpoint has no
+   * phase — D2) and keeps the bare fmtSyncLabel label. The untruncated
+   * result rides the button title + #sync-result like every running
+   * label (A4). */
+  function fmtSyncPhaseLabel(status) {
+    if (status.phase === "overview") return "Writing KB overview…";
+    if (status.phase === "summaries") {
+      const folder = status.current_summary ? ` ${status.current_summary}` : "";
+      return `Summarizing folders…${folder} (${status.summaries_done}/${status.summaries_total})`;
+    }
+    // null phase (the prelude) or "import" — today's byte-identical
+    // sync label (the phase-64 contract, unchanged).
+    return fmtSyncLabel("sync", status.current_file, status.files_done, status.files_total);
+  }
+
  function fmtSyncResult(detail) {
    const d = detail || {};
    const added = d.added || 0;
@@ -425,21 +572,29 @@ export async function mount(root) {

  /* The single running-state entry point (phase 64 task 04: the label
   * carries the live file — `kind` "sync" | "upload", the status
-   * endpoint's current_file + done/total). Same mechanics as before
-   * (disabled, aria-busy, spinning icon, no is-error) plus: the full
-   * untruncated path on the button title (removed when null — no file
-   * yet) and in #sync-result (the aria-live announcer reads the full
-   * live path; CSS ellipsizes the button's label span only). */
-  function enterSyncRunningState(kind, currentFile, done, total) {
+   * endpoint's current_file + done/total). Phase 98 (task 02, D2): the
+   * SYNC job passes the status object as `status` — its running label
+   * is phase-aware (fmtSyncPhaseLabel: "Writing KB overview…" /
+   * "Summarizing folders… <folder> (n/m)"); the UPLOAD job and the
+   * 202/409 click (no status yet) keep the bare fmtSyncLabel label.
+   * Same mechanics as before (disabled, aria-busy, spinning icon, no
+   * is-error) plus: the FULL untruncated label on the button title
+   * (set for EVERY running label — the phase-64 file-only rule is
+   * adjusted for the phase labels) and in #sync-result (the aria-live
+   * announcer reads it; CSS ellipsizes the button's label span only).
+   */
+  function enterSyncRunningState(kind, currentFile, done, total, status) {
    if (!syncBtn) return;
    syncBtn.disabled = true;
    syncBtn.setAttribute("aria-busy", "true");
-    if (currentFile) syncBtn.title = currentFile;
-    else syncBtn.removeAttribute("title");
+    const label =
+      kind === "sync" && status
+        ? fmtSyncPhaseLabel(status)
+        : fmtSyncLabel(kind, currentFile, done, total);
+    syncBtn.title = label; // A4: the untruncated label rides the title, always
    syncBtn.setAttribute("aria-label", "Sync sources");
    syncBtn.classList.remove("is-error");
    if (syncIcon) syncIcon.classList.add("is-spinning");
-    const label = fmtSyncLabel(kind, currentFile, done, total);
    if (syncLabel) syncLabel.textContent = label;
    if (syncResult) syncResult.textContent = label;
  }
@@ -572,11 +727,14 @@ export async function mount(root) {
        syncPollTimer = setTimeout(tick, SYNC_POLL_MS);
        return;
      }
-      // 1. sync running: the live sync file (bare "Syncing…" until the
-      //    import's first file — A4).
+      // 1. sync running: the phase-aware label (phase 98 task 02 — the
+      //    null prelude + "import" keep the byte-identical file label,
+      //    "overview" / "summaries" name themselves; the WHOLE status
+      //    goes in for the phase fields).
      if (syncStatus.state === "running") {
        enterSyncRunningState(
-          "sync", syncStatus.current_file, syncStatus.files_done, syncStatus.files_total
+          "sync", syncStatus.current_file, syncStatus.files_done, syncStatus.files_total,
+          syncStatus
        );
        syncPollTimer = setTimeout(tick, SYNC_POLL_MS);
        return;
@@ -668,7 +826,11 @@ export async function mount(root) {
   * + register — the bare "Importing…" label) adopts the button the
   * same way — the "user clicked upload, then opened sources" case; a
   * terminal upload is a no-op (the boot-time loadTree() already shows
-   * the current catalog). */
+   * the current catalog). Phase 98 (task 02): a RUNNING sync re-enters
+   * with the WHOLE status — the label is phase-aware, so a mid-
+   * summaries reload re-enters with the summaries label (the
+   * never-stale contract); a RUNNING upload re-attach stays on the
+   * bare label (its status has no phase — D2). */
  async function initSyncButton() {
    if (!syncBtn) return;
    if (!(await fetchIsAdmin())) return;
@@ -681,7 +843,8 @@ export async function mount(root) {
    } catch { return; }
    if (status.state === "running") {
      enterSyncRunningState(
-        "sync", status.current_file, status.files_done, status.files_total
+        "sync", status.current_file, status.files_done, status.files_total,
+        status
      );
      emitSyncStatus(status);
      startSyncPolling();
@@ -1040,14 +1203,24 @@ export async function mount(root) {
    }

    // The level block: the current level's STORED description (source:
-    // the (source, "") row; folder: its row) — hidden when none is
-    // stored (the ls rule: count only, no placeholder). Title = the
-    // full source-relative path (e.g. `alpha/two`).
-    if (node.summary) {
+    // the (source, "") row; folder: its row) — OR, since Phase 98
+    // (task 04, D4), the PENDING note when the level is summary_pending
+    // (the summary is due but not stored yet — the next sync's gap-fill
+    // will generate it, or the owner can write one now via the block's
+    // Edit button). Hidden only when NEITHER is stored nor pending
+    // (the ls rule: count only, no placeholder). Title = the full
+    // source-relative path (e.g. `alpha/two`).
+    if (node.summary || node.summary_pending) {
      levelTitleEl.textContent = current.folder
        ? current.source + "/" + current.folder
        : current.source;
-      levelSummaryEl.textContent = node.summary;
+      levelSummaryEl.textContent = node.summary ||
+        "No description stored yet — the next sync will generate one. (You can write one yourself.)";
+      // Phase 98 (D4): the pending text is the muted marker style —
+      // and the block is REUSED across levels, so a stored level must
+      // clear a previous pending level's class (the same state
+      // closeEditor manages on the editor's close).
+      levelSummaryEl.className = node.summary ? "" : "kb-summary-pending";
      levelEl.hidden = false;
    } else {
      levelEl.hidden = true;
@@ -1144,6 +1317,10 @@ export async function mount(root) {
        const node = currentLevelNode();
        return node ? { node, source: current.source, folder: current.folder } : null;
      },
+      // Phase 98 (D4): the level's pending display (the D4 note — no
+      // tooltip on the level's <p>: D4's title is the row cell's).
+      pendingText:
+        "No description stored yet — the next sync will generate one. (You can write one yourself.)",
      onCleared: () => {
        // The phase-57 announcement beat: keep the block visible a short
        // beat so the role=status "Description cleared." is still