phase: 94_ls_tree_drilldown

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`

scripts/import_docs.py

@@ -54,6 +54,20 @@ reports ``overview=failed`` on the summary line: the import's exit code
 is about files, and the previous outline stays (an old outline is better
 than none).
 
+The stored folder summaries (phase 94 — the drill-down ``ls``'s
+per-level descriptions, the ``folder_summaries`` table) regenerate in
+the same run under the same gate: a changed KB (added + updated > 0),
+or an empty table after an unchanged walk (the first full run after
+migration 0017, or after a ``--limit`` first walk that skipped them).
+Same contract — **best-effort, per-folder fail-soft**: a ``lite``
+failure keeps the failed folders' previous rows and only counts into
+the stats; ``--limit`` debug runs skip them entirely (never burning a
+``lite`` call). The stats land on the summary line as
+``folder_summaries=<generated>/<failed>/<pruned>`` (or
+``folder_summaries=skipped`` when the gate did not fire), and the rows
+commit in the run's own short-lived session (the phase-53 convention —
+a failed commit rolls them back with it).
+
 A run that **changed** the knowledge base (added + updated + pruned > 0
 — phase 53, task 02) also advances the single-row ``sources_meta``
 version exactly once (``sources_version=<n>`` on the summary line): the
@@ -78,6 +92,7 @@ from app.core.debugging import configure_debugging
 from app.core.logging import configure_logging
 from app.db import SessionLocal
 from app.models import KbOverview
+from app.rag.folder_summaries import folder_summary_table_empty, generate_folder_summaries
 from app.rag.git_sources import effective_sources
 from app.rag.importer import ImportSummary, import_sources
 from app.rag.llm import LLMClient
@@ -216,6 +231,20 @@ def _overview_row_exists() -> bool:
         return session.get(KbOverview, 1) is not None
 
 
+def _folder_summaries_table_empty() -> bool:
+    """Whether the ``folder_summaries`` table holds any row (phase 94).
+
+    The ``_overview_row_exists`` pattern extended to a table-emptiness
+    check (one bounded ``LIMIT 1`` probe): an empty table after an
+    unchanged re-import — e.g. the first full run after migration 0017,
+    or after a ``--limit`` first walk that skipped generation — still
+    gets a fresh batch of folder summaries, while a populated table is
+    left untouched until the KB actually changes.
+    """
+    with SessionLocal() as session:
+        return folder_summary_table_empty(session)
+
+
 def main(argv: list[str] | None = None) -> int:
     args = build_parser().parse_args(argv)
     settings = get_settings()
@@ -246,9 +275,10 @@ def main(argv: list[str] | None = None) -> int:
 
     llm = LLMClient()
 
-    async def _run() -> tuple[ImportSummary, str, str]:
-        """Import, then (change-gated) advance the sources version and
-        refresh the stored KB overview.
+    async def _run() -> tuple[ImportSummary, str, str, dict[str, int] | None]:
+        """Import, then (change-gated) advance the sources version,
+        refresh the stored KB overview, and regenerate the stored folder
+        summaries.
 
         One event loop, one ``LLMClient`` (phase 31, task 04): the
         outline that every chat prompt injects as ``<knowledge_base>`` is
@@ -268,6 +298,17 @@ def main(argv: list[str] | None = None) -> int:
         deliberately broader than the overview's). ``--limit`` debug
         runs and unchanged re-runs never bump. The returned token is
         the new version, or ``"skipped"``.
+
+        The folder summaries (phase 94, task 02) follow the same gate
+        — a changed KB, or an empty ``folder_summaries`` table after an
+        unchanged walk (the first full run after migration 0017, or
+        after a ``--limit`` first walk) — with per-folder fail-soft
+        inside the generator (a ``lite`` failure keeps the failed
+        folders' previous rows). The generator only flushes: this
+        run's own short-lived session commits (the phase-53
+        convention), and the stats land on the summary line as
+        ``folder_summaries=<generated>/<failed>/<pruned>``
+        (``None`` — rendered ``skipped`` — when the gate did not fire).
         """
         summary = await import_sources(
             sources, llm, prune=args.prune, limit=args.limit,
@@ -294,32 +335,69 @@ def main(argv: list[str] | None = None) -> int:
             sources_version = "skipped"
             logger.info("sources: version bump skipped (KB unchanged)")
         if args.limit is not None:
+            # An incomplete walk must never rewrite the outline or the
+            # folder summaries (the --limit skip, mirrored above for the
+            # sources version).
             logger.info("overview: skipped (--limit)")
-            return summary, "skipped", sources_version
-        if summary.added + summary.updated == 0:
+            return summary, "skipped", sources_version, None
+        changed = summary.added + summary.updated > 0
+        overview_due = changed
+        folders_due = changed
+        if not changed:
             if summary.files == 0:
                 logger.info("overview: skipped (nothing imported)")
-                return summary, "skipped", sources_version
-            if _overview_row_exists():
+                return summary, "skipped", sources_version, None
+            # The unchanged-walk first-run triggers: the overview when
+            # no row exists yet (the first run after migration 0005),
+            # the folder summaries when the table is empty (the first
+            # full run after migration 0017, or after a --limit first
+            # walk that skipped them).
+            overview_due = not _overview_row_exists()
+            folders_due = _folder_summaries_table_empty()
+            if not overview_due and not folders_due:
                 logger.info("overview: skipped (KB unchanged)")
-                return summary, "skipped", sources_version
-            # No outline yet after an unchanged re-import (e.g. the first
-            # run after migration 0005) — fall through and generate one.
-        ok = await regenerate_overview(llm)
-        return summary, "updated" if ok else "failed", sources_version
+                return summary, "skipped", sources_version, None
+        overview_status = "skipped"
+        if overview_due:
+            ok = await regenerate_overview(llm)
+            overview_status = "updated" if ok else "failed"
+        else:
+            logger.info("overview: skipped (KB unchanged)")
+        # Phase 94 (task 02): the folder summaries — same event loop +
+        # LLMClient (phase 31 convention), per-folder fail-soft inside
+        # the generator (a ``lite`` failure never flips the exit code).
+        # It only flushes — this run's own short-lived session commits
+        # (the phase-53 convention), so a failed commit rolls the
+        # summaries back with it.
+        folder_stats: dict[str, int] | None = None
+        if folders_due:
+            session = SessionLocal()
+            try:
+                folder_stats = await generate_folder_summaries(session, llm)
+                session.commit()
+            finally:
+                session.close()
+        return summary, overview_status, sources_version, folder_stats
 
-    summary, overview_status, sources_version = asyncio.run(_run())
+    summary, overview_status, sources_version, folder_stats = asyncio.run(_run())
+    folder_token = (
+        "skipped"
+        if folder_stats is None
+        else f"{folder_stats['generated']}/{folder_stats['failed']}/{folder_stats['pruned']}"
+    )
     print(
         f"import_docs: files={summary.files} added={summary.added} "
         f"updated={summary.updated} unchanged={summary.unchanged} "
         f"pruned={summary.pruned} errors={summary.errors} chunks={summary.chunks} "
         f"embed_batches={summary.embed_batches} summaries={summary.summaries} "
         f"summary_errors={summary.summary_errors} formats={summary.format_counts()} "
-        f"overview={overview_status} sources_version={sources_version}"
+        f"overview={overview_status} sources_version={sources_version} "
+        f"folder_summaries={folder_token}"
     )
     # Non-zero if any file failed, so cron/CI notice — the rest of the KB
     # was imported and the failed files are retried on the next run. The
-    # overview is best-effort: a failed outline never changes the exit code.
+    # overview and the folder summaries are best-effort: a failed outline
+    # or a failed folder batch never changes the exit code.
     return 1 if summary.errors else 0