phase: 97_kb_tree_catalog

All completion criteria verified — everything is green, no defects found. Final report:

## Phase 97 final verification pass — ALL GREEN

**Verified (no code changes needed):**
- `GET /api/docs/tree` (admin), `build_kb_tree` pure builder, `PATCH /api/folders/summary`, migration 0018 (`manually_edited`, head confirmed), generator skip/keep + `kept_manual` stat, RAG tree UI + edit affordance in `sources.js`/`index.html`/`styles.css`
- `tests/e2e/test_kb_tree.py`: 8 passed — top level, drill source/folder, edit round-trip, clear, manual-desc-survives-sync, reload fallback, anonymous gate
- Integration: tree shape/order/403/empty/indexed-only + PATCH update/create/root/clear/404/403/no-LLM + stat-walk equivalence (in `test_docs_api.py`); 3-field `folder_summaries=` import token preserved

**Gates (exact commands):**
- `uv run pytest --cov=app --cov-report=term-missing` → **2053 passed**, TOTAL coverage **99%** (>90% ✓)
- `uv run ruff check . && uv run pyright` → **All checks passed / 0 errors**
- `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` → **8 passed** in isolation
- 30 story/RAG-view E2E suites run **one per process**: all passed, incl. `test_ls_tree_drilldown` (agent `ls` byte-identical ✓), `test_import_documents`, `test_edit_summaries`, `test_admin_auth`, `test_kb_overview`

**Completion criteria:** tree view ✓ · edit round-trip + clear ✓ · manual persists/clear resets ✓ · `ls` unchanged ✓ · pytest/coverage/lint ✓ · E2E isolation ✓ · commit — left to harness per protocol (working tree untouched, `git add/commit` not run)

**Deviations:** none. **Next pending phase:** none — `todo/` contains only 97 (96 already committed).

tests/unit/test_kb_tree_builder.py

@@ -0,0 +1,285 @@
+"""Unit tests: ``app.api.docs.build_kb_tree`` (phase 97, task 02).
+
+The PURE tree builder behind ``GET /api/docs/tree`` — the RAG view's
+drill-down tree, the same tree the agent's ``ls`` walks plus file
+metadata. Driven without a database (module-level function, plain
+inputs): multi-source ordering (the superset rule), the 0-document
+registered source, the nested recursive counts, the phase-94 existence
+rule, the file/folder name collision, ordering, summaries, verbatim
+file metadata, and the ``group_folder_listing`` cross-check property
+("the UI shows what the agent sees") at the root and one nested level.
+"""
+from __future__ import annotations
+
+from app.api.docs import TreeDocRow, build_kb_tree
+from app.rag.agent import group_folder_listing
+
+T0 = "2026-09-01T08:00:00+00:00"
+T1 = "2026-09-02T08:00:00+00:00"
+T2 = "2026-09-03T08:00:00+00:00"
+
+
+def _folder_nodes(node) -> list:
+    """The folder-kind children of a source/folder node, in order."""
+    return [child for child in node.children if child.kind == "folder"]
+
+
+def _file_nodes(node) -> list:
+    """The file-kind children of a source/folder node, in order."""
+    return [child for child in node.children if child.kind == "file"]
+
+
+def test_multi_source_registry_order_leads_and_indexed_only_appended() -> None:
+    """Sources = the registry names in order, then the distinct
+    indexed-only sources in alphabetical order (the superset rule)."""
+    names = ["beta", "alpha", "empty"]  # registry order — NOT alphabetical
+    doc_rows: list[TreeDocRow] = [
+        ("beta", "b.md", "B", 1, T0),
+        ("alpha", "a.md", "A", 1, T0),
+        ("gamma", "g.md", "G", 1, T0),  # indexed-only → appended
+        ("delta", "d.md", "D", 1, T0),  # indexed-only → appended
+    ]
+    tree = build_kb_tree(names, doc_rows, {})
+    assert [s.name for s in tree] == ["beta", "alpha", "empty", "delta", "gamma"]
+    # The registry sources keep their registry order even though it is
+    # not alphabetical; the indexed-only ones trail, sorted.
+    assert tree[0].name == "beta"
+    assert tree[1].name == "alpha"
+    assert tree[3].name == "delta"
+    assert tree[4].name == "gamma"
+
+
+def test_registered_zero_document_source_lists_empty() -> None:
+    """A registered source with no indexed documents lists (the
+    phase-70/72 invariant): ``documents: 0``, no children, summary null
+    when nothing is stored."""
+    tree = build_kb_tree(["solo"], [], {})
+    assert len(tree) == 1
+    assert tree[0].name == "solo"
+    assert tree[0].documents == 0
+    assert tree[0].children == []
+    assert tree[0].summary is None
+
+
+def test_nested_document_counts_into_source_ancestors_and_own_folder() -> None:
+    """A document under ``a/b/c/`` contributes to the source, to ``a``,
+    to ``a/b``, and to ``a/b/c`` (the recursive subtree, the phase-94
+    ``ls`` count rule)."""
+    rows = [
+        ("S", "a/b/c/deep.md", "Deep", 1, T0),
+        ("S", "a/b/shallow.md", "Shallow", 1, T0),
+        ("S", "a/top.md", "Top", 1, T0),
+        ("S", "root.md", "Root", 1, T0),
+    ]
+    (source,) = build_kb_tree(["S"], rows, {})
+    assert source.documents == 4
+    a = _folder_nodes(source)[0]
+    assert a.path == "a"
+    assert a.documents == 3  # deep + shallow + top
+    a_b = _folder_nodes(a)[0]
+    assert a_b.path == "a/b"
+    assert a_b.documents == 2  # deep + shallow
+    a_b_c = _folder_nodes(a_b)[0]
+    assert a_b_c.path == "a/b/c"
+    assert a_b_c.documents == 1  # deep only
+    assert [f.path for f in _file_nodes(a_b_c)] == ["a/b/c/deep.md"]
+    assert [f.path for f in _file_nodes(a)] == ["a/top.md"]
+
+
+def test_existence_rule_a_file_path_is_never_a_folder() -> None:
+    """A folder node appears only with a true descendant (some path
+    starts with ``folder + "/"``); a document's own path — even one
+    with dots — never creates a folder."""
+    rows = [
+        ("S", "x.md", "X", 1, T0),
+        ("S", "x.y/z.md", "Z", 1, T0),
+    ]
+    (source,) = build_kb_tree(["S"], rows, {})
+    folders = _folder_nodes(source)
+    # Only ``x.y`` exists (``x.y/z.md`` starts with ``x.y/``); ``x`` and
+    # ``x.md`` are file names, not folders.
+    assert [f.path for f in folders] == ["x.y"]
+    assert [f.path for f in _file_nodes(source)] == ["x.md"]
+    assert [f.path for f in _file_nodes(folders[0])] == ["x.y/z.md"]
+
+
+def test_file_folder_name_collision_both_appear() -> None:
+    """A document sharing a directory's name: BOTH appear — the folder
+    node (via its descendants) and the file node (its own row); the
+    colliding file counts into the folder's subtree (the ``ls`` count
+    rule's ``path == folder`` arm)."""
+    rows = [
+        ("S", "a", "File A", 1, T0),  # a file wearing the folder's name
+        ("S", "a/b.md", "B", 1, T0),  # makes ``a`` a folder
+    ]
+    (source,) = build_kb_tree(["S"], rows, {})
+    folders = _folder_nodes(source)
+    files = _file_nodes(source)
+    assert [f.path for f in folders] == ["a"]
+    assert folders[0].documents == 2  # "a" itself + "a/b.md"
+    assert [f.path for f in files] == ["a"]
+    assert files[0].title == "File A"
+    assert [f.path for f in _file_nodes(folders[0])] == ["a/b.md"]
+
+
+def test_subfolder_path_order_and_file_catalog_order() -> None:
+    """Direct subfolders list in path (sorted) order; direct files keep
+    the input (catalog — ``GET /api/docs``) order, independent of the
+    subfolder ordering."""
+    rows = [
+        ("S", "zeta/z1.md", "Z1", 1, T0),
+        ("S", "alpha/a1.md", "A1", 1, T0),
+        ("S", "mike/m1.md", "M1", 1, T0),
+        ("S", "beta/b1.md", "B1", 1, T0),
+        ("S", "z-file.md", "Z", 1, T0),  # file AFTER the folders in input
+        ("S", "a-file.md", "A", 1, T0),  # file before it in input
+    ]
+    (source,) = build_kb_tree(["S"], rows, {})
+    assert [f.path for f in _folder_nodes(source)] == ["alpha", "beta", "mike", "zeta"]
+    # Input order is preserved for the files (z-file.md precedes
+    # a-file.md in the input, so it does too here — catalog order is
+    # the INPUT order, not a re-sort).
+    assert [f.path for f in _file_nodes(source)] == ["z-file.md", "a-file.md"]
+
+
+def test_summaries_present_and_absent() -> None:
+    """``summary`` is the stored row (source root ``""`` or folder
+    path) or null when absent — any row (AI or manual is indistinguishable
+    here; the builder carries whatever is stored)."""
+    rows = [
+        ("S", "one/a.md", "A", 1, T0),
+        ("S", "one/b.md", "B", 1, T0),
+        ("S", "two/c.md", "C", 1, T0),
+    ]
+    summaries = {("S", ""): "Source desc.", ("S", "one"): "One desc."}
+    # ("S", "two") is NOT stored → null.
+    (source,) = build_kb_tree(["S"], rows, summaries)
+    assert source.summary == "Source desc."
+    one, two = _folder_nodes(source)
+    assert one.summary == "One desc."
+    assert two.summary is None
+    # File nodes carry no summary key at all (the 00_phase.md shape).
+    file = _file_nodes(one)[0]
+    assert set(file.model_dump()) == {"kind", "path", "title", "chunks", "indexed_at"}
+    assert "summary" not in file.__class__.model_fields
+
+
+def test_file_metadata_unchanged_in_tree() -> None:
+    """File ``title`` / ``chunks`` / ``indexed_at`` ride into the tree
+    verbatim from the catalogue row (no reformatting)."""
+    rows = [("S", "deep/x/y.md", "The Title", 7, T2)]
+    (source,) = build_kb_tree(["S"], rows, {})
+    file = _file_nodes(_folder_nodes(_folder_nodes(source)[0])[0])[0]
+    assert file.path == "deep/x/y.md"
+    assert file.title == "The Title"
+    assert file.chunks == 7
+    assert file.indexed_at == T2
+
+
+def test_empty_inputs_empty_tree() -> None:
+    """No registry sources and no indexed documents → an empty tree
+    (the RAG view's ``{"sources": []}`` case)."""
+    assert build_kb_tree([], [], {}) == []
+
+
+def test_indexed_document_under_unlisted_source_is_impossible() -> None:
+    """By construction (the superset rule) every doc source is listed —
+    the registry names lead and every other doc source follows; there
+    is no input where a document is dropped."""
+    names = ["reg-b", "reg-a"]
+    rows: list[TreeDocRow] = [
+        ("reg-b", "b.md", "B", 1, T0),
+        ("zzz", "z.md", "Z", 1, T0),
+        ("aaa", "a.md", "A", 1, T0),
+        ("reg-a", "a.md", "A2", 1, T0),
+    ]
+    tree = build_kb_tree(names, rows, {})
+    listed = [s.name for s in tree]
+    assert listed == ["reg-b", "reg-a", "aaa", "zzz"]
+    doc_paths = {
+        f.path for s in tree for f in s.children if f.kind == "file"
+    }
+    # Every document from the input is present exactly once.
+    assert doc_paths == {"b.md", "z.md", "a.md"}
+    assert sum(s.documents for s in tree) == len(rows)
+
+
+# --------------------------------------------------------------------
+# The cross-check property — "the UI shows what the agent sees":
+# for a single-source dataset the builder's level equals
+# ``app.rag.agent.group_folder_listing``'s output.
+# --------------------------------------------------------------------
+
+#: The shared single-source dataset: nested folders, root files, a
+#: summary-stored folder and an unstored one. Paths are in (source,
+#: path) catalog order; titles map 1:1 to paths.
+CROSS_ROWS: list[TreeDocRow] = [
+    ("S", "note", "Note", 1, T0),
+    ("S", "one/a.md", "A", 2, T1),
+    ("S", "one/b.md", "B", 0, T1),
+    ("S", "one/two/c.md", "C", 3, T1),
+    ("S", "one/two/d.md", "D", 1, T1),
+    ("S", "root.md", "Root", 4, T0),
+    ("S", "zz/e.md", "E", 2, T2),
+    ("S", "zz/f.md", "F", 2, T2),
+]
+
+CROSS_SUMMARIES = {
+    ("S", ""): "Source desc.",
+    ("S", "one"): "One desc.",
+    ("S", "zz"): "Zz desc.",
+    # ("S", "one/two") deliberately unstored → null at that level.
+}
+
+
+def _cross_check(folder: str, builder_node) -> None:
+    """Assert the builder's level *folder* equals
+    ``group_folder_listing("S", folder, ...)`` — same subfolder
+    ``(path, count, summary)`` triples in order AND same file
+    ``(path, title)`` pairs in order (uncapped — the dataset is well
+    under the ``ls`` 50-line cap, so the cap is inert)."""
+    rows = [(path, title) for _source, path, title, _chunks, _stamp in CROSS_ROWS]
+    source_summaries = {
+        folder_path: summary
+        for (source, folder_path), summary in CROSS_SUMMARIES.items()
+        if source == "S"
+    }
+    subs, files, _total = group_folder_listing("S", folder, rows, source_summaries)
+    assert [(f.path, f.documents, f.summary) for f in _folder_nodes(builder_node)] == subs
+    assert [(f.path, f.title) for f in _file_nodes(builder_node)] == [
+        (path, title) for _source, path, title in files
+    ]
+
+
+def test_cross_check_root_level_matches_group_folder_listing() -> None:
+    """At the source root the builder's direct subfolders / direct
+    files equal the agent's root level, element for element."""
+    (source,) = build_kb_tree(["S"], CROSS_ROWS, CROSS_SUMMARIES)
+    _cross_check("", source)
+    # And the explicit expectations (the test is readable without the
+    # helper): one → 4 docs (its whole subtree), zz → 2.
+    assert source.documents == 8
+    assert [(f.path, f.documents, f.summary) for f in _folder_nodes(source)] == [
+        ("one", 4, "One desc."),
+        ("zz", 2, "Zz desc."),
+    ]
+    assert [(f.path, f.title) for f in _file_nodes(source)] == [
+        ("note", "Note"),
+        ("root.md", "Root"),
+    ]
+
+
+def test_cross_check_nested_level_matches_group_folder_listing() -> None:
+    """One nested level (``one``): subfolder ``one/two`` (2 docs, no
+    stored summary) + the direct files ``one/a.md`` / ``one/b.md`` —
+    equal to the agent's drill into the same folder."""
+    (source,) = build_kb_tree(["S"], CROSS_ROWS, CROSS_SUMMARIES)
+    one = next(f for f in _folder_nodes(source) if f.path == "one")
+    _cross_check("one", one)
+    assert [(f.path, f.documents, f.summary) for f in _folder_nodes(one)] == [
+        ("one/two", 2, None),
+    ]
+    assert [(f.path, f.title) for f in _file_nodes(one)] == [
+        ("one/a.md", "A"),
+        ("one/b.md", "B"),
+    ]