phase: 94_ls_tree_drilldown

All green. Verification complete. **Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)** - Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal - Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths - Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met - `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %) - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings - E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched) - Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed - Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol) **Next pending phase:** `95_read_truncation_cap`
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
@@ -4,12 +4,18 @@ The mock (``tests/e2e/mock_llm.py``) classifies marker requests
 statelessly into one step of the agent tool flow. This file pins the
 classification at unit speed — no Playwright, no LLM process:

-* the phase-37 SINGLE-READ flow (``TOOLS_TRIGGER`` only) stays
-  byte-identical: list → read (first catalog line, ``call_1``) → answer;
-* the phase-45 MULTI-READ flow (``TOOLS_TRIGGER`` + ``MULTI_READ_TRIGGER``)
-  classifies by the count of ``tool``-role read results: list → read #1
-  (``call_1``) → read #2 (second catalog line, ``call_2``) → the
-  byte-stable ``multi_answer`` naming both read paths.
+* the phase-37/94 SINGLE-READ flow (``TOOLS_TRIGGER`` only): list (the
+  top-level source listing) → drill (``ls`` scoped to the first source
+  — phase 94: the top level carries sources only, so the flow drills
+  one level for the file lines) → read (first file line, the combined
+  ``source/path``) → answer;
+* the phase-45 MULTI-READ flow (``TOOLS_TRIGGER`` +
+  ``MULTI_READ_TRIGGER``) classifies by the count of ``tool``-role read
+  results: list → drill → read #1 (first file line) → read #2 (second
+  file line) → the byte-stable ``multi_answer`` naming both read paths;
+* the degenerate empty-KB case: every listed source already drilled
+  with no file lines → the flow falls back to the re-list loop (the
+  round cap settles it — the phase-70 empty-catalog behavior).
 """
 from __future__ import annotations

@@ -36,28 +42,50 @@ SYSTEM_LOW = "<relevance>LOW</relevance>\n"
 #: the phase-70 harness-aligned names).
 TOOLS = [{"type": "function", "function": {"name": "ls"}}]

-#: The agent's ``ls`` output for a two-document KB
-#: (``app/rag/agent.py`` ``_execute_tool``): one
-#: ``source: X | path: Y | title: Z`` line per document (phase 63: labeled,
-#: unambiguous fields), ``(source, path)`` order.
-CATALOG_2 = (
-    "2 documents:\n"
-    "source: Deployments | path: example-record-file.json | title: Example Record File\n"
-    "source: Homelab | path: aws-route53.md | title: AWS Route 53 Notes"
+#: The agent's drill-down ``ls`` output for a two-source KB
+#: (``app/rag/agent.py`` ``render_ls_top`` / ``render_folder_listing``,
+#: phase 94): the top level lists the registered sources (registry
+#: order, recursive counts — no file lines); the folder level carries
+#: the file lines (``source: X | path: Y | title: Z`` — the phase-63
+#: labeled fields, unchanged), ``path`` order.
+TOP_LEVEL_2 = (
+    "2 sources:\n"
+    "\n"
+    "Deployments — 1 documents\n"
+    "Homelab — 1 documents"
 )

-CATALOG_1 = (
-    "1 documents:\n"
+#: The first source's root folder: one file line (the single-read
+#: flow's read target).
+FOLDER_DEPLOYMENTS = (
+    "Deployments — 1 documents, 0 folders:\n"
+    "\n"
    "source: Deployments | path: example-record-file.json | title: Example Record File"
 )

-CATALOG_3 = (
-    "3 documents:\n"
+#: Two file lines in the first source (the multi-read flow's reads).
+FOLDER_DEPLOYMENTS_2 = (
+    "Deployments — 2 documents, 0 folders:\n"
+    "\n"
+    "source: Deployments | path: aaa.md | title: AAA\n"
+    "source: Deployments | path: bbb.md | title: BBB"
+)
+
+#: Three file lines in the first source (the listing-order pin: read
+#: #2 is the SECOND line, not the last).
+FOLDER_DEPLOYMENTS_3 = (
+    "Deployments — 3 documents, 0 folders:\n"
+    "\n"
    "source: Deployments | path: aaa.md | title: AAA\n"
    "source: Deployments | path: bbb.md | title: BBB\n"
-    "source: Homelab | path: ccc.md | title: CCC"
+    "source: Deployments | path: ccc.md | title: CCC"
 )

+#: Empty folder levels (a registered source with no documents — the
+#: header line alone; the drill's degenerate arm).
+FOLDER_DEPLOYMENTS_EMPTY = "Deployments — 0 documents, 0 folders:"
+FOLDER_HOMELAB_EMPTY = "Homelab — 0 documents, 0 folders:"
+
 DOC1_SP = "Deployments/example-record-file.json"
 DOC1_CONTENT = (
    "The record file keeps every hosted zone record — first line is longer "
@@ -122,10 +150,39 @@ def test_single_flow_list_step() -> None:
    assert _tool_flow(_body(SINGLE_USER)) == ("list", "", "")


-def test_single_flow_read_step_first_catalog_line() -> None:
-    flow = _tool_flow(_body(SINGLE_USER, (CATALOG_3,)))
-    # The FIRST listing line (Deployments/aaa.md), labeled fields.
-    assert flow == ("read", "Deployments", "aaa.md", "call_1")
+def test_single_flow_drill_step_after_top_level() -> None:
+    # Phase 94: the top level lists SOURCES only — the flow drills one
+    # level into the FIRST source (listing order = registry order).
+    flow = _tool_flow(_body(SINGLE_USER, (TOP_LEVEL_2,)))
+    assert flow == ("drill", "Deployments", "call_1")
+
+
+def test_single_flow_drill_skips_already_drilled_source() -> None:
+    # The first source's folder level is already in the messages (an
+    # empty listing — header only, no file lines): the drill proceeds
+    # to the NEXT un-drilled source.
+    flow = _tool_flow(_body(SINGLE_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS_EMPTY)))
+    assert flow == ("drill", "Homelab", "call_1")
+
+
+def test_single_flow_all_sources_drilled_empty_falls_back_to_list() -> None:
+    # Degenerate: every listed source already drilled, no file lines
+    # anywhere — the flow falls back to the re-list loop (settled at the
+    # round cap, the phase-70 empty-catalog behavior).
+    flow = _tool_flow(
+        _body(
+            SINGLE_USER,
+            (TOP_LEVEL_2, FOLDER_DEPLOYMENTS_EMPTY, FOLDER_HOMELAB_EMPTY),
+        )
+    )
+    assert flow == ("list", "", "")
+
+
+def test_single_flow_read_step_first_file_line() -> None:
+    # The folder level reached: the FIRST file line (Deployments/aaa.md),
+    # labeled fields, the combined ``source/path`` join.
+    flow = _tool_flow(_body(SINGLE_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS_2)))
+    assert flow == ("read", "Deployments", "aaa.md", "call_2")


 def test_read_step_nested_path_stays_intact() -> None:
@@ -133,19 +190,20 @@ def test_read_step_nested_path_stays_intact() -> None:
    # ``source/path — title`` + ``rpartition("/")`` parse misread the
    # split (``source=brain-of-reese-main/homelab``). The labeled fields
    # recover the nested path intact, however deep.
-    catalog = (
-        "1 documents:\n"
+    listing = (
+        "brain-of-reese-main — 1 documents, 0 folders:\n"
+        "\n"
        "source: brain-of-reese-main | path: homelab/aws-route53.md | title: aws-route53"
    )
-    flow = _tool_flow(_body(SINGLE_USER, (catalog,)))
-    assert flow == ("read", "brain-of-reese-main", "homelab/aws-route53.md", "call_1")
+    flow = _tool_flow(_body(SINGLE_USER, (TOP_LEVEL_2, listing)))
+    assert flow == ("read", "brain-of-reese-main", "homelab/aws-route53.md", "call_2")


 def test_single_flow_answer_step_with_tools_offered() -> None:
    # Phase 45: the round cap keeps the tools offered until it is hit —
    # the answer step fires regardless of the ``tools`` parameter.
    flow = _tool_flow(
-        _body(SINGLE_USER, (CATALOG_2, _read_result(DOC1_SP, DOC1_CONTENT)))
+        _body(SINGLE_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS, _read_result(DOC1_SP, DOC1_CONTENT)))
    )
    assert flow == ("answer", DOC1_SP, DOC1_CONTENT)

@@ -154,7 +212,7 @@ def test_single_flow_answer_step_without_tools() -> None:
    flow = _tool_flow(
        _body(
            SINGLE_USER,
-            (CATALOG_2, _read_result(DOC1_SP, DOC1_CONTENT)),
+            (TOP_LEVEL_2, FOLDER_DEPLOYMENTS, _read_result(DOC1_SP, DOC1_CONTENT)),
            tools=None,
        )
    )
@@ -184,22 +242,39 @@ def test_multi_flow_list_step() -> None:
    assert _tool_flow(_body(MULTI_USER)) == ("list", "", "")


+def test_multi_flow_drill_step_after_top_level() -> None:
+    # Phase 94: the top level lists SOURCES only — the multi flow drills
+    # too, before its first read.
+    flow = _tool_flow(_body(MULTI_USER, (TOP_LEVEL_2,)))
+    assert flow == ("drill", "Deployments", "call_1")
+
+
 def test_multi_flow_read_first_step() -> None:
-    flow = _tool_flow(_body(MULTI_USER, (CATALOG_2,)))
-    assert flow == ("read", DOC1_SP.split("/", 1)[0], DOC1_SP.rsplit("/", 1)[1], "call_1")
+    flow = _tool_flow(_body(MULTI_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS)))
+    assert flow == ("read", DOC1_SP.split("/", 1)[0], DOC1_SP.rsplit("/", 1)[1], "call_2")


 def test_multi_flow_read_second_step_skips_already_read() -> None:
-    flow = _tool_flow(_body(MULTI_USER, (CATALOG_2, _read_result(DOC1_SP, DOC1_CONTENT))))
-    # The second catalog line — the first line differing from DOC1.
-    assert flow == ("read", "Homelab", "aws-route53.md", "call_2")
+    flow = _tool_flow(
+        _body(
+            MULTI_USER,
+            (TOP_LEVEL_2, FOLDER_DEPLOYMENTS_2, _read_result("Deployments/aaa.md", DOC1_CONTENT)),
+        )
+    )
+    # The second file line — the first line differing from the read doc.
+    assert flow == ("read", "Deployments", "bbb.md", "call_3")


 def test_multi_flow_read_second_is_listing_order_not_last() -> None:
-    # Three-doc catalog, first doc read: read #2 is the SECOND line
+    # Three-file listing, first file read: read #2 is the SECOND line
    # (Deployments/bbb.md), not the last one.
-    flow = _tool_flow(_body(MULTI_USER, (CATALOG_3, _read_result("Deployments/aaa.md", "x"))))
-    assert flow == ("read", "Deployments", "bbb.md", "call_2")
+    flow = _tool_flow(
+        _body(
+            MULTI_USER,
+            (TOP_LEVEL_2, FOLDER_DEPLOYMENTS_3, _read_result("Deployments/aaa.md", "x")),
+        )
+    )
+    assert flow == ("read", "Deployments", "bbb.md", "call_3")


 def test_multi_flow_answer_step_names_both_paths() -> None:
@@ -207,7 +282,8 @@ def test_multi_flow_answer_step_names_both_paths() -> None:
        _body(
            MULTI_USER,
            (
-                CATALOG_2,
+                TOP_LEVEL_2,
+                FOLDER_DEPLOYMENTS_2,
                _read_result(DOC1_SP, DOC1_CONTENT),
                _read_result(DOC2_SP, DOC2_CONTENT),
            ),
@@ -227,7 +303,8 @@ def test_multi_flow_answer_step_without_tools_offered() -> None:
        _body(
            MULTI_USER,
            (
-                CATALOG_2,
+                TOP_LEVEL_2,
+                FOLDER_DEPLOYMENTS_2,
                _read_result(DOC1_SP, DOC1_CONTENT),
                _read_result(DOC2_SP, DOC2_CONTENT),
            ),
@@ -238,11 +315,11 @@ def test_multi_flow_answer_step_without_tools_offered() -> None:
    assert flow[0] == "multi_answer"


-def test_multi_flow_one_document_catalog_degenerates_to_single_answer() -> None:
+def test_multi_flow_one_file_listing_degenerates_to_single_answer() -> None:
    # Nothing second to read — the single-read answer shape, quoting the
    # only read result.
    flow = _tool_flow(
-        _body(MULTI_USER, (CATALOG_1, _read_result(DOC1_SP, DOC1_CONTENT)))
+        _body(MULTI_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS, _read_result(DOC1_SP, DOC1_CONTENT)))
    )
    assert flow == ("answer", DOC1_SP, DOC1_CONTENT)

@@ -320,9 +397,12 @@ def test_search_flow_found_step_without_tools_offered() -> None:


 def test_search_flow_ignores_catalog_and_read_results() -> None:
-    # A catalog (labeled lines) and a read result ("Document …" prefix)
-    # are NOT search results — the flow stays at the search step.
-    flow = _search_flow(_body(SEARCH_USER, (CATALOG_2, _read_result(DOC1_SP, DOC1_CONTENT))))
+    # Listings (top-level + folder level, labeled file lines) and a read
+    # result ("Document …" prefix) are NOT search results — the flow
+    # stays at the search step.
+    flow = _search_flow(
+        _body(SEARCH_USER, (TOP_LEVEL_2, FOLDER_DEPLOYMENTS, _read_result(DOC1_SP, DOC1_CONTENT)))
+    )
    assert flow == ("search",)