phase: 94_ls_tree_drilldown

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`

TOOL_CALLING_TESTING.md

@@ -127,14 +127,21 @@ state is snapshotted to **`tests/fixtures/test_kb.dump.sql`** — a
 data-only SQL script (TRUNCATE + one multi-row INSERT per app table:
 documents, chunks + embeddings, the `git_sources` local rows that make
 the source registry self-contained, the static KB overview, the
-sources version). Restoring it puts the whole known state back in one
-transaction; the generated `chunks.tsv` column is recomputed by
-Postgres. The dump is verified by round-trip at build time (restore +
-per-table checksum compare — a serialization bug fails the build).
+sources version, and — since the 2026-09-11 phase-94 refresh — the
+sync-time `folder_summaries` rows the drill-down `ls` shows). The
+build generates those rows against the live `lite` endpoint through
+the real sync-time generator (4 rows for this KB: the two source
+roots + `homelab/containers` + `homelab/networking` — the ≥ 2-document
+folders; single-document folders carry no row by design). Restoring it
+puts the whole known state back in one transaction; the generated
+`chunks.tsv` column is recomputed by Postgres. The dump is verified by
+round-trip at build time (restore + per-table checksum compare — a
+serialization bug fails the build).
 
 ```bash
 # Rebuild the KB + dump — only when the fixture documents, the
-# chunker, or the embedding model change. NOT part of the loop.
+# chunker, the embedding model, or the folder-summary prompt change
+# (its `lite` output is baked in — phase 94). NOT part of the loop.
 uv run python -m scripts.load_test_kb
 # restores the fixture KB standalone (what --restore runs inline)
 uv run python -m scripts.restore_test_kb
@@ -170,22 +177,31 @@ of it.
 
 | # | question | tests | expected ideal |
 |---|---|---|---|
-| 1 | List the files in this directory. | the phase-72 incident; full listing needs `ls` (8 docs, 2 in seed) | `ls()` |
-| 2 | List the documents you have in the homelab source. | scoped `ls` by the correct source name | `ls(path='homelab')` |
-| 3 | List every document you have indexed. | no-arg listing | `ls()` |
+| 1 | List the files in this directory. | the phase-72 incident; the top level lists the sources — a full file listing drills them (phase 94) | `ls()` → drill `ls('deployments')` + `ls('homelab')` (+ leaf folders) |
+| 2 | List the documents you have in the homelab source. | scoped `ls` by the correct source name; the result is the source's folders (+ counts/summaries) — file lines come from the leaf drills (phase 94) | `ls(path='homelab')` (+ leaf drills) |
+| 3 | List every document you have indexed. | the "list everything" expectation is now the whole tree — no single call lists every file (phase 94) | `ls()` → drill both sources and every folder |
 | 4 | Open the document homelab/networking/vela-bridges.md … | `read`, combined form | `read('homelab/networking/vela-bridges.md')` |
 | 5 | Read deployments/quadlet/mimir-service.md and summarize it. | `read`, unseeded target | `read(…)` (or `ls` first, then `read`) |
 | 6 | Open the document homelab/networking/meridian-notes.md … | `read`, unseeded target | `read(…)` |
 | 7 | Find the exact string "rbm-8842" in your documents … | `grep`, pattern only | `grep(pattern='rbm-8842')` — the match line alone answers it |
 | 8 | Which document has the title "Lab Ansible Inventory"? Summarize it. | title lookup; target IS seeded | answer from context (or `ls`) |
 | 9 | What do you know about the qwen 3.8 llama.cpp setup? … | topic lookup; target IS seeded | answer from context |
-| 10 | List the files in the deployments directory. | source name phrased as a directory | `ls(path='deployments')` |
+| 10 | List the files in the deployments directory. | source name phrased as a directory; the result is the source's folders — file lines come from the leaf drills (phase 94) | `ls(path='deployments')` (+ leaf drills) |
 
 Questions 4–6 name the **full combined identity** (no bare-path trap —
 that is the job of the locked derived battery, §6). Questions 7–9 name
 content, so their target document is seeded; the correct behavior
 there is to **not** re-read what is already in the prompt.
 
+Phase-94 note: the listing questions' (1, 2, 3, 10) correct behavior
+**moved with the owner-permitted `ls` surface change** (2026-09-10,
+`TODO.md` L4 — `00_phase.md` of `94_ls_tree_drilldown`): a full file
+listing is now a drill sequence (`ls()` → source → folder; one level
+per call, the old whole-KB flood is structurally gone), and a folder
+line's count + sync-time summary is the tree's picture of that
+subtree. The questions themselves are unchanged (the methodology's
+locked battery) — only the expected-behavior definition moved.
+
 ### The four pass conditions
 
 1. all 10 turns answer (no `LLMError`/`MalformedReplyError`);
@@ -205,10 +221,11 @@ gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/13 executed (6
 gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 7/11 executed (64%)  contract 11/11 (100%)  (wall 46.8s)
 gate: lite PASS turns=10 answered=10 caps=0 tool-turns=9 calls 9/12 executed (75%) contract 11/12 (92%) 2026-09-06 (wall 40.4s)
 gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 9/14 executed (64%) contract 13/14 (93%) 2026-09-06 (wall 40.5s)
+gate: lite PASS turns=10 answered=10 caps=0 tool-turns=9 calls 13/18 executed (72%) contract 18/18 (100%) 2026-09-11 (wall 43.6s)   [phase-94 ls tree]
 ```
 
-Contract accuracy ≥ 90 %: **met** (100 / 92 / 100 / 93 / 92 / 93). The executed
-ratio sits at 58–73 % for the reason documented in §5 — an app
+Contract accuracy ≥ 90 %: **met** (100 / 92 / 100 / 93 / 92 / 93 / 100). The
+executed ratio sits at 58–73 % for the reason documented in §5 — an app
 semantics choice, not a model defect, and the open design question in
 §7.
 
@@ -222,6 +239,7 @@ gate: turbo FAIL turns=10 answered=10 caps=0 tool-turns=5 calls 5/5 executed (10
 gate: turbo PASS turns=10 answered=10 caps=0 tool-turns=7 calls 9/9 executed (100%) contract 9/9 (100%) 2026-09-06 (wall 113.7s)
 gate: turbo PASS turns=10 answered=10 caps=0 tool-turns=7 calls 7/7 executed (100%) contract 7/7 (100%) 2026-09-06 (wall 112.8s)
 gate: turbo FAIL turns=10 answered=10 caps=0 tool-turns=5 calls 5/5 executed (100%) contract 5/5 (100%) 2026-09-06 (wall 97.2s)   [derived battery — MISS: 5/10 tool-turn floor]
+gate: turbo PASS turns=10 answered=10 caps=0 tool-turns=7 calls 18/19 executed (95%) contract 19/19 (100%) 2026-09-11 (wall 98.7s)   [phase-94 ls tree]
 ```
 
 Reads: the re-read habit is model-specific. `lite` re-reads a seeded
@@ -304,7 +322,12 @@ The verdict carries both:
   combined `source/path` belongs, a nonexistent document identity, a
   source name where a document belongs (`ls(path='.')`,
   `ls(path='/')`, `grep(path='homelab')` — the entire phase-72
-  incident class).
+  incident class). Phase 94 moved the `ls` contract (owner-permitted
+  surface revision): `ls(path)` is ALSO contract-correct for an
+  EXISTING `source/folder` — the drill-down; the violation is now an
+  unknown first segment, a bare folder name (no source prefix), or a
+  folder matching no indexed prefix. The gate-side mirror
+  (`classify_call` in `scripts/agent_realmodel_check.py`) follows.
 - **executed/emitted** (the phase-72 locked metric) = calls the app
   actually executed ÷ emitted. Every refusal class counts against it
   — **including `ALREADY_IN_CONTEXT`**, the app's dedupe refusal when
@@ -394,7 +417,8 @@ podman compose up -d db
 cp .env.example .env                # once; LLM endpoint + DB URL
 uv run alembic upgrade head
 
-# 1. Build the controlled KB + dump (one-off, ~2 s — real embeddings)
+# 1. Build the controlled KB + dump (one-off, ~7 s — real embeddings +
+#    4 live-lite folder-summary calls since phase 94)
 uv run python -m scripts.load_test_kb
 #    → prints the retrieval report (all 10 must be grounded) and
 #      verifies the dump by round-trip.
@@ -421,3 +445,147 @@ Diagnosing a bad run: every call is logged by `run_agent`
 refusal templates in `app/rag/agent.py` to see which teaching line the
 model hit, and which refusal class (contract violation vs. in-context
 dedupe) the rejection was.
+
+---
+
+## 9. Phase 94 — the `ls` drill-down tree (2026-09-11)
+
+Phase 94 rebuilt `ls` as a drill-down tree (owner-permitted
+tool-surface revision, `TODO.md` L4 — the `ls` result format and
+`path` semantics changed; the tool name and the `read`/`grep`
+contract are untouched): `ls()` lists the synced sources with a
+per-source sync-time summary, `ls(source)` lists its folders (each
+with its summary) + its own files, `ls(source/folder)` drills one
+level deeper; file lines are capped at 50 with a grep pointer. This
+section is the controlled battery's verdict on that architecture —
+the TODO's gate: "ensure accuracy and performance aren't too badly
+impacted", read per the phase-94 task file as **accuracy at or above
+the recorded baseline** (no contract/verdict regression on the
+10-question battery) and **wall time within ~20% of the baseline
+total**.
+
+**Fixture dump refresh (a precondition of this run).** The 2026-09-04
+dump predates migration 0017 — it had no `folder_summaries` rows, so
+the drill-down's summaries would have been absent from the controlled
+KB. It was rebuilt on 2026-09-11 with
+`uv run python -m scripts.load_test_kb`: the same 8 hand-written
+documents through the real pipeline (real embeddings), the static KB
+overview + sources version re-stored, and the sync-time folder
+summaries generated against the live `lite` endpoint (4 rows: the two
+source roots + `homelab/containers` + `homelab/networking` — the
+≥ 2-document folders). The dump now carries the `folder_summaries`
+table (TRUNCATE + INSERT + round-trip checksum verification — the
+house rule); `--restore` measured **0.03 s** hot.
+
+**The classifier mirror followed the surface.** `classify_call`
+(`scripts/agent_realmodel_check.py`) is documented as the gate-side
+mirror of `app.rag.agent._execute_tool`'s resolution rules — task 03
+changed the `ls` resolution (a `source/folder` path is now valid) but
+the mirror still flagged every folder drill-down as a contract
+violation. Fixed in this phase: `ls(path)` is contract-correct for
+`""`, a registered source, or an EXISTING `source/folder`; without
+the fix the battery would have scored the *correct* new behavior as
+inaccurate. (No battery question changed; the §3 expected-behavior
+notes moved with the owner-permitted surface.)
+
+**The runs.** Against the live aipi endpoint, the refreshed fixture
+KB, the real grounded path — two models: the **configured** chat
+model (`turbo` per `.env` — the verdict) and `lite` (the `.env` model
+temporarily switched, then restored byte-identical — the house
+procedure of the 2026-09-05 turbo comparison) to check the
+phase-70/72 standing block, which was recorded under `lite`. The
+micro-loop (`--turns 3` — the three listing turns) ran twice on
+`turbo` first: 12/12 and 13/13 contract (100 %), zero caps — so the
+full battery went straight to the verdict, with no copy-lever
+iteration needed (the first full run on each model was green).
+
+```
+$ uv run python -m scripts.agent_realmodel_check --restore --mode fixture
+restore: ok in 0.03s (8 docs, 2 sources)
+turn 01 | emitted=1 executed=1 cap=no defl=no |  6.80s | List the files in this directory.
+turn 02 | emitted=3 executed=3 cap=no defl=no |  9.77s | List the documents you have in the …
+turn 03 | emitted=7 executed=7 cap=no defl=no | 14.83s | List every document you have indexed.
+turn 04 | emitted=0 executed=0 cap=no defl=no |  8.89s | Open the document …
+turn 05 | emitted=1 executed=1 cap=no defl=no | 12.48s | Read …
+turn 06 | emitted=1 executed=1 cap=no defl=no | 10.03s | Open the document …
+turn 07 | emitted=1 executed=1 cap=no defl=no |  6.85s | Find the exact string "rbm-8842" in …
+turn 08 | emitted=0 executed=0 cap=no defl=no |  7.92s | Which document has the title "Lab …
+turn 09 | emitted=0 executed=0 cap=no defl=no | 10.90s | What do you know about the qwen 3.8 …
+turn 10 | emitted=5 executed=4 cap=no defl=no | 10.08s | List the files in the deployments …
+gate: turbo PASS turns=10 answered=10 caps=0 tool-turns=7 calls 18/19 executed (95%) contract 19/19 (100%) 2026-09-11 (wall 98.7s)
+```
+
+```
+(same command; `.env` chat model temporarily `lite`, restored after)
+turn 01 | emitted=1 executed=1 cap=no defl=no |  3.82s | List the files in this directory.
+turn 02 | emitted=8 executed=5 cap=no defl=no | 10.93s | List the documents you have in the …
+turn 03 | emitted=1 executed=1 cap=no defl=no |  2.25s | List every document you have indexed.
+turn 04 | emitted=1 executed=0 cap=no defl=no |  3.84s | Open the document …
+turn 05 | emitted=3 executed=3 cap=no defl=no |  4.99s | Read …
+turn 06 | emitted=1 executed=1 cap=no defl=no |  5.64s | Open the document …
+turn 07 | emitted=1 executed=1 cap=no defl=no |  2.25s | Find the exact string "rbm-8842" in …
+turn 08 | emitted=1 executed=0 cap=no defl=no |  5.09s | Which document has the title "Lab …
+turn 09 | emitted=0 executed=0 cap=no defl=no |  2.93s | What do you know about the qwen 3.8 …
+turn 10 | emitted=1 executed=1 cap=no defl=no |  1.75s | List the files in the deployments …
+gate: lite PASS turns=10 answered=10 caps=0 tool-turns=9 calls 13/18 executed (72%) contract 18/18 (100%) 2026-09-11 (wall 43.6s)
+```
+
+**Per-turn reading.** Both runs show the designed drill-down: the
+listing turns hop `ls()` → source → folder (turbo turn 03: top level
++ both sources + five leaf folders in 7 calls; lite turn 02:
+`ls('homelab')` + all three homelab folders), the read turns keep the
+combined `source/path` form (turbo turn 05 drills
+`ls('deployments')` → `ls('deployments/quadlet')` before the read —
+the "or `ls` first, then `read`" branch), the grep turn is
+pattern-only, and the seeded discipline turns (04/08/09) answer from
+the `<documents>` context. Zero cap hits, zero contract violations,
+10/10 answered on both models. The non-executed emitted calls are all
+contract-correct (the contract is 100 %): turbo's one (turn 10) is a
+second call in a single reply that the one-tool-per-reply rule
+discards; lite's five are the same discard class (three on turn 02)
+plus the two documented copy-invariant `ALREADY_IN_CONTEXT` re-reads
+of seeded documents (turns 04/08 — §4: 15/15 across every copy
+variant tried). lite turn 02 also carried one exploratory
+`grep(pattern='homelab')` (pattern-only — contract-correct, executed,
+then the proper folder drills in the following rounds — the same
+self-correcting exploration class §5 documents pre-94).
+
+**Baseline comparison.** Against the last recorded baseline per model
+(the 2026-09-06 runs for `turbo`; the phase-70/72 standing block for
+`lite`):
+
+| model | baseline wall | phase-94 wall | Δ wall | baseline contract | phase-94 contract |
+|---|---|---|---|---|---|
+| `turbo` (configured — the verdict) | 112.8–113.7 s | 98.7 s | −12.5 % to −13.2 % | 100 % (7/7, 9/9) | 100 % (19/19) |
+| `lite` (phase-70/72 standing) | 40.4–40.5 s | 43.6 s | +7.7–7.9 % | 92–100 % (last: 92 / 93) | 100 % (18/18) |
+
+Both inside the ~20 % band — `turbo` *under* its same-model baseline
+despite executing MORE calls (18 vs 7–9: the drill-down adds rounds,
+but per-round latency was lower on this day than on the baseline runs
+— wall is endpoint-load-sensitive, which is why a band, not a point,
+is the bar), `lite` +7.7 % — the modest increase the task file
+expects from a few extra `ls` hops. Accuracy: contract at or above
+the same-model baseline on both models; verdict PASS on both; the
+executed ratios (95 % / 72 %) sit in the documented bands (§4/§5 —
+discard class + app-semantics, not tool-calling errors).
+
+**Conclusion.** The phase-94 gate is **met**: accuracy at or above
+baseline (100 % contract, PASS, zero cap hits, zero contract
+violations — on both models) and wall time within the 20 % band on
+both models (−12.5/−13.2 % same-model `turbo`, +7.7 % `lite`
+standing). The phase-72 incident class is gone under the tree: the
+flood is structurally impossible (one level per call, 50-line cap)
+and the listing turns drill cleanly with the folder summaries doing
+the steering. No copy-lever change was made (nothing regressed to
+iterate on).
+
+**Observation for the owner (not a gate miss).** The top level now
+carries the two source summaries, and on the `lite` run the two
+"list everything" turns (01, 03) answered after a single `ls()` —
+from the source-level lines — without drilling to the files; `turbo`
+drilled (3 / 7 calls on the same turns). The gate measures
+contract/verdict, which is met either way; if the owner wants
+listing answers to reach the file level by default, the lever is the
+`ls` description / `TOOLS_SECTION` copy (teach that a folder's file
+lines appear only after drilling into it) — flagged here rather than
+acted on unilaterally.