phase: 94_ls_tree_drilldown
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 25s

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
+67 -30
View File
@@ -11,19 +11,26 @@ message contains BOTH ``use your tools`` (``TOOLS_TRIGGER``) and ``read
two documents`` (``MULTI_READ_TRIGGER``) **and** the system prompt
carries the ``<tools>`` section of the HIGH prompt; phase 70: the flow
emits the harness-aligned names — ``ls``, then ``read`` on the JOINED
combined ``source/path`` of each catalog line):
combined ``source/path`` of each file line; phase 94: the drill-down
``ls`` — the top level lists sources only, so the flow drills one
level into the first source before the first file line exists):
1. request 1 (``tools`` offered, no tool results yet) → streams ONLY
``tool_calls`` deltas calling ``ls`` (id ``call_0``);
2. request 2 (the ``tool``-role catalog result) → ``read`` on the
JOINED combined ``source/path`` of the FIRST catalog line
(id ``call_1``);
3. request 3 (one ``tool``-role read result) → ``read`` on the JOINED
combined ``source/path`` of the SECOND catalog line (id ``call_2``)
— the pre-phase-45 per-tool budgets would have refused exactly this
2. request 2 (the top-level source listing in the messages — no file
lines yet) → the drill: ``ls`` scoped to the FIRST source of the
listing (id ``call_1``); the seed registers ``Deployments`` first,
and both read documents live in it — so the drill's folder listing
carries BOTH file lines;
3. request 3 (a ``tool``-role folder listing with file lines) →
``read`` on the JOINED combined ``source/path`` of the FIRST file
line (id ``call_2``);
4. request 4 (one ``tool``-role read result) → ``read`` on the JOINED
combined ``source/path`` of the SECOND file line (id ``call_3``) —
the pre-phase-45 per-tool budgets would have refused exactly this
second read (``No reading budget left — answer with what you
have.``);
4. request 4 (two read results) → the forced answer, byte-stable: the
5. request 5 (two read results) → the forced answer, byte-stable: the
single-read shape quoting the FIRST read result, plus the line
``I read <sp1> and <sp2>.`` naming both read paths in read order.
@@ -49,10 +56,11 @@ rejection, not the multi-read
flow this story proves.
Test → story mapping (Playwright Mapping Rule):
1. ``test_multi_read_turn`` — the turn streams THREE ``tool`` frames /
``.tool-call`` lines in order (one list — "is listing documents" —
and two reads — "is reading <source/path>" — the #send-status
transition recorded deterministically via MutationObserver), then a
1. ``test_multi_read_turn`` — the turn streams FOUR ``tool`` frames /
``.tool-call`` lines in order (the top-level list, the drill list —
phase 94 — "is listing documents", and two reads — "is reading
<source/path>" — the #send-status transition recorded
deterministically via MutationObserver), then a
final non-deflected answer containing the mock's byte-stable
``I read <sp1> and <sp2>.`` line; the round cap (default 10) bounds
the turn, no budget refusal anywhere.
@@ -84,7 +92,7 @@ from sqlalchemy import select, text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, QueryLog
from app.models import Chunk, Document, GitSource, QueryLog
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
@@ -209,7 +217,23 @@ def _doc(source: str, path: str, title: str, content: str) -> Document:
def _seed(db: Session) -> None:
"""The three-document KB from the module docstring."""
"""The three-document KB from the module docstring.
Phase 94: the drill-down ``ls`` top level reads the registry —
register BOTH sources (TRUNCATEd in ``_reset_db``), ``Deployments``
FIRST (registry order is ``(added_at, id)``): the mock's drill
(first source of the listing) lands on the folder that carries
BOTH file lines. A non-empty table also ignores the operator's
``BOR_GIT_SOURCES`` fallback — deterministic.
"""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the RANDOM uuid ``id``, so two rows in one
# transaction order nondeterministically (the integration
# ``registry`` fixture's pattern).
db.add(GitSource(url=READ1_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = _doc(SEED_SOURCE, SEED_PATH, "AWS Route 53 Notes", ROUTE53_CONTENT)
db.add(md)
db.flush()
@@ -240,7 +264,10 @@ def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""
with SessionLocal() as db:
db.execute(
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources"
)
)
db.commit()
if seed is not None:
@@ -399,13 +426,16 @@ def test_multi_read_turn(
_submit(page, MULTI_QUESTION)
_wait_settled(page)
# Wire level: exactly THREE `tool` frames — ls, read #1, read #2
# (each read's argument is the JOINED combined source/path), in
# order — and all ahead of the first `delta` frame. This third
# frame is the one the pre-phase-45 read budget refused.
# Wire level: exactly FOUR `tool` frames — the top-level ls, the
# drill ls scoped to the first source (phase 94), then read #1 and
# read #2 (each read's argument is the JOINED combined
# source/path), in order — and all ahead of the first `delta`
# frame. The fourth frame is the one the pre-phase-45 read budget
# refused.
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "ls", "argument": None},
{"type": "tool", "name": "ls", "argument": READ1_SOURCE},
{"type": "tool", "name": "read", "argument": READ1_SP},
{"type": "tool", "name": "read", "argument": READ2_SP},
]
@@ -439,14 +469,17 @@ def test_multi_read_turn(
), statuses
assert i_list < i_read1 < i_read2, statuses
# Three visible tool lines, in order, above the answer.
# Four visible tool lines, in order, above the answer (phase 94:
# the drill line is "Listing documents in <source>").
lines = page.locator(".msg.brain .tool-call")
expect(lines).to_have_count(3)
expect(lines).to_have_count(4)
expect(lines.nth(0)).to_contain_text("Listing documents")
expect(lines.nth(1)).to_contain_text("Reading ")
expect(lines.nth(1)).to_contain_text(READ1_SP)
expect(lines.nth(1)).to_contain_text("Listing documents in")
expect(lines.nth(1)).to_contain_text(READ1_SOURCE)
expect(lines.nth(2)).to_contain_text("Reading ")
expect(lines.nth(2)).to_contain_text(READ2_SP)
expect(lines.nth(2)).to_contain_text(READ1_SP)
expect(lines.nth(3)).to_contain_text("Reading ")
expect(lines.nth(3)).to_contain_text(READ2_SP)
# The final answer is non-deflected, quotes the FIRST read result,
# and names BOTH read paths (the mock's byte-stable line).
@@ -563,19 +596,23 @@ def test_single_tool_flow_regression(
_submit(page, SINGLE_QUESTION)
_wait_settled(page)
# Exactly TWO tool frames — ls then ONE read of the first catalog
# line (the JOINED combined source/path) — no second read (the
# marker carries no multi-read trigger).
# Exactly THREE tool frames — the top-level ls, the drill ls
# (phase 94), then ONE read of the first file line (the JOINED
# combined source/path) — no second read (the marker carries no
# multi-read trigger).
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "ls", "argument": None},
{"type": "tool", "name": "ls", "argument": READ1_SOURCE},
{"type": "tool", "name": "read", "argument": READ1_SP},
]
lines = page.locator(".msg.brain .tool-call")
expect(lines).to_have_count(2)
expect(lines).to_have_count(3)
expect(lines.nth(0)).to_contain_text("Listing documents")
expect(lines.nth(1)).to_contain_text("Reading ")
expect(lines.nth(1)).to_contain_text(READ1_SP)
expect(lines.nth(1)).to_contain_text("Listing documents in")
expect(lines.nth(1)).to_contain_text(READ1_SOURCE)
expect(lines.nth(2)).to_contain_text("Reading ")
expect(lines.nth(2)).to_contain_text(READ1_SP)
# The single-read answer shape: quotes the read document; it does
# NOT carry the multi-read both-named line (READ2 was never read).