phase: 94_ls_tree_drilldown
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 25s

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
+53 -21
View File
@@ -11,15 +11,22 @@ deterministic marker flow in ``tests/e2e/mock_llm.py`` (user message
contains ``use your tools`` **and** the system prompt carries the
``<tools>`` section of the HIGH prompt; phase 70: the flow emits the
harness-aligned names — ``ls`` / ``read`` with the combined
``source/path`` identity):
``source/path`` identity; phase 94: the drill-down ``ls`` — the top
level lists sources only, so the flow drills one level into the first
source before the first file line exists):
1. request 1 (``tools`` offered, no tool results yet) → streams ONLY
``tool_calls`` deltas calling ``ls`` (id ``call_0``, no arguments,
``finish_reason: "tool_calls"``);
2. request 2 (a ``tool``-role catalog result in the messages) → streams a
``tool_calls`` delta calling ``read`` on the JOINED combined
``source/path`` of the FIRST catalog line (id ``call_1``);
3. request 3 (a ``tool``-role read result in the messages) → the content
2. request 2 (the top-level source listing in the messages — no file
lines yet) → a ``tool_calls`` delta — ``ls`` scoped to the FIRST
source of the listing (id ``call_1``) — the drill step (the seed
registers ``Deployments`` first, so the drill — and therefore the
read — lands on the JSON file);
3. request 3 (a ``tool``-role folder listing with file lines) → streams
a ``tool_calls`` delta calling ``read`` on the JOINED combined
``source/path`` of the FIRST file line (id ``call_2``);
4. request 4 (a ``tool``-role read result in the messages) → the content
answer ``Read <source/path>. <first 80 chars of the read document's
content>`` — so the suite can assert the read document reached the
model and landed in the answer.
@@ -72,7 +79,7 @@ from sqlalchemy import select, text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, QueryLog
from app.models import Chunk, Document, GitSource, QueryLog
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
@@ -148,7 +155,22 @@ READ_CHIP_HREF = f"/document.html?source={READ_SOURCE}&path={READ_PATH}&back=%2F
def _seed(db: Session) -> None:
"""The two-document pair from the TODO (see the module docstring)."""
"""The two-document pair from the TODO (see the module docstring).
Phase 94: the drill-down ``ls`` top level reads the registry — the
seed registers BOTH sources (TRUNCATEd in ``_reset_db``),
``Deployments`` FIRST: registry order is ``(added_at, id)``, so the
mock's drill (first source of the listing) lands on the JSON file
deterministically — independent of the operator's
``BOR_GIT_SOURCES`` (a non-empty table ignores the env fallback).
"""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
@@ -195,7 +217,10 @@ def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""
with SessionLocal() as db:
db.execute(
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources"
)
)
db.commit()
if seed is not None:
@@ -389,12 +414,14 @@ def test_marker_question_lists_reads_and_quotes(
assert i_list is not None and i_read is not None, statuses
assert i_list < i_read, statuses
# Wire level: exactly two `tool` frames — ``ls`` then ``read`` (the
# combined source/path as the model passed it) — and both ahead of
# the first `delta` frame.
# Wire level: exactly three `tool` frames — ``ls`` (the top level),
# the drill ``ls`` scoped to the first source (phase 94), then
# ``read`` (the combined source/path as the model passed it) — and
# all three ahead of the first `delta` frame.
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "ls", "argument": None},
{"type": "tool", "name": "ls", "argument": READ_SOURCE},
{"type": "tool", "name": "read", "argument": READ_SP},
]
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
@@ -408,12 +435,15 @@ def test_marker_question_lists_reads_and_quotes(
(READ_SOURCE, READ_PATH),
]
# Both tool lines, in order, above the answer.
# All three tool lines, in order, above the answer (phase 94: the
# drill line is "Listing documents in <source>").
lines = page.locator(".msg.brain .tool-call")
expect(lines).to_have_count(2)
expect(lines).to_have_count(3)
expect(lines.nth(0)).to_contain_text("Listing documents")
expect(lines.nth(1)).to_contain_text("Reading ")
expect(lines.nth(1)).to_contain_text(READ_SP)
expect(lines.nth(1)).to_contain_text("Listing documents in")
expect(lines.nth(1)).to_contain_text(READ_SOURCE)
expect(lines.nth(2)).to_contain_text("Reading ")
expect(lines.nth(2)).to_contain_text(READ_SP)
# The final answer quotes the read document (the mock's deterministic
# quote: "Read <source/path>. <first 80 chars of its content>").
@@ -451,18 +481,20 @@ def test_tool_lines_re_render_after_reload(
_submit(page, MARKER_QUESTION)
_wait_settled(page)
expect(page.locator(".msg.brain .tool-call")).to_have_count(2)
expect(page.locator(".msg.brain .tool-call")).to_have_count(3)
page.reload()
expect(page.locator("#empty-state")).to_be_hidden()
# The persisted record re-renders BOTH tool lines, in saved order,
# through the same append helper as the live frames.
# The persisted record re-renders ALL THREE tool lines, in saved
# order, through the same append helper as the live frames.
restored = page.locator(".msg.brain .tool-call")
expect(restored).to_have_count(2)
expect(restored).to_have_count(3)
expect(restored.nth(0)).to_contain_text("Listing documents")
expect(restored.nth(1)).to_contain_text("Reading ")
expect(restored.nth(1)).to_contain_text(READ_SP)
expect(restored.nth(1)).to_contain_text("Listing documents in")
expect(restored.nth(1)).to_contain_text(READ_SOURCE)
expect(restored.nth(2)).to_contain_text("Reading ")
expect(restored.nth(2)).to_contain_text(READ_SP)
# Answer + the read-document chip are intact (phase-14 restore path).
bubble = page.locator(".msg.brain .bubble").last