phase: 94_ls_tree_drilldown
All green. Verification complete. **Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)** - Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal - Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths - Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met - `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %) - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings - E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched) - Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed - Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol) **Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
@@ -11,15 +11,22 @@ deterministic marker flow in ``tests/e2e/mock_llm.py`` (user message
|
||||
contains ``use your tools`` **and** the system prompt carries the
|
||||
``<tools>`` section of the HIGH prompt; phase 70: the flow emits the
|
||||
harness-aligned names — ``ls`` / ``read`` with the combined
|
||||
``source/path`` identity):
|
||||
``source/path`` identity; phase 94: the drill-down ``ls`` — the top
|
||||
level lists sources only, so the flow drills one level into the first
|
||||
source before the first file line exists):
|
||||
|
||||
1. request 1 (``tools`` offered, no tool results yet) → streams ONLY
|
||||
``tool_calls`` deltas calling ``ls`` (id ``call_0``, no arguments,
|
||||
``finish_reason: "tool_calls"``);
|
||||
2. request 2 (a ``tool``-role catalog result in the messages) → streams a
|
||||
``tool_calls`` delta calling ``read`` on the JOINED combined
|
||||
``source/path`` of the FIRST catalog line (id ``call_1``);
|
||||
3. request 3 (a ``tool``-role read result in the messages) → the content
|
||||
2. request 2 (the top-level source listing in the messages — no file
|
||||
lines yet) → a ``tool_calls`` delta — ``ls`` scoped to the FIRST
|
||||
source of the listing (id ``call_1``) — the drill step (the seed
|
||||
registers ``Deployments`` first, so the drill — and therefore the
|
||||
read — lands on the JSON file);
|
||||
3. request 3 (a ``tool``-role folder listing with file lines) → streams
|
||||
a ``tool_calls`` delta calling ``read`` on the JOINED combined
|
||||
``source/path`` of the FIRST file line (id ``call_2``);
|
||||
4. request 4 (a ``tool``-role read result in the messages) → the content
|
||||
answer ``Read <source/path>. <first 80 chars of the read document's
|
||||
content>`` — so the suite can assert the read document reached the
|
||||
model and landed in the answer.
|
||||
@@ -72,7 +79,7 @@ from sqlalchemy import select, text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document, QueryLog
|
||||
from app.models import Chunk, Document, GitSource, QueryLog
|
||||
from e2e.auth_helpers import login
|
||||
from tests.e2e.mock_llm import embed_text
|
||||
|
||||
@@ -148,7 +155,22 @@ READ_CHIP_HREF = f"/document.html?source={READ_SOURCE}&path={READ_PATH}&back=%2F
|
||||
|
||||
|
||||
def _seed(db: Session) -> None:
|
||||
"""The two-document pair from the TODO (see the module docstring)."""
|
||||
"""The two-document pair from the TODO (see the module docstring).
|
||||
|
||||
Phase 94: the drill-down ``ls`` top level reads the registry — the
|
||||
seed registers BOTH sources (TRUNCATEd in ``_reset_db``),
|
||||
``Deployments`` FIRST: registry order is ``(added_at, id)``, so the
|
||||
mock's drill (first source of the listing) lands on the JSON file
|
||||
deterministically — independent of the operator's
|
||||
``BOR_GIT_SOURCES`` (a non-empty table ignores the env fallback).
|
||||
"""
|
||||
# COMMIT between the inserts (not flush): ``added_at`` is
|
||||
# ``server_default now()`` — the transaction timestamp — and the
|
||||
# tie-break is the random uuid ``id``, so one-transaction rows order
|
||||
# nondeterministically.
|
||||
db.add(GitSource(url=READ_SOURCE, kind="local"))
|
||||
db.commit()
|
||||
db.add(GitSource(url=SEED_SOURCE, kind="local"))
|
||||
md = Document(
|
||||
source=SEED_SOURCE,
|
||||
path=SEED_PATH,
|
||||
@@ -195,7 +217,10 @@ def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
|
||||
"""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview, git_sources"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
if seed is not None:
|
||||
@@ -389,12 +414,14 @@ def test_marker_question_lists_reads_and_quotes(
|
||||
assert i_list is not None and i_read is not None, statuses
|
||||
assert i_list < i_read, statuses
|
||||
|
||||
# Wire level: exactly two `tool` frames — ``ls`` then ``read`` (the
|
||||
# combined source/path as the model passed it) — and both ahead of
|
||||
# the first `delta` frame.
|
||||
# Wire level: exactly three `tool` frames — ``ls`` (the top level),
|
||||
# the drill ``ls`` scoped to the first source (phase 94), then
|
||||
# ``read`` (the combined source/path as the model passed it) — and
|
||||
# all three ahead of the first `delta` frame.
|
||||
frames = _frames(page)
|
||||
assert _tool_frames(frames) == [
|
||||
{"type": "tool", "name": "ls", "argument": None},
|
||||
{"type": "tool", "name": "ls", "argument": READ_SOURCE},
|
||||
{"type": "tool", "name": "read", "argument": READ_SP},
|
||||
]
|
||||
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
|
||||
@@ -408,12 +435,15 @@ def test_marker_question_lists_reads_and_quotes(
|
||||
(READ_SOURCE, READ_PATH),
|
||||
]
|
||||
|
||||
# Both tool lines, in order, above the answer.
|
||||
# All three tool lines, in order, above the answer (phase 94: the
|
||||
# drill line is "Listing documents in <source>").
|
||||
lines = page.locator(".msg.brain .tool-call")
|
||||
expect(lines).to_have_count(2)
|
||||
expect(lines).to_have_count(3)
|
||||
expect(lines.nth(0)).to_contain_text("Listing documents")
|
||||
expect(lines.nth(1)).to_contain_text("Reading ")
|
||||
expect(lines.nth(1)).to_contain_text(READ_SP)
|
||||
expect(lines.nth(1)).to_contain_text("Listing documents in")
|
||||
expect(lines.nth(1)).to_contain_text(READ_SOURCE)
|
||||
expect(lines.nth(2)).to_contain_text("Reading ")
|
||||
expect(lines.nth(2)).to_contain_text(READ_SP)
|
||||
|
||||
# The final answer quotes the read document (the mock's deterministic
|
||||
# quote: "Read <source/path>. <first 80 chars of its content>").
|
||||
@@ -451,18 +481,20 @@ def test_tool_lines_re_render_after_reload(
|
||||
|
||||
_submit(page, MARKER_QUESTION)
|
||||
_wait_settled(page)
|
||||
expect(page.locator(".msg.brain .tool-call")).to_have_count(2)
|
||||
expect(page.locator(".msg.brain .tool-call")).to_have_count(3)
|
||||
|
||||
page.reload()
|
||||
expect(page.locator("#empty-state")).to_be_hidden()
|
||||
|
||||
# The persisted record re-renders BOTH tool lines, in saved order,
|
||||
# through the same append helper as the live frames.
|
||||
# The persisted record re-renders ALL THREE tool lines, in saved
|
||||
# order, through the same append helper as the live frames.
|
||||
restored = page.locator(".msg.brain .tool-call")
|
||||
expect(restored).to_have_count(2)
|
||||
expect(restored).to_have_count(3)
|
||||
expect(restored.nth(0)).to_contain_text("Listing documents")
|
||||
expect(restored.nth(1)).to_contain_text("Reading ")
|
||||
expect(restored.nth(1)).to_contain_text(READ_SP)
|
||||
expect(restored.nth(1)).to_contain_text("Listing documents in")
|
||||
expect(restored.nth(1)).to_contain_text(READ_SOURCE)
|
||||
expect(restored.nth(2)).to_contain_text("Reading ")
|
||||
expect(restored.nth(2)).to_contain_text(READ_SP)
|
||||
|
||||
# Answer + the read-document chip are intact (phase-14 restore path).
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
|
||||
Reference in New Issue
Block a user