phase: 94_ls_tree_drilldown
All green. Verification complete. **Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)** - Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal - Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths - Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met - `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %) - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings - E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched) - Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed - Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol) **Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
@@ -13,9 +13,13 @@ deterministic marker flows in ``tests/e2e/mock_llm.py`` (phase 70: the
|
||||
flows emit the NEW names with the NEW argument shapes):
|
||||
|
||||
* the READ flow (``use your tools`` (``TOOLS_TRIGGER``) + the HIGH
|
||||
prompt's ``<tools>`` section): ``ls`` (id ``call_0``, no arguments) →
|
||||
``read`` on the JOINED combined ``source/path`` of the first catalog
|
||||
line (id ``call_1``) → the ``Read <source/path>. <quote>`` answer;
|
||||
prompt's ``<tools>`` section; phase 94: the drill-down ``ls`` — the
|
||||
top level lists sources only, so the flow drills one level before the
|
||||
first file line exists): ``ls`` (id ``call_0``, no arguments) → the
|
||||
drill ``ls`` scoped to the first source of the listing
|
||||
(id ``call_1``) → ``read`` on the JOINED combined ``source/path`` of
|
||||
the first file line (id ``call_2``) → the ``Read <source/path>.
|
||||
<quote>`` answer;
|
||||
* the SEARCH flow (``search your documents`` (``SEARCH_TRIGGER``) + the
|
||||
``<tools>`` section): ``grep`` with ``{"pattern": SEARCH_PATTERN}``
|
||||
(id ``call_0``) → the ``Found <matched line>`` answer.
|
||||
@@ -41,13 +45,14 @@ KB fixtures:
|
||||
|
||||
Test → phase mapping (Playwright Mapping Rule):
|
||||
1. ``test_read_flow_lines_answer_sources_no_raw_markup`` — the
|
||||
grounded READ turn: the UI shows the ``ls`` line (unscoped "🔎
|
||||
Listing documents", no argument) then the "📄 Reading <source/path>"
|
||||
line with the combined path in a ``<code>`` element, the answer
|
||||
streams and quotes the read document, the done-state sources
|
||||
include the read document, and NO raw tool markup (``<|…|>``,
|
||||
``tool_call``) appears anywhere in the DOM — the live incident this
|
||||
phase fixes.
|
||||
grounded READ turn: the UI shows the unscoped ``ls`` line ("🔎
|
||||
Listing documents", no argument), the drill line ("🔎 Listing
|
||||
documents in <source>" — phase 94, the source in a ``<code>``
|
||||
element), then the "📄 Reading <source/path>" line with the
|
||||
combined path in a ``<code>`` element, the answer streams and quotes
|
||||
the read document, the done-state sources include the read document,
|
||||
and NO raw tool markup (``<|…|>``, ``tool_call``) appears anywhere
|
||||
in the DOM — the live incident this phase fixes.
|
||||
2. ``test_grep_flow_line_then_answer`` — the grounded SEARCH turn: the
|
||||
"🔎 Searching for <pattern>" line (sentinel in ``<code>``) then the
|
||||
matched-line answer.
|
||||
@@ -74,7 +79,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document
|
||||
from app.models import Chunk, Document, GitSource
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient
|
||||
from e2e.auth_helpers import login
|
||||
@@ -156,7 +161,22 @@ READ_ANSWER_QUOTE = RECORD_FILE_CONTENT[:80]
|
||||
|
||||
|
||||
def _seed_read_pair(db: Session) -> None:
|
||||
"""The two-document READ-flow KB (see the module docstring)."""
|
||||
"""The two-document READ-flow KB (see the module docstring).
|
||||
|
||||
Phase 94: the drill-down ``ls`` top level reads the registry —
|
||||
register BOTH sources (TRUNCATEd in ``_reset_db_read_pair``),
|
||||
``Deployments`` FIRST (registry order ``(added_at, id)``): the
|
||||
mock's drill (first source of the listing) lands on the JSON file
|
||||
— the read the assertions expect. A non-empty table also ignores
|
||||
the operator's ``BOR_GIT_SOURCES`` fallback — deterministic.
|
||||
"""
|
||||
# COMMIT between the inserts (not flush): ``added_at`` is
|
||||
# ``server_default now()`` — the transaction timestamp — and the
|
||||
# tie-break is the random uuid ``id``, so one-transaction rows order
|
||||
# nondeterministically.
|
||||
db.add(GitSource(url=READ_SOURCE, kind="local"))
|
||||
db.commit()
|
||||
db.add(GitSource(url=SEED_SOURCE, kind="local"))
|
||||
md = Document(
|
||||
source=SEED_SOURCE,
|
||||
path=SEED_PATH,
|
||||
@@ -267,7 +287,10 @@ def _reset_db_read_pair() -> None:
|
||||
answers."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview, git_sources"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
_seed_read_pair(db)
|
||||
@@ -386,15 +409,18 @@ def test_read_flow_lines_answer_sources_no_raw_markup(
|
||||
_submit(page, READ_QUESTION)
|
||||
_wait_settled(page)
|
||||
|
||||
# The UI shows the ls line (UNSCOPED — no argument, no <code>) then
|
||||
# The UI shows the ls line (UNSCOPED — no argument, no <code>),
|
||||
# the drill line (phase 94 — the source in a <code> element), then
|
||||
# the "📄 Reading <source/path>" line with the COMBINED path in a
|
||||
# <code> element (the path is data, never markup).
|
||||
lines = page.locator(".msg.brain .tool-call")
|
||||
expect(lines).to_have_count(2)
|
||||
expect(lines).to_have_count(3)
|
||||
expect(lines.nth(0)).to_contain_text("Listing documents")
|
||||
expect(lines.nth(0).locator("code")).to_have_count(0)
|
||||
expect(lines.nth(1)).to_contain_text("Reading ")
|
||||
expect(lines.nth(1).locator("code")).to_have_text(READ_SP)
|
||||
expect(lines.nth(1)).to_contain_text("Listing documents in")
|
||||
expect(lines.nth(1).locator("code")).to_have_text(READ_SOURCE)
|
||||
expect(lines.nth(2)).to_contain_text("Reading ")
|
||||
expect(lines.nth(2).locator("code")).to_have_text(READ_SP)
|
||||
|
||||
# The answer streamed and quotes the read document (the mock's
|
||||
# deterministic echo: "Read <source/path>. <first 80 chars>").
|
||||
@@ -402,12 +428,14 @@ def test_read_flow_lines_answer_sources_no_raw_markup(
|
||||
expect(bubble).to_contain_text(READ_ANSWER_PREFIX)
|
||||
expect(bubble).to_contain_text(READ_ANSWER_QUOTE)
|
||||
|
||||
# Wire level: ls then read — the phase-70 argument rule (ls
|
||||
# unscoped → null; read → the combined path as passed) — ahead of
|
||||
# the first delta.
|
||||
# Wire level: ls, the drill ls scoped to the first source (phase
|
||||
# 94), then read — the phase-70 argument rule (ls unscoped → null;
|
||||
# scoped ls → the source name; read → the combined path as passed)
|
||||
# — ahead of the first delta.
|
||||
frames = _drain_frames(page)
|
||||
assert _tool_frames(frames) == [
|
||||
{"type": "tool", "name": "ls", "argument": None},
|
||||
{"type": "tool", "name": "ls", "argument": READ_SOURCE},
|
||||
{"type": "tool", "name": "read", "argument": READ_SP},
|
||||
]
|
||||
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
|
||||
@@ -508,10 +536,12 @@ def test_wire_argument_rule_across_both_flows(
|
||||
read_tools = _tool_frames(read_frames)
|
||||
search_tools = _tool_frames(search_frames)
|
||||
# The ordered, combined tool-frame sequence across both flows: the
|
||||
# argument rule end-to-end — read → the combined path as passed,
|
||||
# grep → the pattern, ls → null when unscoped.
|
||||
# argument rule end-to-end — the drill ls (phase 94) → the source
|
||||
# name, read → the combined path as passed, grep → the pattern,
|
||||
# ls → null when unscoped.
|
||||
assert read_tools + search_tools == [
|
||||
{"type": "tool", "name": "ls", "argument": None},
|
||||
{"type": "tool", "name": "ls", "argument": READ_SOURCE},
|
||||
{"type": "tool", "name": "read", "argument": READ_SP},
|
||||
{"type": "tool", "name": "grep", "argument": SEARCH_PATTERN},
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user