phase: 94_ls_tree_drilldown
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 25s

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
+58 -31
View File
@@ -19,27 +19,33 @@ phase-64 default 0.15 is far below the 5 s tool-line threshold):
NOTE the prelude: the chat flow's QUESTION EMBEDDING also travels
through the proxy (one 6 s sleep) before the agent loop starts, so
the wall-clock timeline of one turn is:
the wall-clock timeline of one turn is (phase 94: the drill-down
``ls`` adds a drill step — the top level lists sources only, so the
flow drills one level before the first file line exists):
1. t≈12 s — the first SSE ``tool`` frame ("🔎 Listing documents"): the
6 s embedding sleep + request 1's 6 s sleep (``tools`` offered, no
tool results yet — streams ONLY the ``ls`` ``tool_calls`` delta,
id ``call_0``) + the mock's ~0.2 s tool-call stream;
2. t≈19 s — the second ``tool`` frame ("📄 Reading
Deployments/example-record-file.json"): request 2's 6 s sleep (a
``tool``-role catalog result → streams a ``read`` ``tool_calls``
delta on the JOINED combined ``source/path`` of the FIRST catalog
line, id ``call_1``);
3. t≈25 s — the content answer ``Read <source/path>. <first 80 chars
of the read document's content>``: request 3's 6 s sleep (a
2. t≈19 s — the second ``tool`` frame ("🔎 Listing documents in
Deployments"): request 2's 6 s sleep (the top-level source listing
in the messages — no file lines yet → streams the drill ``ls``
scoped to the first source, id ``call_1``);
3. t≈25 s — the third ``tool`` frame ("📄 Reading
Deployments/example-record-file.json"): request 3's 6 s sleep (a
``tool``-role folder listing with file lines → streams a ``read``
``tool_calls`` delta on the JOINED combined ``source/path`` of the
FIRST file line, id ``call_2``);
4. t≈31 s — the content answer ``Read <source/path>. <first 80 chars
of the read document's content>``: request 4's 6 s sleep (a
``tool``-role read result) → the first answer ``delta``.
So each post-tool-frame gap (≈6.3 s — the 6 s sleep + the short
tool-call stream) is PAST the 5 s tool-line threshold
(``TOOL_LINE_ELAPSED_AFTER_MS = 5_000``), and the 10 s pre-token
typing hint (the existing ``startThinkingClock`` gate, ticking from
t≈0) is visible throughout both post-tool gaps — everything before the
answer's first delta (the whole turn runs ≈25 s + overhead, acceptable
t≈0) is visible throughout all post-tool gaps — everything before the
answer's first delta (the whole turn runs ≈31 s + overhead, acceptable
for an isolated story suite). The four tests pin: (1) the latest tool
line's ticking "(Ns)" suffix, (2) the typing indicator's visible "Ns"
hint (+ the kept aria channel), (3) BOTH settling the instant the
@@ -55,15 +61,20 @@ task that writes the visible span) — so the two channels read
polls until they align (the deterministic same-moment pin).
**KB fixture** — byte-identical to the phase-37 suite
(``tests/e2e/test_agent_document_tools.py``): ``Homelab/aws-route53.md``
(``tests/e2e/test_agent_document_tools.py``), plus the phase-94
registry rows: both sources are registered in ``git_sources``
(``Deployments`` FIRST — registry order ``(added_at, id)``), so the
mock's drill (first source of the top-level listing) lands on the
folder that carries the file lines. ``Homelab/aws-route53.md``
carries one chunk embedded with the mock's own bag-of-words vector —
the marker question (phase 37's exact question, which carries the
trigger phrase) cosines ≈0.69 against it, well past the E2E 0.30
threshold, so the turn is grounded and the HIGH prompt carries the
``<tools>`` section; ``Deployments/example-record-file.json`` is
indexed WITHOUT chunks, and its ``(source, path)`` sorts FIRST in the
catalog (``Deployments`` < ``Homelab``) — exactly the line the mock's
second request reads, so the answer is byte-stable.
folder listing's file lines (``Deployments`` < ``Homelab``) — exactly
the line the mock's read request targets, so the answer is
byte-stable.
Test → story mapping (Playwright Mapping Rule):
1. ``test_tool_line_shows_ticking_elapsed``
@@ -92,7 +103,7 @@ from sqlalchemy.orm import Session
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Chunk, Document
from app.models import Chunk, Document, GitSource
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
@@ -211,8 +222,10 @@ SUFFIX_TIMEOUT_MS = 12_000
#: the indicator is removed at the first delta (≈25 s): 20 s from the
#: check start cannot flake.
TYPING_HINT_TIMEOUT_MS = 20_000
#: The answer settles at ≈19 s from submit; every settle wait here starts
#: ≥12 s in — 30 s keeps ≥2× headroom on the remaining window.
#: The answer settles at ≈31 s from submit (phase 94: the drill step
#: adds a third tool round); every settle wait here starts ≈17 s in
#: (after the first line's suffix) — 30 s keeps ≥2× headroom on the
#: remaining window.
SETTLE_TIMEOUT_MS = 30_000
#: The restore is a synchronous boot re-render — 15 s is ample.
RESTORE_TIMEOUT_MS = 15_000
@@ -360,7 +373,16 @@ def app_url(app_server: str) -> str:
def _seed(db: Session) -> None:
"""The phase-37 two-document pair, byte-identical (see the module
docstring)."""
docstring), plus the phase-94 registry rows: both sources
registered, ``Deployments`` FIRST — the mock's drill (first
source of the top-level listing) lands on the JSON file."""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
@@ -408,7 +430,7 @@ def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, "
"steering_notes, kb_overview, saved_chats"
"steering_notes, kb_overview, saved_chats, git_sources"
)
)
db.commit()
@@ -514,7 +536,7 @@ def _wait_settled(page: Page) -> None:
#: The FIRST tool line's suffix, pinned to the line ("first .tool-call
#: child of the .tool-calls container" — the "Listing documents" line;
#: the read frame's second line arms its OWN clock 5 s later and is
#: the drill frame's second line arms its OWN clock 5 s later and is
#: never the target here).
FIRST_LINE_SUFFIX = "#messages .tool-calls .tool-call:first-child .tool-elapsed"
@@ -611,8 +633,8 @@ def test_typing_indicator_shows_visible_elapsed(
)
# The clock ticks while the gap holds: ≥1.5 s later the value is
# strictly greater (the read frame is ≥6 s away; the first delta
# — which removes the whole indicator — is ≈25 s from submit).
# strictly greater (the drill frame is ≥6 s away; the first delta
# — which removes the whole indicator — is ≈31 s from submit).
page.wait_for_timeout(int(SAMPLE_GAP_S * 1000))
v2 = _elapsed_value(
page.locator("#typing-indicator .typing-elapsed").first.text_content(),
@@ -644,7 +666,7 @@ def test_indicators_settle_when_the_answer_arrives(
# First the ticking state (the test-1 wait, reused)…
expect(page.locator(FIRST_LINE_SUFFIX)).to_be_visible(timeout=SUFFIX_TIMEOUT_MS)
# …then the answer arrives (≈25 s from submit — see the module
# …then the answer arrives (≈31 s from submit — see the module
# docstring's timeline) and settles.
_wait_settled(page)
@@ -661,13 +683,16 @@ def test_indicators_settle_when_the_answer_arrives(
expect(page.locator("#messages .tool-elapsed")).to_have_count(0)
expect(page.locator("#typing-indicator")).to_have_count(0)
# The tool lines remain — the permanent record, both present with
# their pinned text (phase 37's pattern).
# The tool lines remain — the permanent record, all three present
# with their pinned text (phase 37's pattern + the phase-94 drill
# line).
lines = page.locator("#messages .tool-call")
expect(lines).to_have_count(2)
expect(lines).to_have_count(3)
expect(lines.nth(0)).to_contain_text("Listing documents")
expect(lines.nth(1)).to_contain_text("Reading ")
expect(lines.nth(1)).to_contain_text(READ_SP)
expect(lines.nth(1)).to_contain_text("Listing documents in")
expect(lines.nth(1)).to_contain_text(READ_SOURCE)
expect(lines.nth(2)).to_contain_text("Reading ")
expect(lines.nth(2)).to_contain_text(READ_SP)
# The answer bubble is complete (the mock's deterministic quote).
bubble = page.locator(".msg.brain .bubble").last
@@ -705,12 +730,14 @@ def test_restored_turn_has_no_timer(
expect(page.locator("#empty-state")).to_be_hidden(timeout=RESTORE_TIMEOUT_MS)
# The persisted record re-renders the tool lines, in saved order
# (the phase-37 reload pin, mirrored)…
# (the phase-37 reload pin, mirrored + the phase-94 drill line)…”
restored = page.locator("#messages .tool-call")
expect(restored).to_have_count(2, timeout=RESTORE_TIMEOUT_MS)
expect(restored).to_have_count(3, timeout=RESTORE_TIMEOUT_MS)
expect(restored.nth(0)).to_contain_text("Listing documents")
expect(restored.nth(1)).to_contain_text("Reading ")
expect(restored.nth(1)).to_contain_text(READ_SP)
expect(restored.nth(1)).to_contain_text("Listing documents in")
expect(restored.nth(1)).to_contain_text(READ_SOURCE)
expect(restored.nth(2)).to_contain_text("Reading ")
expect(restored.nth(2)).to_contain_text(READ_SP)
# …with NO timer (A6 — the restore never arms the clock) and the
# page otherwise settled.