phase: 94_ls_tree_drilldown
All green. Verification complete. **Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)** - Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal - Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths - Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met - `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %) - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings - E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched) - Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed - Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol) **Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
+432
-84
@@ -1,5 +1,6 @@
|
||||
"""Agent loop: the grounded-turn document tools (phase 37, task 03; the
|
||||
harness-aligned ``ls``/``read``/``grep`` surface, phase 70).
|
||||
harness-aligned ``ls``/``read``/``grep`` surface, phase 70; the
|
||||
drill-down tree ``ls`` + sync-time folder summaries, phase 94).
|
||||
|
||||
Probe verdict (task 01 — ``uv run python -m scripts.llm_probe --tools``
|
||||
run live against aipi): **``probe: turbo tool_calls=supported 2026-08-26``**
|
||||
@@ -48,14 +49,41 @@ task 04):
|
||||
split (with its self-correction and "teach the split" refusals) is
|
||||
gone — the model's combined form is now simply correct. The
|
||||
phase-68 A5 match/output contract rides along under the new name.
|
||||
Phase 94 revision (owner permission 2026-09-10, ``TODO.md`` L4 — the
|
||||
tool-surface revision recorded in the phase 94 overview ``00_phase.md``;
|
||||
PLAN.md is being redone by the owner): the ``ls`` RESULT format and
|
||||
``path`` semantics changed — ``ls`` is now a filesystem-style
|
||||
drill-down tree (no path: the synced sources with counts + stored
|
||||
source-root summaries; a source name: its top-level folders + files;
|
||||
a ``source/folder`` path: that folder's subfolders + files — the
|
||||
model drills one level per call instead of flooding the whole
|
||||
catalog into one result), while the tool NAME and the
|
||||
``read``/``grep`` contract (combined ``source/path``) are untouched.
|
||||
2. Each tool call the model emits is executed server-side against
|
||||
Postgres only (no LLM, no network): ``ls`` returns the indexed
|
||||
catalog — one ``source: X | path: Y | title: Z`` line per document
|
||||
(phase 63: labeled fields — unambiguous for LLM parsing),
|
||||
``GET /api/docs`` order (uncapped in v1; the UI never shows it, only
|
||||
the model does) — optionally scoped to one source name (a ``path``
|
||||
argument matching no source name is a refusal; a registered source
|
||||
with no indexed documents lists as ``0 documents:`` and counts) —
|
||||
Postgres only (no LLM, no network): ``ls`` lists ONE level of the
|
||||
drill-down tree (phase 94, task 03) — no ``path``: every registered
|
||||
source (registry order; a 0-document source still lists) as
|
||||
``{source} — {n} documents`` plus the indented stored source-root
|
||||
summary line when one is in ``folder_summaries``; a ``path`` that is
|
||||
a registered source name (no ``/``): that source's root folder —
|
||||
each direct subfolder `` {sub}/ — {m} documents[: {summary}]``
|
||||
(the count is the subfolder's recursive subtree — every document
|
||||
whose path equals the folder or starts with ``folder + "/"``, the
|
||||
same set the sync-time folder summary describes — and the file
|
||||
lines ``source: X | path: Y | title: Z`` (the canonical
|
||||
``read``/``grep`` identity — the phase-63 labeled format,
|
||||
unchanged) in path order (``GET /api/docs`` order), capped at
|
||||
:data:`LS_MAX_FILE_LINES` lines + one deterministic grep-pointer
|
||||
note for the rest (a 500-file folder costs 50 lines, never 500);
|
||||
a ``source/folder`` ``path``: that folder's subfolders + own file
|
||||
lines (the same template, ``identity = source + "/" + folder``);
|
||||
a registered source with no documents lists its header line alone
|
||||
(``… — 0 documents, 0 folders:`` — the old ``0 documents:``
|
||||
behavior preserved in spirit) — every successful listing (top/root/
|
||||
folder) counts; a ``source/…`` argument whose first segment names no
|
||||
registered source is the no-source refusal (below, the segment
|
||||
echoed), and a folder segment matching no indexed prefix is the
|
||||
:data:`NOT_A_FOLDER` teaching (below) —
|
||||
``read`` takes the combined ``source/path`` string, splits it at the
|
||||
FIRST ``'/'`` (source names are directory basenames — they can never
|
||||
contain ``'/'``), and returns the document's **full** content
|
||||
@@ -82,15 +110,19 @@ task 04):
|
||||
blank or non-string) → ``"read requires a string argument
|
||||
'path'."``; a ``grep`` without a usable ``pattern`` (missing, blank
|
||||
or non-string) → ``"grep requires a string argument
|
||||
'pattern'."``; a scoped ``ls`` whose (stripped) ``path`` contains a
|
||||
``/`` — a document path where a source name belongs (source names
|
||||
are directory basenames and can never contain one; the 2026-09-03
|
||||
incident's ``ls(path='app/rag/importer.py')``) →
|
||||
:data:`LS_PATH_NOT_A_SOURCE`, the document-path teaching line with
|
||||
the argument echoed; a scoped ``ls`` whose ``path`` names no
|
||||
registered source (no ``/`` — the incident's ``ls(path='.')``)
|
||||
→ :data:`NO_SOURCE_NOT_A_DIRECTORY`, the no-source refusal with the
|
||||
teaching parenthetical appended; a document
|
||||
'pattern'."``; a scoped ``ls`` whose ``path``'s FIRST segment (split
|
||||
at the first ``/``) names no registered source — a bare unknown
|
||||
name (the 2026-09-03 incident's ``ls(path='.')``) or the source
|
||||
segment of a ``source/…`` argument (phase 94: a ``/`` now names a
|
||||
folder, so the phase-72 document-path teaching refusal is deleted)
|
||||
→ :data:`NO_SOURCE_NOT_A_DIRECTORY`, the
|
||||
no-source refusal with the teaching parenthetical appended, the
|
||||
segment echoed; a ``source/…`` argument whose folder segment
|
||||
matches no indexed prefix (the phase-94 existence rule — some
|
||||
indexed path of the source starts with ``folder + "/"``) →
|
||||
:data:`NOT_A_FOLDER`, the drill-down teaching with the argument
|
||||
echoed and the deepest existing ancestor's direct subfolders
|
||||
listed, so the model self-corrects in the next round; a document
|
||||
already in context (seed or previously read) →
|
||||
:data:`ALREADY_IN_CONTEXT` (phase 72, task 05 gate iteration:
|
||||
the line names the correct action — answer from the text already
|
||||
@@ -164,25 +196,30 @@ needs none either (the tool ran) — the policy keys on the no-calls
|
||||
exit only. No model participates in detection or repair: the
|
||||
registry + the fixed retry policy are the whole guardrail.
|
||||
|
||||
The DB accessors (:func:`list_catalog`, :func:`list_source_names`,
|
||||
:func:`find_document`, :func:`all_documents`) and the
|
||||
:func:`grep_document` line matcher are module-level functions so unit
|
||||
tests can monkeypatch them without a database.
|
||||
The DB accessors — the drill-down ``ls`` (:func:`ls_top`,
|
||||
:func:`ls_folder`; the pure grouping :func:`group_folder_listing` and
|
||||
the pure renderers :func:`render_ls_top` /
|
||||
:func:`render_folder_listing` sit next to them),
|
||||
:func:`list_source_names`, :func:`find_document`,
|
||||
:func:`all_documents` — and the :func:`grep_document` line matcher are
|
||||
module-level functions so unit tests can monkeypatch them without a
|
||||
database.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from collections.abc import AsyncIterator, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings
|
||||
from app.models import Document
|
||||
from app.models import Document, FolderSummary
|
||||
from app.rag.folder_summaries import folder_of
|
||||
from app.rag.git_sources import effective_sources
|
||||
from app.rag.llm import (
|
||||
LLMClient,
|
||||
@@ -212,9 +249,17 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"function": {
|
||||
"name": "ls",
|
||||
"description": (
|
||||
"List the indexed documents as `source: X | path: Y | "
|
||||
"title: Z` lines. Call one tool at a time — wait for "
|
||||
"this result before your next call."
|
||||
"List the knowledge base as a tree, one level at a "
|
||||
"time. With no path: the synced sources — each with "
|
||||
"its document count and a summary of its contents. "
|
||||
"With a source name (no '/'): that source's top-level "
|
||||
"folders and files. With a `source/folder` path: that "
|
||||
"folder's subfolders and files. Folder lines carry a "
|
||||
"summary of what the folder contains. File lines are "
|
||||
"`source: X | path: Y | title: Z` — use the combined "
|
||||
"`source/path` with `read` and `grep`. Call one tool "
|
||||
"at a time — wait for this result before your next "
|
||||
"call."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -222,13 +267,11 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Source name to list one source's documents "
|
||||
"(e.g. 'homelab') — a source name, not a "
|
||||
"file or directory path; omit to list "
|
||||
"every document. This is the only tool "
|
||||
"whose `path` is a source name — for "
|
||||
"`read` and `grep` it must be a document's "
|
||||
"combined `source/path`."
|
||||
"Optional — a source name (e.g. "
|
||||
"'homelab') to list its top level, or a "
|
||||
"`source/folder` path to drill down "
|
||||
"(e.g. 'homelab/active'). Omit it to list "
|
||||
"every source."
|
||||
),
|
||||
}
|
||||
},
|
||||
@@ -338,21 +381,6 @@ UNKNOWN_TOOL = "Unknown tool."
|
||||
MISSING_READ_ARGS = "read requires a string argument 'path'."
|
||||
MISSING_SEARCH_ARGS = "grep requires a string argument 'pattern'."
|
||||
|
||||
#: Teaching refusal for a scoped ``ls`` whose stripped ``path``
|
||||
#: contains a ``/`` (phase 72): a source name is a directory basename
|
||||
#: and can never contain one, so the argument is a document path passed
|
||||
#: where a source name belongs (the 2026-09-03 incident's
|
||||
#: ``ls(path='app/rag/importer.py')``). One ``{path}`` field — the
|
||||
#: argument echoed; a fixed template states the correct contract
|
||||
#: instead of the terse pre-phase-72 line, so the harness-prior misuse
|
||||
#: self-corrects in one round.
|
||||
LS_PATH_NOT_A_SOURCE = (
|
||||
"'{path}' looks like a document path, not a source name. The "
|
||||
"'path' argument of ls filters by source name (e.g. 'homelab') — "
|
||||
"omit it to list every document, or read a document by its "
|
||||
"combined 'source/path' string."
|
||||
)
|
||||
|
||||
#: The no-source ``ls`` refusal with the teaching parenthetical
|
||||
#: appended (phase 72): used when a stripped scope has no ``/`` and
|
||||
#: matches no registered source (the incident's ``ls(path='.')``). The
|
||||
@@ -394,6 +422,29 @@ NO_DOCUMENT_DID_YOU_MEAN_MANY = (
|
||||
#: dropped).
|
||||
SUGGESTION_LIMIT = 3
|
||||
|
||||
#: The drill-down ``ls`` file-line cap (phase 94, task 03): a folder's
|
||||
#: own files list at most this many ``source: X | path: Y | title: Z``
|
||||
#: lines (path order), then one deterministic grep-pointer note — a
|
||||
#: 500-file folder costs the model 50 lines + the note, never 500.
|
||||
#: Pinned module constant (no env var — the phase-94 TODO asks for a
|
||||
#: shape change, not a knob; the constant lives next to
|
||||
#: :data:`SEARCH_MAX_MATCHES`).
|
||||
LS_MAX_FILE_LINES = 50
|
||||
|
||||
#: The NOT-A-FOLDER ``ls`` teaching refusal (phase 94, task 03):
|
||||
#: a ``source/…`` argument whose folder segment matches no indexed
|
||||
#: prefix (the ``00_phase.md`` existence rule — a folder exists iff
|
||||
#: some indexed path starts with ``folder + "/"``; a document's own
|
||||
#: path is never a folder). Phase-72 teaching style: one line, the
|
||||
#: argument echoed (``{arg}``), the DEEPEST existing ancestor's name
|
||||
#: (``{parent}`` — the source for a top-level miss, ``source/folder``
|
||||
#: for a nested one) and its direct subfolders (``{subfolders}`` —
|
||||
#: space-joined ``name/`` entries in path order, so the model
|
||||
#: self-corrects in the next round; ``none`` when the ancestor has no
|
||||
#: subfolders). Still a refusal: it counts in nothing and consumes a
|
||||
#: round (no silent argument normalization).
|
||||
NOT_A_FOLDER = "'{arg}' is not a folder — {parent} has: {subfolders}"
|
||||
|
||||
#: The harness-owned recovery line (phase 71, task 03) — folded into the
|
||||
#: ORIGINAL single system message of the one bounded recovery request
|
||||
#: (``system_prompt + "\n" + CORRECTION_INSTRUCTION``; provider-safe,
|
||||
@@ -507,20 +558,6 @@ def plain_form(pattern: str) -> str:
|
||||
return p.strip()
|
||||
|
||||
|
||||
def list_catalog(db: Session) -> list[tuple[str, str, str]]:
|
||||
"""Every indexed document as ``(source, path, title)``.
|
||||
|
||||
Ordered by ``(source, path)`` — the same order as ``GET /api/docs``.
|
||||
Module-level (not a method) so unit tests can monkeypatch it.
|
||||
"""
|
||||
rows = db.execute(
|
||||
select(Document.source, Document.path, Document.title).order_by(
|
||||
Document.source, Document.path
|
||||
)
|
||||
).all()
|
||||
return [(source, path, title) for source, path, title in rows]
|
||||
|
||||
|
||||
def list_source_names(db: Session) -> list[str]:
|
||||
"""Every registered source name, deduped, in registry order.
|
||||
|
||||
@@ -546,6 +583,286 @@ def list_source_names(db: Session) -> list[str]:
|
||||
return names
|
||||
|
||||
|
||||
#: The drill-down ``ls`` fetchers (phase 94, task 03) — module-level so
|
||||
#: unit tests can monkeypatch them without a database (the house style:
|
||||
#: :func:`ls_top` / :func:`ls_folder` compose them; the pure grouping
|
||||
#: :func:`group_folder_listing` and renderers
|
||||
#: :func:`render_ls_top` / :func:`render_folder_listing` sit below).
|
||||
|
||||
|
||||
def _source_document_counts(db: Session) -> list[tuple[str, int]]:
|
||||
"""``(source, document count)`` per indexed source (one grouped
|
||||
query — the top-level ``ls`` counts, phase 94 task 03)."""
|
||||
return [
|
||||
(source, count)
|
||||
for source, count in db.execute(
|
||||
select(Document.source, func.count()).group_by(Document.source)
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def _source_root_summaries(db: Session) -> list[tuple[str, str]]:
|
||||
"""The stored source-root summaries ``(source, summary)``
|
||||
(``folder_path = ""`` — the top-level ``ls`` indented lines, phase
|
||||
94 task 03; absent when never generated)."""
|
||||
return [
|
||||
(source, summary)
|
||||
for source, summary in db.execute(
|
||||
select(FolderSummary.source, FolderSummary.summary).where(
|
||||
FolderSummary.folder_path == ""
|
||||
)
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def _source_document_rows(db: Session, source: str) -> list[tuple[str, str]]:
|
||||
"""``(path, title)`` of every document under *source*, ordered by
|
||||
``path`` — the one bounded fetch a folder drill level lists (phase
|
||||
94 task 03; one source's paths, not the whole KB)."""
|
||||
return [
|
||||
(path, title)
|
||||
for path, title in db.execute(
|
||||
select(Document.path, Document.title)
|
||||
.where(Document.source == source)
|
||||
.order_by(Document.path)
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def _source_folder_summaries(db: Session, source: str) -> dict[str, str]:
|
||||
"""The stored folder summaries ``{folder_path: summary}`` of one
|
||||
source (phase 94 task 03; includes the ``""`` source-root row when
|
||||
stored)."""
|
||||
return {
|
||||
folder_path: summary
|
||||
for folder_path, summary in db.execute(
|
||||
select(FolderSummary.folder_path, FolderSummary.summary).where(
|
||||
FolderSummary.source == source
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
def ls_top(db: Session) -> list[tuple[str, int, str | None]]:
|
||||
"""The top level of the drill-down ``ls`` (phase 94, task 03).
|
||||
|
||||
Every registered source in :func:`list_source_names` order (the
|
||||
registry is the source of truth — a registered source with 0
|
||||
indexed documents still lists, the phase-70/72 invariant) as
|
||||
``(name, recursive_document_count, source_root_summary)``: the
|
||||
count is the source's whole subtree (all of its documents — the
|
||||
same set its stored summary describes) and the summary is the
|
||||
stored ``folder_summaries`` row for ``(source, "")`` (the source
|
||||
root, phase 94 task 01) or ``None`` when absent. Module-level so
|
||||
unit tests can monkeypatch the fetchers.
|
||||
"""
|
||||
names = list_source_names(db)
|
||||
if not names:
|
||||
return []
|
||||
counts = dict(_source_document_counts(db))
|
||||
summaries = dict(_source_root_summaries(db))
|
||||
return [(name, counts.get(name, 0), summaries.get(name)) for name in names]
|
||||
|
||||
|
||||
def group_folder_listing(
|
||||
source: str,
|
||||
folder: str,
|
||||
rows: Sequence[tuple[str, str]],
|
||||
summaries: Mapping[str, str],
|
||||
) -> tuple[list[tuple[str, int, str | None]], list[tuple[str, str, str]], int]:
|
||||
"""One level of the drill-down tree (phase 94, task 03) — pure.
|
||||
|
||||
Given *rows* — the source's ``(path, title)`` pairs in catalog
|
||||
(path) order — and *summaries* (the source's stored
|
||||
``folder_summaries`` rows: ``folder_path → summary``), the folder
|
||||
level *folder* (source-relative; ``""`` = the source root):
|
||||
|
||||
* **(a) direct subfolders** — the folders whose parent is exactly
|
||||
*folder*, in path order, each
|
||||
``(sub_path_relative_to_source, recursive_count,
|
||||
summary_or_None)``. A folder is a slash-boundary prefix of at
|
||||
least one indexed path (the ``00_phase.md`` existence rule: a
|
||||
folder ``F`` exists ⟺ some path starts with ``F + "/"`` — a
|
||||
document's OWN path is never a folder); the count is the
|
||||
folder's recursive subtree — every path equal to the folder or
|
||||
starting with ``folder + "/"`` (the same set the sync-time
|
||||
folder summary describes, phase 94 task 01 — one concept end to
|
||||
end).
|
||||
* **(b) direct file lines** — the documents whose folder (the
|
||||
prefix before the last ``/`` —
|
||||
:func:`app.rag.folder_summaries.folder_of`, the shared notion) IS
|
||||
*folder*, in path order (catalog order — the same order
|
||||
``GET /api/docs`` serves), as ``(source, path, title)`` triples
|
||||
— the canonical ``read``/``grep`` identity, capped at
|
||||
:data:`LS_MAX_FILE_LINES` (the rest fold into the renderer's
|
||||
note; a 500-file folder never costs 500 lines).
|
||||
* **(c) the TOTAL direct-file count** — pre-cap, for the note.
|
||||
|
||||
Pure (no I/O) — unit tests drive the grouping without a database;
|
||||
:func:`ls_folder` is the DB-composing wrapper.
|
||||
"""
|
||||
# The source's existing folders: every slash-boundary prefix of an
|
||||
# indexed path (the existence rule's candidate set — a folder is
|
||||
# present iff at least one path starts with ``folder + "/"``).
|
||||
folders: set[str] = set()
|
||||
for path, _title in rows:
|
||||
f = folder_of(path)
|
||||
while f:
|
||||
folders.add(f)
|
||||
f = folder_of(f)
|
||||
# The recursive count per folder — the ``path == folder`` arm (a
|
||||
# document sharing a folder's name) plus the ``startswith
|
||||
# folder + "/"`` arm (the folder's true descendants), one pass per
|
||||
# document.
|
||||
counts: dict[str, int] = {f: 0 for f in folders}
|
||||
for path, _title in rows:
|
||||
if path in folders:
|
||||
counts[path] += 1
|
||||
f = folder_of(path)
|
||||
while f:
|
||||
counts[f] += 1
|
||||
f = folder_of(f)
|
||||
subfolders = [
|
||||
(g, counts[g], summaries.get(g))
|
||||
for g in sorted(g for g in folders if folder_of(g) == folder)
|
||||
]
|
||||
files = [
|
||||
(source, path, title)
|
||||
for path, title in rows
|
||||
if folder_of(path) == folder
|
||||
]
|
||||
return subfolders, files[:LS_MAX_FILE_LINES], len(files)
|
||||
|
||||
|
||||
def ls_folder(
|
||||
db: Session, source: str, folder: str
|
||||
) -> tuple[list[tuple[str, int, str | None]], list[tuple[str, str, str]], int]:
|
||||
"""One folder level of the drill-down ``ls`` (phase 94, task 03).
|
||||
|
||||
The source's document rows (:func:`_source_document_rows`) and
|
||||
stored folder summaries (:func:`_source_folder_summaries`) through
|
||||
:func:`group_folder_listing` — the pure grouping the unit tests
|
||||
drive directly. ``folder = ""`` is the source root. Module-level
|
||||
so unit tests can monkeypatch the fetchers.
|
||||
"""
|
||||
return group_folder_listing(
|
||||
source,
|
||||
folder,
|
||||
_source_document_rows(db, source),
|
||||
_source_folder_summaries(db, source),
|
||||
)
|
||||
|
||||
|
||||
def _folder_exists_in(rows: Sequence[tuple[str, str]], folder: str) -> bool:
|
||||
"""The phase-94 folder-existence rule (``00_phase.md``), pure.
|
||||
|
||||
Folder *folder* (source-relative) under a registered source
|
||||
exists ⟺ ``folder == ""`` OR some indexed path starts with
|
||||
``folder + "/"`` — a document's OWN path is never a folder
|
||||
(nothing starts with ``path + "/"``), so ``ls`` of a file path
|
||||
refuses with :data:`NOT_A_FOLDER` rather than listing.
|
||||
"""
|
||||
if not folder:
|
||||
return True
|
||||
prefix = folder + "/"
|
||||
return any(path.startswith(prefix) for path, _title in rows)
|
||||
|
||||
|
||||
def _deepest_existing_ancestor(
|
||||
rows: Sequence[tuple[str, str]], folder: str
|
||||
) -> str:
|
||||
"""The deepest EXISTING folder prefix of a missing *folder* (pure).
|
||||
|
||||
The :data:`NOT_A_FOLDER` refusal's teaching context: the argument's
|
||||
segments are walked from the top; the walk stops at the first
|
||||
segment that is no folder, so the returned prefix is the deepest
|
||||
existing ancestor (``""`` = the source root when the first segment
|
||||
is already the miss) and its direct subfolders are the bounded
|
||||
self-correction list the refusal prints.
|
||||
"""
|
||||
parent = ""
|
||||
for part in folder.split("/"):
|
||||
candidate = f"{parent}/{part}" if parent else part
|
||||
if not _folder_exists_in(rows, candidate):
|
||||
break
|
||||
parent = candidate
|
||||
return parent
|
||||
|
||||
|
||||
def render_ls_top(entries: Sequence[tuple[str, int, str | None]]) -> str:
|
||||
"""The top-level ``ls`` result (phase 94, task 03) — the pinned
|
||||
template.
|
||||
|
||||
``{N} sources:`` — the line alone when the registry is empty (the
|
||||
old ``0 documents:`` behavior preserved in spirit) — then, when at
|
||||
least one source is registered, a blank line and one block per
|
||||
source in registry order: ``{source} — {n} documents`` plus the
|
||||
indented `` {summary}`` line ONLY when the source-root summary is
|
||||
stored (absent → the count line alone, no placeholder) — with NO
|
||||
blank line between blocks.
|
||||
"""
|
||||
lines = [f"{len(entries)} sources:"]
|
||||
if entries:
|
||||
lines.append("")
|
||||
for source, count, summary in entries:
|
||||
lines.append(f"{source} — {count} documents")
|
||||
if summary:
|
||||
lines.append(f" {summary}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_folder_listing(
|
||||
identity: str,
|
||||
subfolders: Sequence[tuple[str, int, str | None]],
|
||||
files: Sequence[tuple[str, str, str]],
|
||||
total_files: int,
|
||||
) -> str:
|
||||
"""One folder level of the drill-down ``ls`` (phase 94, task 03) —
|
||||
the pinned template.
|
||||
|
||||
The header ``{identity} — {n_files} documents, {n_folders}
|
||||
folders:`` (``n_files`` = *total_files*, the PRE-cap direct-file
|
||||
count; ``identity`` is the source name at the root level and
|
||||
``source/folder`` below it), then — when the level carries anything
|
||||
below the header — a blank line, the 2-space-indented subfolder
|
||||
lines `` {sub}/ — {m} documents`` in path order (``: {summary}``
|
||||
appended ONLY when the subfolder's summary is stored), a blank
|
||||
line, the file lines in EXACTLY the existing
|
||||
``source: X | path: Y | title: Z`` format (the canonical
|
||||
``read``/``grep`` identity — unchanged), and the cap note
|
||||
``…and {hidden} more documents in this folder — use grep
|
||||
(pattern) to find a specific one.`` ONLY when the folder's own
|
||||
files outnumber :data:`LS_MAX_FILE_LINES` (*files* arrives capped;
|
||||
*total_files* carries the pre-cap count). A header with no
|
||||
subfolders and no files — a registered source with no documents —
|
||||
is the header line alone (``… — 0 documents, 0 folders:``).
|
||||
"""
|
||||
header = f"{identity} — {total_files} documents, {len(subfolders)} folders:"
|
||||
if not subfolders and not files:
|
||||
return header
|
||||
body: list[str] = []
|
||||
if subfolders:
|
||||
body.append("")
|
||||
for sub, count, summary in subfolders:
|
||||
line = f" {sub}/ — {count} documents"
|
||||
if summary:
|
||||
line += f": {summary}"
|
||||
body.append(line)
|
||||
if files or total_files > len(files):
|
||||
body.append("")
|
||||
body.extend(
|
||||
f"source: {source} | path: {path} | title: {title}"
|
||||
for source, path, title in files
|
||||
)
|
||||
hidden = total_files - len(files)
|
||||
if hidden > 0:
|
||||
body.append(
|
||||
f"…and {hidden} more documents in this folder — use grep "
|
||||
"(pattern) to find a specific one."
|
||||
)
|
||||
return "\n".join([header, *body])
|
||||
|
||||
|
||||
def find_document(db: Session, source: str, path: str) -> Document | None:
|
||||
"""The indexed document at ``(source, path)``, or ``None``.
|
||||
|
||||
@@ -704,30 +1021,61 @@ def _execute_tool(
|
||||
phase 70).
|
||||
"""
|
||||
if call.name == "ls":
|
||||
# Phase 94 (task 03): the drill-down tree — one level per call
|
||||
# (the owner-permitted tool-surface revision; the old
|
||||
# whole-catalog listing is gone). A successful listing at ANY
|
||||
# level (top/root/folder) counts; a refusal counts in nothing
|
||||
# and consumes a round like every refusal.
|
||||
raw_path = call.arguments.get("path")
|
||||
scope = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
rows = list_catalog(db)
|
||||
if scope:
|
||||
if "/" in scope:
|
||||
# A source name (a directory basename) can never
|
||||
# contain '/' — this is a document path where a source
|
||||
# name belongs (phase 72): teach the contract; no
|
||||
# registry lookup needed, counts in nothing, consumes
|
||||
# a round like every refusal.
|
||||
return LS_PATH_NOT_A_SOURCE.format(path=scope)
|
||||
if scope not in list_source_names(db):
|
||||
# The no-source refusal with the teaching parenthetical
|
||||
# (phase 72) — the prefix byte-identical to the
|
||||
# pre-phase-72 line; counts in nothing, consumes a
|
||||
# round like every refusal.
|
||||
return NO_SOURCE_NOT_A_DIRECTORY.format(scope=scope)
|
||||
rows = [row for row in rows if row[0] == scope]
|
||||
listing = f"{len(rows)} documents:\n" + "\n".join(
|
||||
f"source: {source} | path: {path} | title: {title}"
|
||||
for source, path, title in rows
|
||||
)
|
||||
if not scope:
|
||||
# The top level: the synced sources, registry order, each
|
||||
# with its recursive document count and its stored
|
||||
# source-root summary (``None`` → no indented line).
|
||||
holder.tool_calls += 1
|
||||
return render_ls_top(ls_top(db))
|
||||
source, _, rest = scope.partition("/")
|
||||
if source not in list_source_names(db):
|
||||
# The no-source refusal with the teaching parenthetical
|
||||
# (phase 72, the prefix byte-identical to the pre-phase-72
|
||||
# line): the FIRST segment is the source candidate — a bare
|
||||
# unknown name (the incident's ``ls(path='.')``) or the
|
||||
# source segment of a ``source/…`` argument (phase 94: a
|
||||
# ``/`` now names a folder, so the phase-72 document-path
|
||||
# teaching is deleted) — the segment echoed; counts in
|
||||
# nothing, consumes a round like every refusal.
|
||||
return NO_SOURCE_NOT_A_DIRECTORY.format(scope=source)
|
||||
if not rest:
|
||||
# The source's ROOT folder: subfolders + own file lines
|
||||
# (capped + note) — the pinned template; a registered
|
||||
# source with no documents lists its header line alone
|
||||
# (``… — 0 documents, 0 folders:`` — the old
|
||||
# ``0 documents:`` behavior preserved in spirit).
|
||||
subfolders, files, total = ls_folder(db, source, "")
|
||||
holder.tool_calls += 1
|
||||
return render_folder_listing(source, subfolders, files, total)
|
||||
rows = _source_document_rows(db, source)
|
||||
summaries = _source_folder_summaries(db, source)
|
||||
if not _folder_exists_in(rows, rest):
|
||||
# The NOT-A-FOLDER teaching (phase 94, task 03): the
|
||||
# argument echoed, the DEEPEST existing ancestor's name and
|
||||
# its direct subfolders listed (bounded — the parent's own
|
||||
# listing, so no new flood path), so the model
|
||||
# self-corrects in the next round; counts in nothing,
|
||||
# consumes a round like every refusal.
|
||||
parent = _deepest_existing_ancestor(rows, rest)
|
||||
parent_subs = group_folder_listing(source, parent, rows, summaries)[0]
|
||||
return NOT_A_FOLDER.format(
|
||||
arg=scope,
|
||||
parent=source if not parent else f"{source}/{parent}",
|
||||
subfolders=" ".join(f"{sub}/" for sub, _c, _s in parent_subs)
|
||||
or "none",
|
||||
)
|
||||
# The folder level: the same template as the root, identity =
|
||||
# source + "/" + folder.
|
||||
subfolders, files, total = group_folder_listing(source, rest, rows, summaries)
|
||||
holder.tool_calls += 1
|
||||
return listing
|
||||
return render_folder_listing(f"{source}/{rest}", subfolders, files, total)
|
||||
if call.name == "read":
|
||||
raw_path = call.arguments.get("path")
|
||||
arg = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
|
||||
Reference in New Issue
Block a user