phase: 94_ls_tree_drilldown
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 25s

All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
+218 -38
View File
@@ -1,19 +1,28 @@
"""Integration: the agent DB accessors against real Postgres (phase 37;
the harness-aligned ``ls``/``read``/``grep`` surface, phase 70).
the harness-aligned ``ls``/``read``/``grep`` surface, phase 70; the
drill-down tree ``ls``, phase 94).
``list_catalog`` must order rows by ``(source, path)`` — the same order
as ``GET /api/docs`` — ``list_source_names`` must resolve the
registered source names (the scoped ``ls`` join), and ``find_document``
``_source_document_rows`` must order a source's rows by ``path`` (the
file lines' catalog order), ``list_source_names`` must resolve the
registered source names (the registry join), and ``find_document``
must resolve a hit to the full document row (content included, for the
never-truncated read) and return ``None`` for unknown pairs. Phase 70:
the ``ls``/``read``/``grep`` tools are pinned here too — the locked
parameter shape in ``AGENT_TOOLS``, and scripted ``ToolCallPiece``s
executed through ``run_agent`` against the real DB: ``ls`` scoped to a
registered source name (unknown name → refusal), ``read`` on the
canonical combined ``source/path`` form (first-slash split; a bare
source name and an unknown identity get the no-document refusal), and
``grep`` (``all_documents`` for a whole-KB search, ``find_document`` for
a scoped one).
executed through ``run_agent`` against the real DB. Phase 94: the
drill-down ``ls`` against the REAL tables — ``ls()`` lists the
registered sources (registry order, recursive counts, stored
source-root summaries from ``folder_summaries``), ``ls(source)`` /
``ls(source/folder)`` list one folder level (the SQL prefix logic:
subfolders = slash-boundary prefixes, counts = the recursive subtree,
file lines in catalog order, capped at 50 + the grep-pointer note),
and the refusals (unknown source segment → the no-source refusal; an
unknown folder → NOT-A_FOLDER with the parent's subfolders).
``read`` runs on the canonical combined ``source/path`` form
(first-slash split; a bare source name and an unknown identity get the
no-document refusal), and ``grep`` (``all_documents`` for a whole-KB
search, ``find_document`` for a scoped one) — both byte-identical
across the phase-94 change.
Requires: podman compose up -d db
"""
@@ -30,7 +39,7 @@ from sqlalchemy import delete, text
from sqlalchemy.orm import Session
from app.config import Settings
from app.models import Document, GitSource
from app.models import Document, FolderSummary, GitSource
from app.rag import agent
from app.rag.agent import AGENT_TOOLS, AgentHolder, run_agent
from app.rag.llm import LLMClient, RetryPiece, StreamPiece, ToolCallPiece
@@ -55,11 +64,30 @@ def _doc(db: Session, source: str, path: str, title: str, content: str) -> Docum
@pytest.fixture()
def kb(db) -> Iterator[None]:
"""Fresh documents table (chunks first — the FK) for these accessors."""
db.execute(text("TRUNCATE chunks, documents"))
"""Fresh documents + folder_summaries tables (chunks first — the FK)
for these accessors (phase 94: the drill-down ``ls`` reads the
stored summaries too)."""
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
db.commit()
yield
db.execute(text("TRUNCATE chunks, documents"))
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
db.commit()
@pytest.fixture()
def registry(db) -> Iterator[None]:
"""A FRESH two-source registry (phase 94): the drill-down ``ls``
top level IS the registry, so the table is truncated and re-seeded
around the tests in a controlled ``(added_at, id)`` order —
``Deployments`` before ``Homelab`` (the top-level listing order)."""
db.execute(text("TRUNCATE git_sources"))
db.commit()
db.add(GitSource(url="https://github.com/reese/Deployments.git", kind="git"))
db.commit()
db.add(GitSource(url="https://github.com/reese/Homelab.git", kind="git"))
db.commit()
yield
db.execute(text("TRUNCATE git_sources"))
db.commit()
@@ -76,21 +104,24 @@ def src(db) -> Iterator[GitSource]:
db.commit()
def test_list_catalog_orders_by_source_then_path(kb, db) -> None:
def test_source_document_rows_order_by_path_within_the_source(kb, db) -> None:
"""Phase 94: the file lines' order — the source's rows in ``path``
order (the old ``list_catalog``'s per-source ordering, now the
``ls`` folder-level accessor's contract; a different source's rows
never leak in)."""
_doc(db, "Zeta", "b/second.md", "Zeta B", "ZB")
_doc(db, "Zeta", "a/first.md", "Zeta A", "ZA")
_doc(db, "Alpha", "c/third.md", "Alpha C", "AC")
db.commit()
assert agent.list_catalog(db) == [
("Alpha", "c/third.md", "Alpha C"),
("Zeta", "a/first.md", "Zeta A"),
("Zeta", "b/second.md", "Zeta B"),
assert agent._source_document_rows(db, "Zeta") == [
("a/first.md", "Zeta A"),
("b/second.md", "Zeta B"),
]
def test_list_catalog_is_empty_without_rows(kb, db) -> None:
assert agent.list_catalog(db) == []
def test_source_document_rows_is_empty_without_rows(kb, db) -> None:
assert agent._source_document_rows(db, "Zeta") == []
def test_list_source_names_resolves_registry_rows(db) -> None:
@@ -238,30 +269,156 @@ async def _consume(
return out
# ---------- ls (scoped through the real registry) ----------
# ---------- ls (the drill-down tree, phase 94 — the real registry + DB) ----------
def test_ls_scoped_to_registered_source_through_run_agent(kb, src, db) -> None:
_doc(db, "Homelab", "a.md", "A", "A-CONTENT")
_doc(db, "Other", "b.md", "B", "B-CONTENT")
def test_ls_top_level_lists_registered_sources_through_run_agent(
kb, registry, db
) -> None:
"""No path: the TOP level against the real tables — registry order
(``(added_at, id)`` — Deployments before Homelab), recursive counts
(all of a source's documents), the stored source-root summary
(``folder_path = ''``) shown only when stored."""
_doc(db, "Deployments", "a/one.md", "A1", "A1-CONTENT")
_doc(db, "Homelab", "x.md", "X", "X-CONTENT")
_doc(db, "Homelab", "y/z.md", "Z", "Z-CONTENT")
db.add(FolderSummary(source="Homelab", folder_path="", summary="Home lab notes."))
db.commit()
holder, llm = _run_call(db, "ls", {"path": "Homelab"})
holder, llm = _run_call(db, "ls", {})
# Offered: the first request carries AGENT_TOOLS (the 3-tool list).
assert llm.requests[0][1] == AGENT_TOOLS
# Executed against the real DB: the listing filtered to the source.
assert llm.requests[1][0][3]["content"] == (
"1 documents:\nsource: Homelab | path: a.md | title: A"
"2 sources:\n"
"\n"
"Deployments — 1 documents\n"
"Homelab — 2 documents\n"
" Home lab notes."
)
assert holder.tool_calls == 1
assert holder.read_docs == []
def test_ls_top_level_empty_registry_through_run_agent(
kb, db, monkeypatch: pytest.MonkeyPatch
) -> None:
"""No registered sources: the top level is the header line alone
(``0 sources:`` — the old ``0 documents:`` behavior preserved in
spirit), still counted. The env fallback (``BOR_GIT_SOURCES`` — the
operator's ``.env`` may name sources) is emptied for the test, so
the registry is genuinely empty."""
import app.rag.git_sources as git_sources_mod
db.execute(text("TRUNCATE git_sources"))
db.commit()
monkeypatch.setattr(
git_sources_mod,
"get_settings",
lambda: _settings(git_sources=""),
)
_doc(db, "Orphan", "a.md", "A", "A-CONTENT") # indexed but unregistered
db.commit()
holder, llm = _run_call(db, "ls", {})
assert llm.requests[1][0][3]["content"] == "0 sources:"
assert holder.tool_calls == 1
def test_ls_source_scope_lists_root_folder_through_run_agent(kb, registry, db) -> None:
"""A registered source name: the source's ROOT folder — the direct
subfolders (path order, recursive counts, stored summaries attached)
+ the root's own file lines in catalog order — against the real
tables; a registered source with no documents lists its header
line alone."""
_doc(db, "Homelab", "backups/cron.md", "Cron", "CRON")
_doc(db, "Homelab", "backups/restic.md", "Restic", "RESTIC")
_doc(db, "Homelab", "networking/lan.md", "LAN", "LAN")
_doc(db, "Homelab", "readme.md", "Readme", "README")
db.add(
FolderSummary(
source="Homelab", folder_path="backups", summary="Backup notes."
)
)
db.commit()
holder, llm = _run_call(db, "ls", {"path": "Homelab"})
assert llm.requests[1][0][3]["content"] == (
"Homelab — 1 documents, 2 folders:\n"
"\n"
" backups/ — 2 documents: Backup notes.\n"
" networking/ — 1 documents\n"
"\n"
"source: Homelab | path: readme.md | title: Readme"
)
assert holder.tool_calls == 1
assert holder.read_docs == []
# A registered source with no documents: the header line alone.
holder0, llm0 = _run_call(db, "ls", {"path": "Deployments"})
assert llm0.requests[1][0][3]["content"] == "Deployments — 0 documents, 0 folders:"
assert holder0.tool_calls == 1
def test_ls_nested_folder_scope_drills_one_level_through_run_agent(
kb, registry, db
) -> None:
"""A ``source/folder`` path: that folder's subfolders + own file
lines (identity = ``source/folder``) — the drill-down against the
real tables."""
_doc(db, "Homelab", "networking/lan/a.md", "A", "A")
_doc(db, "Homelab", "networking/lan/b.md", "B", "B")
_doc(db, "Homelab", "networking/vpn/c.md", "C", "C")
db.commit()
holder, llm = _run_call(db, "ls", {"path": "Homelab/networking"})
assert llm.requests[1][0][3]["content"] == (
"Homelab/networking — 0 documents, 2 folders:\n"
"\n"
" networking/lan/ — 2 documents\n"
" networking/vpn/ — 1 documents"
)
assert holder.tool_calls == 1
assert holder.read_docs == []
# One level deeper.
holder2, llm2 = _run_call(db, "ls", {"path": "Homelab/networking/lan"})
assert llm2.requests[1][0][3]["content"] == (
"Homelab/networking/lan — 2 documents, 0 folders:\n"
"\n"
"source: Homelab | path: networking/lan/a.md | title: A\n"
"source: Homelab | path: networking/lan/b.md | title: B"
)
assert holder2.tool_calls == 1
def test_ls_folder_file_cap_through_run_agent(kb, registry, db) -> None:
"""The cap end-to-end: 51 direct files in one folder cost 50 file
lines + the deterministic grep-pointer note, never 51."""
for i in range(51):
_doc(db, "Homelab", f"big/f{i:03d}.md", f"T{i}", "BODY")
db.commit()
holder, llm = _run_call(db, "ls", {"path": "Homelab/big"})
content = llm.requests[1][0][3]["content"]
lines = content.splitlines()
assert lines[0] == "Homelab/big — 51 documents, 0 folders:"
assert lines[2] == "source: Homelab | path: big/f000.md | title: T0"
assert lines[51] == "source: Homelab | path: big/f049.md | title: T49"
assert lines[52] == (
"…and 1 more documents in this folder — use grep (pattern) to "
"find a specific one."
)
assert len(lines) == 53
assert holder.tool_calls == 1
def test_ls_scoped_unknown_source_refused_through_run_agent(kb, src, db) -> None:
"""Phase 72: the no-source refusal now carries the teaching
parenthetical — the prefix byte-identical to the pre-phase-72 line;
still not counted."""
"""A ``path`` without ``/`` matching no source name is a refusal —
the extended line with the teaching parenthetical (phase 72, the
prefix byte-identical to the pre-phase-72 line); still not counted."""
_doc(db, "Homelab", "a.md", "A", "A-CONTENT")
db.commit()
@@ -274,11 +431,11 @@ def test_ls_scoped_unknown_source_refused_through_run_agent(kb, src, db) -> None
assert holder.tool_calls == 0 and holder.read_docs == []
def test_ls_path_like_scope_teaching_refusal_through_run_agent(kb, src, db) -> None:
"""Phase 72: a ``/``-containing ``path`` is a document path, not a
source name — the ``LS_PATH_NOT_A_SOURCE`` teaching line (no
registry lookup needed), not counted, the tools stay offered on the
next request."""
def test_ls_path_like_scope_unknown_source_gets_no_source_refusal(kb, src, db) -> None:
"""Phase 94: a ``/`` now names a folder — the phase-72 document-path
teaching is DELETED; a ``source/…`` argument whose FIRST segment
names no registered source gets the no-source refusal (the segment
echoed), not counted, the tools stay offered."""
_doc(db, "Homelab", "a.md", "A", "A-CONTENT")
db.commit()
@@ -286,7 +443,30 @@ def test_ls_path_like_scope_teaching_refusal_through_run_agent(kb, src, db) -> N
assert (
llm.requests[1][0][3]["content"]
== agent.LS_PATH_NOT_A_SOURCE.format(path="app/rag/importer.py")
== agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app")
)
assert holder.tool_calls == 0 and holder.read_docs == []
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
def test_ls_unknown_folder_gets_not_a_folder_with_parents_subfolders(
kb, src, db,
) -> None:
"""Phase 94: a folder segment matching no indexed prefix gets the
NOT-A_FOLDER teaching — the argument echoed, the source named, its
DIRECT subfolders listed (the self-correction list), not counted,
the tools stay offered."""
_doc(db, "Homelab", "backups/cron.md", "Cron", "CRON")
_doc(db, "Homelab", "containers/caddy.md", "Caddy", "CADDY")
_doc(db, "Homelab", "networking/lan.md", "LAN", "LAN")
_doc(db, "Homelab", "readme.md", "Readme", "README")
db.commit()
holder, llm = _run_call(db, "ls", {"path": "Homelab/netwoking"})
assert llm.requests[1][0][3]["content"] == (
"'Homelab/netwoking' is not a folder — Homelab has: "
"backups/ containers/ networking/"
)
assert holder.tool_calls == 0 and holder.read_docs == []
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
+4 -1
View File
@@ -943,7 +943,10 @@ def test_tool_execution_db_failure_yields_error_event(
def boom(*_a: Any, **_k: Any) -> Any:
raise RuntimeError("db exploded mid tool call")
monkeypatch.setattr(agent, "list_catalog", boom)
# Phase 94: the no-arg ``ls`` executes through ``ls_top`` — the
# failure hook moves with the rewrite (same contract: the tool
# frame goes out first, the structured error ends the turn).
monkeypatch.setattr(agent, "ls_top", boom)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
+18
View File
@@ -104,6 +104,21 @@ def _stub_bump(monkeypatch: pytest.MonkeyPatch) -> list[None]:
return bumps
def _stub_folder_summaries(monkeypatch: pytest.MonkeyPatch) -> list[dict]:
"""Stub the phase-94 folder-summary regeneration (this file keeps
its no-real-DB / no-network style — the real generator would read
the global ``documents`` table and burn ``lite`` calls). Returns
the call record; the canned stats are the zero dict."""
calls: list[dict] = []
async def fake_generate(db: object, llm: object, *, skip: bool = False) -> dict[str, int]:
calls.append({"skip": skip})
return {"generated": 0, "failed": 0, "pruned": 0}
monkeypatch.setattr(import_docs, "generate_folder_summaries", fake_generate)
return calls
# --- repo_name -------------------------------------------------------------
@@ -293,6 +308,7 @@ def test_main_rows_branch_passes_ignore_map_to_import(
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
_stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
@@ -329,6 +345,7 @@ def test_main_git_sources_clone_then_import(
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
bumps = _stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
@@ -371,6 +388,7 @@ def test_main_cli_source_still_imports_manual_dir(
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
bumps = _stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main(["--source", str(manual)])
+61 -20
View File
@@ -23,6 +23,15 @@ and unchanged re-runs never bump (``sources_version=skipped``), and a
failed ``lite`` never rolls the bump back. The counter is pinned to the
migration-0010 seed (0) around every test by
:func:`_reset_sources_version`.
Phase 94 (task 02, line-extension house rule): the summary line now
ends with the folder-summary stats —
``folder_summaries=<generated>/<failed>/<pruned>`` when the gate fired
(this fixture's 2-doc source holds exactly ONE qualifying subtree: the
source root) or ``folder_summaries=skipped`` otherwise — so the line
pinned here gains that token, and a KB-changing run burns exactly ONE
extra ``lite`` call (the source-root folder summary, markdown files
never get a document summary).
"""
from __future__ import annotations
@@ -98,10 +107,10 @@ def src(tmp_path: Path) -> Path:
@pytest.fixture(autouse=True)
def _clean_kb(db: Session) -> Iterator[None]:
db.execute(text("TRUNCATE chunks, documents, kb_overview"))
db.execute(text("TRUNCATE chunks, documents, kb_overview, folder_summaries"))
db.commit()
yield
db.execute(text("TRUNCATE chunks, documents, kb_overview"))
db.execute(text("TRUNCATE chunks, documents, kb_overview, folder_summaries"))
db.commit()
@@ -149,16 +158,24 @@ def test_changed_import_writes_overview_row(
assert rc == 0
assert "added=2" in out
assert out.rstrip().endswith("overview=updated sources_version=1")
# Phase 94: the line gains the folder-stats token — the 2-doc source
# holds one qualifying subtree (the source root): 1 generated.
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=1/0/0"
)
assert _version(db) == 1 # phase 53: a changed import bumps exactly once
# Exactly one lite call — the overview itself (markdown files never
# get a summary, so nothing else may touch ``chat``).
assert len(llm.chat_calls) == 1
# Exactly two lite calls — the overview + the source-root folder
# summary (markdown files never get a document summary, so nothing
# else may touch ``chat``).
assert len(llm.chat_calls) == 2
by_role = {m["role"]: m["content"] for m in llm.chat_calls[0]}
assert "KB_OVERVIEW_MODE" in by_role["system"]
# One line per doc: source — path — title (no summary for markdown).
assert "MyDocs — alpha.md — Alpha" in by_role["user"]
assert "MyDocs — beta.md — Beta" in by_role["user"]
by_role = {m["role"]: m["content"] for m in llm.chat_calls[1]}
assert "FOLDER_SUMMARY_MODE" in by_role["system"]
assert by_role["user"].splitlines()[0] == "Folder: MyDocs"
# The model's outline lands in the single row.
row = _row(db)
assert row is not None
@@ -177,16 +194,20 @@ def test_unchanged_reimport_does_not_call_lite(
llm = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm, ["--source", str(src)], capsys)
assert rc == 0
assert out.rstrip().endswith("overview=updated sources_version=1")
assert len(llm.chat_calls) == 1
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=1/0/0"
)
assert len(llm.chat_calls) == 2 # overview + source-root folder summary
assert _row(db) is not None
# Same hashes → no KB change → no lite call, previous outline kept.
rc, out = _run_main(monkeypatch, llm, ["--source", str(src)], capsys)
assert rc == 0
assert "unchanged=2" in out
assert out.rstrip().endswith("overview=skipped sources_version=skipped")
assert len(llm.chat_calls) == 1 # no new lite call
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert len(llm.chat_calls) == 2 # no new lite call
row = _row(db)
assert row is not None and row.content == "Summary of MyDocs"
assert _version(db) == 1 # phase 53: an unchanged re-run never bumps
@@ -201,7 +222,9 @@ def test_lite_failure_is_fail_soft(
good = FakeEmbedder()
rc, out = _run_main(monkeypatch, good, ["--source", str(src)], capsys)
assert rc == 0
assert out.rstrip().endswith("overview=updated sources_version=1")
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=1/0/0"
)
previous = _row(db)
assert previous is not None
previous_content = previous.content
@@ -213,8 +236,12 @@ def test_lite_failure_is_fail_soft(
rc, out = _run_main(monkeypatch, bad, ["--source", str(src)], capsys)
assert rc == 0 # a failed outline must not fail the import
assert "updated=1" in out
assert out.rstrip().endswith("overview=failed sources_version=2")
assert len(bad.chat_calls) == 1 # the (failed) attempt was made
# Phase 94: the folder batch fails too (per-folder fail-soft) — the
# failed attempt counts into the stats, the previous row stays.
assert out.rstrip().endswith(
"overview=failed sources_version=2 folder_summaries=0/1/0"
)
assert len(bad.chat_calls) == 2 # the (failed) attempts were made
row = _row(db)
assert row is not None
assert row.content == previous_content # previous row untouched
@@ -232,8 +259,10 @@ def test_limit_run_skips_overview(
llm = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm, ["--source", str(src)], capsys)
assert rc == 0
assert out.rstrip().endswith("overview=updated sources_version=1")
assert len(llm.chat_calls) == 1
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=1/0/0"
)
assert len(llm.chat_calls) == 2
# An incomplete walk must not rewrite the outline (mirrors the
# --prune-with---limit guard) — and must not advance the version.
@@ -241,8 +270,10 @@ def test_limit_run_skips_overview(
rc, out = _run_main(monkeypatch, llm, ["--source", str(src), "--limit", "1"], capsys)
assert rc == 0
assert "updated=1" in out
assert out.rstrip().endswith("overview=skipped sources_version=skipped")
assert len(llm.chat_calls) == 1 # --limit never burns a lite call
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert len(llm.chat_calls) == 2 # --limit never burns a lite call
row = _row(db)
assert row is not None and row.content == "Summary of MyDocs"
assert _version(db) == 1 # phase 53: --limit debug runs never bump
@@ -262,7 +293,9 @@ def test_empty_source_without_row_creates_nothing(
assert rc == 0
assert "files=0" in out
assert out.rstrip().endswith("overview=skipped sources_version=skipped")
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert llm.chat_calls == [] # no KB → no outline, no wasted model call
assert _row(db) is None # nothing created
assert _version(db) == 0 # nothing changed → nothing bumped
@@ -283,7 +316,9 @@ def test_prune_only_run_bumps_sources_version(
rc, out = _run_main(monkeypatch, llm, ["--source", str(src)], capsys)
assert rc == 0
assert "added=2" in out
assert out.rstrip().endswith("overview=updated sources_version=1")
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=1/0/0"
)
assert _version(db) == 1
# Delete one file; a --prune run drops exactly it: no add/update,
@@ -292,6 +327,12 @@ def test_prune_only_run_bumps_sources_version(
rc, out = _run_main(monkeypatch, llm, ["--source", str(src), "--prune"], capsys)
assert rc == 0
assert "pruned=1" in out
assert out.rstrip().endswith("overview=skipped sources_version=2")
# Phase 94: the folder gate is the overview's (added + updated > 0
# or empty table) — a prune-only re-walk with a populated table
# skips generation (the remaining 1-doc source stays summarized by
# its existing root row, which still describes it).
assert out.rstrip().endswith(
"overview=skipped sources_version=2 folder_summaries=skipped"
)
assert _version(db) == 2 # the prune-only change bumped exactly once
assert _row(db) is not None # the outline row is untouched
+248
View File
@@ -0,0 +1,248 @@
"""Integration: migration 0017 (folder_summaries) schema contract
(phase 94, task 01).
Drives the **real Alembic engine** against the live dev database
(``podman compose up -d db``), mirroring the house pattern of
``test_migration_0016.py`` (information_schema assertions on the state
the migration must leave). The tests target the 0016 → 0017 step
explicitly so later migrations cannot break them:
* upgrade 0016 → 0017 → the ``folder_summaries`` table exists with the
full column contract — PK ``(source, folder_path)`` (VARCHAR(120) /
VARCHAR(1000) NOT NULL, mirroring ``documents.source`` /
``documents.path``), ``summary`` TEXT NOT NULL, ``updated_at``
TIMESTAMPTZ NOT NULL with the now() server default (house style) —
while the 0016 ``ui_settings`` schema survives;
* inserted rows round-trip their values (a source-root row with
``folder_path = ''`` and a nested-folder row);
* downgrade to 0016 → the table is GONE (A13 — reversible), the rest of
the schema (``ui_settings`` + ``api_tokens``) survives;
* upgrade back to 0017 → the table is back (round-trip).
The ``alembic`` fixture guarantees the DB ends at head even if a test
fails or the process is interrupted.
"""
from __future__ import annotations
from collections.abc import Iterator
from typing import Any
import pytest
from alembic.config import Config
from sqlalchemy import text
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from alembic import command
from app.db import db_available
@pytest.fixture()
def alembic(db: Session) -> Iterator[Config]:
"""Real Alembic config bound to the dev DB (URL from app settings).
Starts at head (repairs an interrupted earlier run); teardown upgrades
to head no matter what happened, so the dev DB is never left below
head.
"""
if not db_available():
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
cfg.set_main_option("script_location", "alembic")
command.upgrade(cfg, "head")
try:
yield cfg
finally:
command.upgrade(cfg, "head")
def _version(db: Session) -> str | None:
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
def _table_exists(db: Session, table: str) -> bool:
return (
db.execute(
text("SELECT 1 FROM information_schema.tables WHERE table_name = :t"),
{"t": table},
).scalar()
is not None
)
def _column(db: Session, table: str, column: str) -> tuple[Any, ...] | None:
"""(data_type, is_nullable, column_default, character_maximum_length)
for one table column."""
row = db.execute(
text(
"SELECT data_type, is_nullable, column_default, character_maximum_length"
" FROM information_schema.columns"
" WHERE table_name = :t AND column_name = :c"
),
{"t": table, "c": column},
).fetchone()
return tuple(row) if row is not None else None
def _pk_columns(db: Session, table: str) -> list[str]:
"""The table's PRIMARY KEY columns in ordinal position."""
rows = db.execute(
text(
"SELECT kcu.column_name"
" FROM information_schema.table_constraints tc"
" JOIN information_schema.key_column_usage kcu"
" ON kcu.constraint_name = tc.constraint_name"
" AND kcu.table_name = tc.table_name"
" WHERE tc.table_name = :t"
" AND tc.constraint_type = 'PRIMARY KEY'"
" ORDER BY kcu.ordinal_position"
),
{"t": table},
).fetchall()
return [r[0] for r in rows]
def _clear_rows(db: Session) -> None:
db.execute(text("DELETE FROM folder_summaries"))
db.commit()
def test_upgrade_to_0017_creates_folder_summaries(db: Session, alembic: Config) -> None:
"""Upgrade 0016 → 0017: the table exists with the full column
contract (PK ``(source, folder_path)`` mirroring
``documents.source`` / ``documents.path``; TEXT summary NOT NULL;
TIMESTAMPTZ updated_at NOT NULL with the now() server default), and
the table is ABSENT at 0016 while the 0016 ``ui_settings`` schema
survives the upgrade."""
command.downgrade(alembic, "0016") # start from the pre-0017 state
assert _version(db) == "0016"
assert not _table_exists(db, "folder_summaries"), (
"folder_summaries must be absent at 0016"
)
command.upgrade(alembic, "0017")
assert _version(db) == "0017", "alembic_version must be at 0017"
assert _table_exists(db, "folder_summaries"), "the table must exist at 0017"
source = _column(db, "folder_summaries", "source")
assert source is not None, "folder_summaries.source is missing"
assert source[0] == "character varying", "source must be VARCHAR"
assert source[1] == "NO", "source must be NOT NULL (PK part 1)"
assert source[2] is None, "source must have no server default"
assert source[3] == 120, "source must be String(120) — documents.source"
folder = _column(db, "folder_summaries", "folder_path")
assert folder is not None, "folder_summaries.folder_path is missing"
assert folder[0] == "character varying", "folder_path must be VARCHAR"
assert folder[1] == "NO", "folder_path must be NOT NULL (PK part 2)"
assert folder[2] is None, "folder_path must have no server default"
assert folder[3] == 1000, "folder_path must be String(1000) — documents.path"
summary = _column(db, "folder_summaries", "summary")
assert summary is not None, "folder_summaries.summary is missing"
assert summary[0] == "text", "summary must be TEXT"
assert summary[1] == "NO", "summary must be NOT NULL (never stored empty)"
updated = _column(db, "folder_summaries", "updated_at")
assert updated is not None, "folder_summaries.updated_at is missing"
assert updated[0] == "timestamp with time zone", "updated_at must be TIMESTAMPTZ"
assert updated[1] == "NO", "updated_at must be NOT NULL"
assert updated[2] is not None and "now" in str(updated[2]), (
"updated_at must carry the now() server default (house style)"
)
assert _pk_columns(db, "folder_summaries") == ["source", "folder_path"]
# The 0016 schema survives the additive upgrade.
semantic = _column(db, "ui_settings", "ok_bg")
assert semantic is not None and semantic[3] == 7, (
"ui_settings.ok_bg (0016) must survive the upgrade"
)
def test_inserted_rows_round_trip(db: Session, alembic: Config) -> None:
"""At 0017, a source-root row (``folder_path = ''``) and a nested
folder row round-trip their values, and the composite PK rejects a
duplicate (source, folder_path) pair."""
command.upgrade(alembic, "head")
try:
db.execute(
text(
"INSERT INTO folder_summaries (source, folder_path, summary)"
" VALUES ('Homelab', '', 'Root summary.')"
)
)
db.execute(
text(
"INSERT INTO folder_summaries (source, folder_path, summary)"
" VALUES ('Homelab', 'deployments/ansible', 'Ansible summary.')"
)
)
db.commit()
rows = db.execute(
text(
"SELECT source, folder_path, summary, updated_at"
" FROM folder_summaries ORDER BY folder_path"
)
).fetchall()
assert len(rows) == 2, "both rows must be stored"
assert rows[0][0] == "Homelab" and rows[0][1] == "", (
"the source-root row uses folder_path = ''"
)
assert rows[0][2] == "Root summary.", "summary must round-trip verbatim"
assert rows[0][3] is not None, "updated_at must be stamped (server default)"
assert rows[1][1] == "deployments/ansible", (
"a nested folder path must round-trip verbatim"
)
assert rows[1][2] == "Ansible summary."
with pytest.raises(IntegrityError):
db.execute(
text(
"INSERT INTO folder_summaries (source, folder_path, summary)"
" VALUES ('Homelab', 'deployments/ansible', 'dup')"
)
)
db.rollback() # the IntegrityError aborts the open transaction
finally:
_clear_rows(db)
def test_downgrade_to_0016_drops_the_table(db: Session, alembic: Config) -> None:
"""Downgrade 0017 → 0016: the table is gone (A13 — fully
reversible) while the rest of the schema survives (the 0016
``ui_settings`` semantic columns, ``api_tokens``, ``documents``)."""
command.downgrade(alembic, "0016")
assert _version(db) == "0016"
assert not _table_exists(db, "folder_summaries"), (
"folder_summaries must be dropped"
)
semantic = _column(db, "ui_settings", "accent_line")
assert semantic is not None and semantic[3] == 7, (
"ui_settings.accent_line (0016) must survive the downgrade"
)
token_col = _column(db, "api_tokens", "token_hash")
assert token_col is not None and token_col[0] == "character varying", (
"api_tokens.token_hash must survive the downgrade"
)
doc_path = _column(db, "documents", "path")
assert doc_path is not None and doc_path[3] == 1000, (
"documents.path must survive the downgrade"
)
def test_upgrade_round_trip_restores_the_table(db: Session, alembic: Config) -> None:
"""Downgrade to 0016, then upgrade back to 0017: the table is back
with the column contract and PK intact."""
command.downgrade(alembic, "0016")
command.upgrade(alembic, "0017")
assert _version(db) == "0017", "round-trip upgrade must land at 0017"
assert _table_exists(db, "folder_summaries"), "the table must be back"
folder = _column(db, "folder_summaries", "folder_path")
assert folder is not None, "folder_summaries.folder_path must be back"
assert folder[0] == "character varying", "folder_path must be VARCHAR"
assert folder[1] == "NO", "folder_path must be NOT NULL after the round-trip"
assert folder[3] == 1000, "folder_path must be String(1000) after the round-trip"
assert _pk_columns(db, "folder_summaries") == ["source", "folder_path"]
+62 -3
View File
@@ -48,9 +48,16 @@ and every failure path (git error, model down) never bumps. The
counter is pinned to the migration-0010 seed (0) around every test by
:func:`_reset_sources_version`.
The git / import / overview layers are monkeypatched in ``app.api.sync``
(same fake style as ``test_import_docs_git.py``) — no real git, no LLM:
the runner's state machine and HTTP surface are under test.
The git / import / overview / folder-summary layers are monkeypatched
in ``app.api.sync`` (same fake style as ``test_import_docs_git.py``) —
no real git, no LLM: the runner's state machine and HTTP surface are
under test. Phase 94 (task 02): the folder-summary layer gets the same
treatment — the fake-import tests' canned summaries would otherwise
steer the REAL ``generate_folder_summaries`` at the global
``documents`` table and the real ``LLMClient`` (network); the
real-import tests (host temp dirs, deterministic ``FakeEmbedder``) keep
the real generator, with ``folder_summaries`` truncated around every
test (:func:`_clean_folder_summaries`).
The admin client is used **as a context manager** on purpose: the
background sync task lives on the app's event loop, so the loop must
@@ -182,6 +189,19 @@ def clean_documents(db: Session) -> Iterator[None]:
db.commit()
@pytest.fixture(autouse=True)
def _clean_folder_summaries(db: Session) -> Iterator[None]:
"""Phase 94: the ``folder_summaries`` table is global state the real
generator (the real-import tests) writes — truncated around every
sync test so the change-gate / table-empty-gate assertions start
from a known (empty) table."""
db.execute(text("TRUNCATE folder_summaries"))
db.commit()
yield
db.execute(text("TRUNCATE folder_summaries"))
db.commit()
def _real_llm(monkeypatch: pytest.MonkeyPatch) -> None:
"""The pipeline's ``LLMClient`` becomes the deterministic in-process
``FakeEmbedder`` (real import, no network). ``FakeEmbedder``
@@ -276,6 +296,34 @@ class FakeOverview:
return self.ok
class FakeFolderSummaries:
"""Records every ``generate_folder_summaries`` call; canned stats.
Phase 94 (task 02): keeps the fake-import tests at the deterministic
layer boundary — the real generator would read the global
``documents`` table and call the (real) ``LLMClient`` over the
network. The generator only flushes, so the fake honours the
``skip`` flag the same way (the zero stats, no side effects)."""
ZERO = {"generated": 0, "failed": 0, "pruned": 0}
def __init__(self, stats: dict[str, int] | None = None) -> None:
self.stats = stats if stats is not None else dict(self.ZERO)
self.llms: list[LLMClient] = []
self.sessions: list[Session] = []
self.skip_flags: list[bool] = []
async def __call__(
self, db: Session, llm: LLMClient, *, skip: bool = False
) -> dict[str, int]:
self.skip_flags.append(skip)
if skip:
return dict(self.ZERO)
self.llms.append(llm)
self.sessions.append(db)
return dict(self.stats)
def _fake_clone() -> tuple[list[tuple[str, Path]], object]:
"""A ``clone_or_pull`` that materialises a checkout with one .md file."""
calls: list[tuple[str, Path]] = []
@@ -327,6 +375,7 @@ def test_admin_sync_success_reports_full_detail(
monkeypatch.setattr(sync_api, "import_sources", fake_import)
fake_overview = FakeOverview(ok=True)
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
assert sync_client.get("/api/sync/status").json() == {
@@ -399,6 +448,7 @@ def test_unchanged_kb_skips_overview_refresh(
monkeypatch.setattr(sync_api, "import_sources", fake_import)
fake_overview = FakeOverview(ok=True)
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
@@ -435,6 +485,7 @@ def test_double_trigger_while_running_returns_409(
fake_import = FakeImportSources(ImportSummary(files=1, added=1), delay=0.5)
monkeypatch.setattr(sync_api, "import_sources", fake_import)
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
@@ -551,12 +602,17 @@ def test_db_rows_win_over_env(
fake_import = FakeImportSources(ImportSummary(files=1, added=1))
monkeypatch.setattr(sync_api, "import_sources", fake_import)
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
fake_folders = FakeFolderSummaries()
monkeypatch.setattr(sync_api, "generate_folder_summaries", fake_folders)
_login(sync_client)
with caplog.at_level(logging.INFO, logger="app.api.sync"):
assert sync_client.post("/api/sync").status_code == 202
body = _poll(sync_client, "success")
# Phase 94: the changed canned summary fired the (stubbed) folder
# step with the run's own session + the import's LLM client.
assert fake_folders.llms == [fake_import.llms[0]]
assert clone_calls == [(db_url, tmp_path / "bor" / "managed")]
assert fake_import.sources == [[tmp_path / "bor" / "managed"]]
assert "env.example.com" not in str(body) # the env URL never reaches the UI
@@ -584,6 +640,7 @@ def test_env_fallback_when_table_empty(
fake_import = FakeImportSources(ImportSummary(files=1, added=1))
monkeypatch.setattr(sync_api, "import_sources", fake_import)
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
with caplog.at_level(logging.INFO, logger="app.api.sync"):
@@ -988,6 +1045,7 @@ def test_sync_builds_ignore_map_by_root_string_with_union(
fake_import = FakeImportSources(ImportSummary(files=1, added=1))
monkeypatch.setattr(sync_api, "import_sources", fake_import)
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
@@ -1022,6 +1080,7 @@ def test_sync_without_ignore_lists_passes_empty_map(
fake_import = FakeImportSources(ImportSummary(files=1, added=1))
monkeypatch.setattr(sync_api, "import_sources", fake_import)
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
monkeypatch.setattr(sync_api, "generate_folder_summaries", FakeFolderSummaries())
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
@@ -0,0 +1,664 @@
"""Integration: the sync-time folder-summary wiring (phase 94, task 02).
Extends the KB-overview sync pattern (``test_import_docs_overview.py``
is the template) to the phase-94 folder summaries — both sync paths
regenerate ``folder_summaries`` change-gated (added + updated > 0) or
on an empty table (the first full run after migration 0017 / a
``--limit`` first walk), per-folder fail-soft, best-effort, and in the
run's own short-lived session (the phase-53 flush-then-caller-commits
convention — the generator only flushes, the sync path commits).
Script path (``scripts.import_docs.main`` end to end, fake LLM, real
DB, explicit ``--source``):
- a KB-changing import → one row per ≥ 2-doc subtree (the source root
+ the 2-doc folder; the 1-doc folder gets none), committed in the
run's transaction, the summary line ending
``folder_summaries=<generated>/<failed>/<pruned>``;
- an unchanged re-import → zero ``lite`` calls,
``folder_summaries=skipped``, rows untouched;
- a subtree dropping below 2 docs after a changed re-walk → its row
pruned;
- one folder's ``lite`` failure → its previous row kept, the other
lands, exit code 0, stats ``1/1/0``;
- a ``--limit`` debug run → no generation, no rows,
``folder_summaries=skipped``;
- a fresh (empty) table after a ``--limit`` first walk → an unchanged
full walk generates (the table-empty first-run trigger).
API path (``POST /api/sync`` end to end, real import over a host temp
local dir, deterministic ``FakeEmbedder``):
- a KB-changing sync → the rows land (visible via the test's own
session) and the status detail keeps its exact pre-phase key set
(no folder-summary surface — the stats are log-only);
- an unchanged re-sync → zero ``FOLDER_SUMMARY_MODE`` calls;
- a ``lite`` outage (one folder failing) → the failed folder's row is
kept, the run reports ``success`` (never ``failed``), and the
sources-version bump still lands (the bump is change-gated on the
KB, not on the summaries);
- an empty table after a populated sync (the migration-0017 scenario)
→ an unchanged walk regenerates (the overview's API gate, purely
change-gated, does not).
"""
from __future__ import annotations
import logging
import time
from collections.abc import Iterator
from datetime import datetime
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select, text
from sqlalchemy.orm import Session
from app.api import sync as sync_api
from app.config import Settings
from app.db import SessionLocal, db_available
from app.main import app as fastapi_app
from app.models import FolderSummary, GitSource
from app.rag import git_sources as git_sources_resolver
from app.rag.llm import LLMError
from app.rag.sources_meta import current_sources_version
from scripts import import_docs
from tests.conftest import ADMIN_PASSWORD
from tests.fakes import FakeEmbedder
# --- shared helpers ---------------------------------------------------------
def _folder_calls(llm: FakeEmbedder) -> list[list[dict[str, str]]]:
"""The ``FOLDER_SUMMARY_MODE`` chat calls on the client (the marker
in the system prompt — the KB-overview marker never contains it)."""
return [
msgs
for msgs in llm.chat_calls
if any(
"FOLDER_SUMMARY_MODE" in m.get("content", "")
for m in msgs
if m.get("role") == "system"
)
]
def _rows(db: Session) -> dict[tuple[str, str], str]:
"""The stored folder summaries as ``{(source, folder_path): summary}``."""
db.expire_all()
return {
(source, folder_path): summary
for source, folder_path, summary in db.execute(
select(FolderSummary.source, FolderSummary.folder_path, FolderSummary.summary)
).all()
}
def _updated_at(db: Session, source: str, folder_path: str) -> datetime | None:
db.expire_all()
row = db.get(FolderSummary, (source, folder_path))
return row.updated_at if row is not None else None
class FolderFailingChatEmbedder(FakeEmbedder):
"""``lite`` stand-in that fails exactly one folder-summary call —
the one whose user message contains *fail_label* (a ``Folder: …``
header) — and answers everything else (incl. the KB overview)
normally. Drives the per-folder fail-soft path."""
def __init__(self, fail_label: str) -> None:
super().__init__()
self.fail_label = fail_label
async def chat(
self, messages: list[dict[str, str]], model: str | None = None
) -> str:
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
if self.fail_label in user:
self.chat_calls.append(list(messages))
raise LLMError("simulated folder-summary outage (test sentinel)")
return await super().chat(messages, model)
# --- fixtures ----------------------------------------------------------------
@pytest.fixture(autouse=True)
def _clean_kb(db: Session) -> Iterator[None]:
"""Global KB + registry state, truncated around every test (the
real import and the real generator write ``documents``/``chunks``/
``folder_summaries``; the API tests seed ``git_sources``)."""
db.execute(
text("TRUNCATE chunks, documents, kb_overview, folder_summaries, git_sources")
)
db.commit()
yield
db.execute(
text("TRUNCATE chunks, documents, kb_overview, folder_summaries, git_sources")
)
db.commit()
@pytest.fixture(autouse=True)
def _reset_sources_version() -> Iterator[None]:
"""The sources version counter is global mutable state — pin it to
the migration-0010 seed (0) around every test (own session: both
sync paths bump through their own short-lived ``SessionLocal``).
Skips like the ``db`` fixture when Postgres is down."""
if not db_available():
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
session = SessionLocal()
try:
session.execute(text("UPDATE sources_meta SET version = 0 WHERE id = 1"))
session.commit()
yield
finally:
session.execute(text("UPDATE sources_meta SET version = 0 WHERE id = 1"))
session.commit()
session.close()
@pytest.fixture(autouse=True)
def _fresh_sync_state() -> Iterator[None]:
"""The module-level status object + task are process-global: reset
them around every test (harmless for the script-path tests)."""
sync_api._status = sync_api.SyncStatus()
sync_api._task = None
yield
sync_api._status = sync_api.SyncStatus()
sync_api._task = None
@pytest.fixture()
def src(tmp_path: Path) -> Path:
"""MyDocs: a/ (2 docs) + b/ (1 doc) → subtree counts root 3, a 2, b 1.
md → no document-summary chat calls, so the ``lite`` traffic is
exactly the overview + the folder summaries."""
root = tmp_path / "MyDocs"
(root / "a").mkdir(parents=True)
(root / "b").mkdir()
(root / "a" / "one.md").write_text("# A One\nFirst folder document.\n", encoding="utf-8")
(root / "a" / "two.md").write_text("# A Two\nSecond folder document.\n", encoding="utf-8")
(root / "b" / "three.md").write_text("# B Three\nLone folder document.\n", encoding="utf-8")
return root
def _run_main(
monkeypatch: pytest.MonkeyPatch,
llm: FakeEmbedder,
argv: list[str],
capsys: pytest.CaptureFixture[str],
) -> tuple[int, str]:
"""Run ``import_docs.main`` with fresh settings, a fake LLM, and a
fail-loud git mock (``--source`` always wins, so git must stay
idle) — the phase-31 overview test's runner."""
monkeypatch.setattr(
import_docs,
"get_settings",
lambda: Settings(_env_file=None), # pyright: ignore[reportCallIssue]
)
def _no_git(url: str, dest: Path | str) -> Path:
raise AssertionError("git sync must not run with explicit --source")
monkeypatch.setattr(import_docs, "clone_or_pull", _no_git)
monkeypatch.setattr(import_docs, "LLMClient", lambda: llm)
rc = import_docs.main(argv)
return rc, capsys.readouterr().out
# --- script path: scripts/import_docs.py -------------------------------------
def test_changed_import_generates_folder_rows(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""A KB-changing import upserts one row per ≥ 2-doc subtree in the
run's own transaction — committed and visible afterwards — with the
stats on the summary line (PLAN §9)."""
llm = FakeEmbedder()
records: list[logging.LogRecord] = []
class _Sink(logging.Handler):
def emit(self, record: logging.LogRecord) -> None:
records.append(record)
fs_logger = logging.getLogger("app.rag.folder_summaries")
sink = _Sink()
fs_logger.addHandler(sink)
fs_logger.setLevel(logging.INFO)
try:
rc, out = _run_main(monkeypatch, llm, ["--source", str(src)], capsys)
finally:
fs_logger.removeHandler(sink)
assert rc == 0
assert "added=3" in out
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=2/0/0"
)
# Two FOLDER_SUMMARY_MODE calls — the source root (3 docs) and the
# 2-doc folder a; the 1-doc folder b gets no row (no lite burn).
calls = _folder_calls(llm)
assert len(calls) == 2
headers = [c[1]["content"].splitlines()[0] for c in calls]
assert headers == ["Folder: MyDocs", "Folder: MyDocs/a"]
for c in calls:
assert "FOLDER_SUMMARY_MODE" in c[0]["content"]
# The rows land committed (the run's own session committed them) —
# one per ≥ 2-doc subtree, never empty, stamped.
rows = _rows(db)
assert set(rows) == {("MyDocs", ""), ("MyDocs", "a")}
assert all(rows.values())
assert _updated_at(db, "MyDocs", "a") is not None
# The stats log line (PLAN §9 ample logging).
assert any(
"folder_summaries: generated=2 failed=0 pruned=0" in r.getMessage()
for r in records
)
def test_unchanged_reimport_burns_zero_folder_calls(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""Same hashes → no KB change → the populated table stays untouched
and zero ``lite`` calls burn (overview AND folder summaries)."""
llm1 = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm1, ["--source", str(src)], capsys)
assert rc == 0
assert out.rstrip().endswith(
"overview=updated sources_version=1 folder_summaries=2/0/0"
)
rows = _rows(db)
assert set(rows) == {("MyDocs", ""), ("MyDocs", "a")}
llm2 = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
assert rc == 0
assert "unchanged=3" in out
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert llm2.chat_calls == [] # zero lite calls, any mode
assert _rows(db) == rows # rows byte-identical
def test_subtree_dropping_below_two_docs_is_pruned(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""A subtree that drops below the ≥ 2 rule on a changed re-walk
loses its (stale) row; the remaining qualifiers regenerate."""
llm1 = FakeEmbedder()
rc, _ = _run_main(monkeypatch, llm1, ["--source", str(src)], capsys)
assert rc == 0
assert set(_rows(db)) == {("MyDocs", ""), ("MyDocs", "a")}
# a/ loses one of its two docs (below the ≥ 2 rule) while b's doc
# changes → the re-walk is a KB change, so generation runs and the
# stale a/ row is pruned; b (1 doc) still gets no row.
(src / "a" / "two.md").unlink()
(src / "b" / "three.md").write_text("# B Three\nChanged content.\n", encoding="utf-8")
llm2 = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src), "--prune"], capsys)
assert rc == 0
assert "pruned=1" in out
assert "updated=1" in out
assert out.rstrip().endswith(
"overview=updated sources_version=2 folder_summaries=1/0/1"
)
# Only the source root (the 2 remaining docs) still qualifies.
assert set(_rows(db)) == {("MyDocs", "")}
assert _updated_at(db, "MyDocs", "a") is None # the row is gone
def test_folder_lite_failure_keeps_previous_row_and_stays_green(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""One folder's ``lite`` failure: its previous row is kept, the
other folder lands, and the run's exit code stays 0 — the stats
carry the failure (``1/1/0``)."""
llm1 = FakeEmbedder()
rc, _ = _run_main(monkeypatch, llm1, ["--source", str(src)], capsys)
assert rc == 0
rows_before = _rows(db)
a_stamp_before = _updated_at(db, "MyDocs", "a")
root_stamp_before = _updated_at(db, "MyDocs", "")
assert a_stamp_before is not None and root_stamp_before is not None
# a/ changes (a KB change → generation runs); lite fails for a/
# only — its previous row must stay, the root row must land.
(src / "a" / "one.md").write_text("# A One\nChanged content.\n", encoding="utf-8")
llm2 = FolderFailingChatEmbedder(fail_label="Folder: MyDocs/a")
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
assert rc == 0 # a failed folder must not fail the import
assert "updated=1" in out
assert out.rstrip().endswith(
"overview=updated sources_version=2 folder_summaries=1/1/0"
)
rows_after = _rows(db)
assert rows_after["MyDocs", "a"] == rows_before["MyDocs", "a"] # kept
assert _updated_at(db, "MyDocs", "a") == a_stamp_before # untouched
root_stamp_after = _updated_at(db, "MyDocs", "")
assert root_stamp_after is not None and root_stamp_after > root_stamp_before
def test_limit_run_skips_folder_generation(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""A ``--limit`` debug run (an incomplete walk) never generates —
no ``lite`` call at all, no rows, the line says ``skipped``."""
llm = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm, ["--source", str(src), "--limit", "2"], capsys)
assert rc == 0
assert "added=2" in out
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert llm.chat_calls == [] # no lite call, any mode
assert _rows(db) == {} # an incomplete walk never writes rows
def test_empty_table_generates_on_unchanged_walk(
db: Session,
src: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""The table-empty first-run trigger: after a ``--limit`` first
walk (populated KB, empty table), an unchanged full walk generates
— for the folder summaries AND the missing outline, still never
bumping the version."""
llm1 = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm1, ["--source", str(src), "--limit", "3"], capsys)
assert rc == 0
assert "added=3" in out
assert out.rstrip().endswith(
"overview=skipped sources_version=skipped folder_summaries=skipped"
)
assert _rows(db) == {}
llm2 = FakeEmbedder()
rc, out = _run_main(monkeypatch, llm2, ["--source", str(src)], capsys)
assert rc == 0
assert "unchanged=3" in out
assert out.rstrip().endswith(
"overview=updated sources_version=skipped folder_summaries=2/0/0"
)
assert set(_rows(db)) == {("MyDocs", ""), ("MyDocs", "a")}
assert len(_folder_calls(llm2)) == 2
assert current_sources_version(db) == 0 # the unchanged walk never bumps
# --- API path: POST /api/sync -------------------------------------------------
@pytest.fixture()
def sync_client() -> Iterator[TestClient]:
"""Context-managed TestClient — one app event loop across requests
(the background task must survive between the POST and the polls)."""
with TestClient(fastapi_app) as client:
yield client
def _settings(sources_dir: str) -> Settings:
return Settings(_env_file=None, sources_dir=sources_dir) # pyright: ignore[reportCallIssue]
def _stub_env(monkeypatch: pytest.MonkeyPatch, git_sources: str = "") -> None:
"""The resolver's env fallback, driven by a fresh ``Settings`` (the
dev ``.env`` never leaks in)."""
monkeypatch.setattr(
git_sources_resolver,
"get_settings",
lambda: Settings(_env_file=None, git_sources=git_sources), # pyright: ignore[reportCallIssue]
)
def _seed_local(db: Session, path: Path) -> None:
"""A ``kind=local`` row as the phase-38 API stores it: the expanded
absolute path in both ``path`` and the NOT-NULL ``url`` column."""
db.add(GitSource(url=str(path), kind="local", path=str(path)))
db.commit()
def _capture_llm(monkeypatch: pytest.MonkeyPatch) -> list[FakeEmbedder]:
"""The pipeline's ``LLMClient`` becomes a recorded
``FakeEmbedder`` (real import + real generator, no network)."""
clients: list[FakeEmbedder] = []
def _factory() -> FakeEmbedder:
client = FakeEmbedder()
clients.append(client)
return client
monkeypatch.setattr(sync_api, "LLMClient", _factory)
return clients
def _login(client: TestClient) -> None:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
def _poll(client: TestClient, want: str, timeout: float = 10.0) -> dict:
"""Poll ``GET /api/sync/status`` until ``state == want`` (terminal).
Any state other than ``running`` before the deadline fails loudly —
an unexpected ``failed`` must never be masked by the wait."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
body = client.get("/api/sync/status").json()
if body["state"] == want:
return body
assert body["state"] == "running", (
f"unexpected state {body['state']!r} while waiting for {want!r}: {body}"
)
time.sleep(0.05)
raise AssertionError(f"sync did not reach {want!r} within {timeout}s")
@pytest.fixture()
def local_dir(tmp_path: Path) -> Path:
"""LocalDocs: a/ (2 docs) → subtree counts root 2, a 2 (one
qualifying source-root row + one folder row)."""
d = tmp_path / "LocalDocs"
(d / "a").mkdir(parents=True)
(d / "a" / "one.md").write_text("# A One\nfirst local file\n", encoding="utf-8")
(d / "a" / "two.md").write_text("# A Two\nsecond local file\n", encoding="utf-8")
return d
def test_api_changed_sync_generates_folder_rows(
sync_client: TestClient,
monkeypatch: pytest.MonkeyPatch,
db: Session,
local_dir: Path,
) -> None:
"""A KB-changing sync upserts the rows (visible via the test's own
session) and keeps the status detail's exact pre-phase shape — the
folder stats are log-only, not a new status surface."""
_seed_local(db, local_dir)
_stub_env(monkeypatch)
monkeypatch.setattr(
sync_api,
"get_settings",
lambda: _settings(str(local_dir.parent / "bor")),
)
clients = _capture_llm(monkeypatch)
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
body = _poll(sync_client, "success")
assert body["error"] is None
# No new sync-status surface: the detail keeps its exact key set.
assert set(body["detail"]) == {
"files", "added", "updated", "unchanged", "pruned", "errors",
"chunks", "summaries", "summary_errors", "overview",
"sources_version",
}
assert body["detail"]["files"] == 2
assert body["detail"]["added"] == 2
assert body["detail"]["overview"] is True
assert body["detail"]["sources_version"] == 1
# One LLM client for probe + import + overview + folder summaries;
# exactly two FOLDER_SUMMARY_MODE calls (root + the 2-doc a/ folder).
assert len(clients) == 1
calls = _folder_calls(clients[0])
assert len(calls) == 2
headers = [c[1]["content"].splitlines()[0] for c in calls]
assert headers == ["Folder: LocalDocs", "Folder: LocalDocs/a"]
# The rows land committed — visible from the test's own session.
rows = _rows(db)
assert set(rows) == {("LocalDocs", ""), ("LocalDocs", "a")}
assert all(rows.values())
assert current_sources_version(db) == 1
def test_api_unchanged_resync_burns_zero_folder_calls(
sync_client: TestClient,
monkeypatch: pytest.MonkeyPatch,
db: Session,
local_dir: Path,
) -> None:
"""Same files → unchanged re-sync: no ``lite`` call of any kind
(populated table), rows untouched, version unadvanced."""
_seed_local(db, local_dir)
_stub_env(monkeypatch)
monkeypatch.setattr(
sync_api,
"get_settings",
lambda: _settings(str(local_dir.parent / "bor")),
)
clients = _capture_llm(monkeypatch)
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
_poll(sync_client, "success")
rows = _rows(db)
assert set(rows) == {("LocalDocs", ""), ("LocalDocs", "a")}
assert sync_client.post("/api/sync").status_code == 202
body = _poll(sync_client, "success")
assert body["detail"]["overview"] is False
assert body["detail"]["sources_version"] == 1 # unchanged → no bump
assert len(clients) == 2
# Zero GENERATION calls — no folder summary, no KB overview. The
# only chat the re-sync's client makes is the phase-41 probe's ping.
assert _folder_calls(clients[1]) == []
assert all(
"KB_OVERVIEW_MODE" not in m.get("content", "")
for msgs in clients[1].chat_calls
for m in msgs
if m.get("role") == "system"
)
assert len(clients[1].chat_calls) == 1
assert clients[1].chat_calls[0][0]["content"] == "ping"
assert _rows(db) == rows
assert current_sources_version(db) == 1
def test_api_folder_lite_failure_keeps_rows_stays_green_and_bumps(
sync_client: TestClient,
monkeypatch: pytest.MonkeyPatch,
db: Session,
local_dir: Path,
) -> None:
"""A ``lite`` outage for one folder: the run stays ``success``
(never ``failed``), the failed folder's previous row is kept, the
rest regenerates, and the sources-version bump still lands (the
bump is change-gated on the KB, not on the summaries)."""
_seed_local(db, local_dir)
_stub_env(monkeypatch)
monkeypatch.setattr(
sync_api,
"get_settings",
lambda: _settings(str(local_dir.parent / "bor")),
)
clients = _capture_llm(monkeypatch)
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
_poll(sync_client, "success")
rows_before = _rows(db)
a_stamp_before = _updated_at(db, "LocalDocs", "a")
root_stamp_before = _updated_at(db, "LocalDocs", "")
assert a_stamp_before is not None and root_stamp_before is not None
assert len(clients) == 1 # sync 1 used the recording factory
# a/ changes (the KB changes) while lite fails for a/ only.
(local_dir / "a" / "one.md").write_text(
"# A One\nchanged local file\n", encoding="utf-8"
)
monkeypatch.setattr(
sync_api,
"LLMClient",
lambda: FolderFailingChatEmbedder(fail_label="Folder: LocalDocs/a"),
)
assert sync_client.post("/api/sync").status_code == 202
body = _poll(sync_client, "success") # GREEN — a lite outage is fail-soft
assert body["error"] is None
assert body["detail"]["updated"] == 1
assert body["detail"]["sources_version"] == 2 # the bump was not blocked
assert current_sources_version(db) == 2
rows_after = _rows(db)
assert rows_after["LocalDocs", "a"] == rows_before["LocalDocs", "a"]
assert _updated_at(db, "LocalDocs", "a") == a_stamp_before # kept as-is
root_stamp_after = _updated_at(db, "LocalDocs", "")
assert root_stamp_after is not None and root_stamp_after > root_stamp_before
def test_api_empty_table_first_sync_regenerates(
sync_client: TestClient,
monkeypatch: pytest.MonkeyPatch,
db: Session,
local_dir: Path,
) -> None:
"""The migration-0017 scenario: the KB predates the table — wipe
the rows and re-sync an unchanged KB: the empty-table trigger
fires for the folder summaries (the overview's API gate, purely
change-gated, does not)."""
_seed_local(db, local_dir)
_stub_env(monkeypatch)
monkeypatch.setattr(
sync_api,
"get_settings",
lambda: _settings(str(local_dir.parent / "bor")),
)
clients = _capture_llm(monkeypatch)
_login(sync_client)
assert sync_client.post("/api/sync").status_code == 202
_poll(sync_client, "success")
assert set(_rows(db)) == {("LocalDocs", ""), ("LocalDocs", "a")}
db.execute(text("TRUNCATE folder_summaries"))
db.commit()
assert sync_client.post("/api/sync").status_code == 202
body = _poll(sync_client, "success")
assert body["detail"]["overview"] is False # unchanged → no overview
assert body["detail"]["sources_version"] == 1 # unchanged → no bump
assert len(clients) == 2
assert len(_folder_calls(clients[1])) == 2 # the folders regenerated
assert set(_rows(db)) == {("LocalDocs", ""), ("LocalDocs", "a")}