phase: 155_folder_summary_voice
All verification is complete — the whole project is 100% green. Final report: **Phase 155 final verification pass — complete.** **Verified (all tasks 01–04 done in `complete/`):** - Instruction nudge in `app/rag/folder_summaries.py` (forbidden openers as the pin, 4 constraints preserved, docstring cites the phase) - `scripts/regen_folder_summaries.py` (import_docs idiom, full mode, cron-safe summary line, fail-loud boot) - Unit pins, integration suite, mock voice update, dedicated E2E suite **Test / lint / coverage:** - `uv run pytest --cov=app -q` → 100% passed (exit 0, zero failures); coverage **99%** on `app/` (>90% ✓) - `uv run pytest tests/e2e/test_folder_summary_voice.py -v --no-cov` → **3 passed** in isolation - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings **Completion criteria:** 1. Marker + header prefix byte-identical — ✅ (git vs HEAD + unit pins verbatim, green) 2. Regen script full regeneration — ✅ (live smoke: exit 0, `folders=… limit=7` summary line; misconfig boot → named stderr, exit 1; `updated_at`-advance/no-prune/canned-text semantics proven by the integration suite, green) 3. Sync path untouched — ✅ (0 diffs to `app/api/sync.py` / `app/rag/importer.py`; phase-127 pins green) 4. E2E stored + rendered without forbidden openers — ✅ (3/3, exact canned-string assertions + phase-127 regression guard) 5. Full suite green, E2E isolated green, ruff + pyright clean, coverage >90% — ✅ 6. Atomic commit — deferred to the harness (working tree left intact per protocol) **Documentation:** README.md and AGENTS.md reviewed in full — verified consistent, no changes needed (README already documents the new script in the update workflow; all `.env.example` vars documented; pages table matches routes; 17 colors = 9+8; no stale mock-voice references). **Decisions:** No live-infrastructure changes (dev-DB read + a 0-row script smoke only). The `TODO.md` working-tree edit is the owner's own, left untouched. **Next pending phase:** `156_table_ux_polish`.
This commit is contained in:
+12
-8
@@ -19,7 +19,9 @@ Implements just enough of the aipi surface:
|
||||
of the user message (the generator puts the document list there) —
|
||||
byte-stable for a given KB (KB overview, phase 31)
|
||||
- ``FOLDER_SUMMARY_MODE`` -> the deterministic folder one-liner
|
||||
``Fixture folder summary for <folder>.`` — <folder> is the user
|
||||
``Documentation for <folder>.`` (phase 155 task 01: the new voice
|
||||
— a natural opener naming the folder, never the forbidden
|
||||
"This folder contains…" openers) — <folder> is the user
|
||||
message's ``Folder: …`` header line (the generator names the
|
||||
source/folder there, phase 94), so the stored row always names its
|
||||
folder and an E2E can assert on it. Checked BEFORE the
|
||||
@@ -836,7 +838,7 @@ test_llm_retry.py``). The mock is single-conversation per e2e server, so
|
||||
""`` and ``choices[0].finish_reason = "length"`` (the exact wire
|
||||
shape of the empty ``lite`` reply: the budget spent in
|
||||
``reasoning_content``) — and every later POST returns the normal
|
||||
``Fixture folder summary for <folder>.`` line (the one-shot retry,
|
||||
``Documentation for <folder>.`` line (the one-shot retry,
|
||||
phase 96 task 01, recovers the row).
|
||||
- the label ends with ``/e2e_empty_always``: EVERY non-stream POST
|
||||
for that label answers the empty envelope (the 1 +
|
||||
@@ -3337,14 +3339,16 @@ def compose_answer(body: dict[str, Any]) -> str:
|
||||
if LONG_ANSWER_TRIGGER in user.lower():
|
||||
answer = long_answer()
|
||||
elif "FOLDER_SUMMARY_MODE" in system:
|
||||
# Folder summary (phase 94, TODO.md L4): the ``lite`` stand-in
|
||||
# returns the deterministic one-liner
|
||||
# ``Fixture folder summary for <folder>.`` — <folder> is the
|
||||
# Folder summary (phase 94, voice nudge phase 155 task 01):
|
||||
# the ``lite`` stand-in returns the deterministic one-liner
|
||||
# ``Documentation for <folder>.`` — <folder> is the
|
||||
# user message's ``Folder: …`` header line (the generator puts
|
||||
# the canonical source/folder identity there, imported as
|
||||
# ``FOLDER_HEADER_PREFIX`` so the mock can never drift from it).
|
||||
# The stored row therefore always names its folder — the drill-
|
||||
# down E2E (``test_ls_tree_drilldown.py``) asserts on it.
|
||||
# The stored row therefore always names its folder with a
|
||||
# natural opener (never the forbidden "This folder contains…"
|
||||
# openers — phase 155's pin) — the drill-down E2E
|
||||
# (``test_ls_tree_drilldown.py``) asserts on it.
|
||||
# Checked BEFORE the ``SUMMARY_MODE`` branch: the folder marker
|
||||
# CONTAINS the summary marker as a substring (``FOLDER_`` +
|
||||
# ``SUMMARY_MODE``), so the summary branch would otherwise
|
||||
@@ -3363,7 +3367,7 @@ def compose_answer(body: dict[str, Any]) -> str:
|
||||
if header.startswith(FOLDER_HEADER_PREFIX)
|
||||
else "(unnamed folder)"
|
||||
)
|
||||
answer = f"Fixture folder summary for {folder}."
|
||||
answer = f"Documentation for {folder}."
|
||||
elif "SUMMARY_MODE" in system:
|
||||
# Document summaries (phase 30): the ``lite`` stand-in returns a
|
||||
# deterministic digest — the first 24 tokens of the user message
|
||||
|
||||
@@ -0,0 +1,697 @@
|
||||
"""Phase 155 task 04 E2E (Playwright, mock-only): the new voice, end to end.
|
||||
|
||||
The dedicated phase suite for ``155_folder_summary_voice`` (A16 — one
|
||||
Playwright file per phase, run in isolation): the owner's audit found
|
||||
that every ``lite`` folder summary opened with the formulaic "This
|
||||
folder contains…" catalogue. The fix under test, proven here end to
|
||||
end: a KB change triggers the sync's folder regeneration (the
|
||||
change-gated path, phase 127) under the deterministic mock, the
|
||||
STORED ``folder_summaries`` row for the regenerated folder carries the
|
||||
mock's new-voice canned text — ``Documentation for <folder>.`` (a
|
||||
natural opener naming the folder, NEVER the two forbidden "This folder
|
||||
contains" / "The <folder> folder contains" openers — phase 155's pin)
|
||||
— and the RAG catalogue (the phase-97 view) renders that stored voice
|
||||
in its Description cells.
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_folder_summary_voice.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
|
||||
deterministic mock (the ``FOLDER_SUMMARY_MODE`` branch's canned
|
||||
``Documentation for <folder>.`` line, ``tests/e2e/mock_llm.py``): the
|
||||
stored rows are a pure function of the request (the label is the
|
||||
``Folder: …`` header tail — ``<source>`` for the source root,
|
||||
``<source>/<folder>`` for a folder), so the exact-text and the
|
||||
forbidden-opener assertions are deterministic only against the mock.
|
||||
|
||||
No slow-LLM proxy: the assertions are on the POST-terminal DB state
|
||||
(the rows' ``summary`` AND ``updated_at``) plus the terminal status
|
||||
(the keep-final-counts convention — the terminal keeps
|
||||
``summaries_done == summaries_total == <affected count>``) and the
|
||||
rendered RAG view, never on in-flight sampling, so the fast mock is
|
||||
dense enough by construction.
|
||||
|
||||
KB fixture — one host temp dir (``tmp_path_factory``; the app runs on
|
||||
the same host) registered as a ``kind=local`` source (the
|
||||
``test_local_directory_sources.py`` / phase-94 API-registration
|
||||
pattern; no git anywhere): ``voice/`` with TWO 2-doc folders —
|
||||
``alpha/`` (the folder every test changes) and ``bravo/`` (the
|
||||
unchanged sibling — the phase-127 regression guard) — plus ONE
|
||||
root-level doc (5 documents total). The generator's candidate set is
|
||||
therefore exactly 3: the source root (``folder_path ""``) + the two
|
||||
folders (the recursive-subtree ≥ 1 rule).
|
||||
|
||||
Autouse cleanup (the phase-96/127 pattern): before each test wait for
|
||||
no running sync on the module app, then truncate the shared registry +
|
||||
KB tables (documents/chunks/query_log/steering_notes/kb_overview/
|
||||
git_sources/folder_summaries) — each test seeds its own source and
|
||||
runs its own syncs; after each test the same truncate. (The temp tree
|
||||
itself persists for the module; the tests' disk mutations are content
|
||||
revisions of existing fixture files — no file is ever added or
|
||||
removed — so every test's syncs are deltas against its own truncated
|
||||
KB and the counts below hold independently of the tree state the
|
||||
previous test left behind.)
|
||||
|
||||
Test → observable mapping (Playwright Mapping Rule):
|
||||
1. ``test_regen_summary_avoids_forbidden_openers`` — a first (full)
|
||||
sync settles ``success``; ONE doc inside ``alpha/`` is content-
|
||||
revised and the second sync (the change-gated path, phase 127)
|
||||
settles ``success`` with the single-file detail and
|
||||
``summaries_done == summaries_total == 2`` (the AFFECTED count —
|
||||
alpha + the source root): the stored ``folder_summaries`` row for
|
||||
``alpha`` EQUALS the mock's canned reply for that folder
|
||||
(``Documentation for voice/alpha.`` — the exact string, a pure
|
||||
function of the request) AND does not start with "This folder
|
||||
contains" or "The voice/alpha folder contains" (the phase-155
|
||||
shape pin — the regenerated voice, not the formulaic catalogue),
|
||||
while the source-root row — the other affected row — likewise
|
||||
carries its exact canned line.
|
||||
2. ``test_rag_table_renders_the_stored_voice`` — the same
|
||||
sync-#1 + alpha-revision + sync-#2 sequence (the regenerated
|
||||
folder's row is re-stamped under the new instruction), then the
|
||||
REAL admin login (the ``auth_helpers`` form pattern) opens the RAG
|
||||
view: the top-level source row's Description cell renders the
|
||||
stored source-root voice (the cell text AND its ``title`` — the
|
||||
phase-99 one-line clamp keeps the FULL text in the DOM and on the
|
||||
hover escape hatch, so the assertion holds whether or not the
|
||||
display ellipsizes); drilling into the source, the ``alpha`` row's
|
||||
Description cell renders ``Documentation for voice/alpha.`` (text +
|
||||
``title``) and the ``bravo`` row its own canned line, and the
|
||||
level block shows the stored source-root voice — the stored voice
|
||||
is what the catalogue renders, end to end.
|
||||
3. ``test_unchanged_folders_keep_their_rows`` (the phase-127
|
||||
regression guard) — sync #1 (full), capturing every row's
|
||||
``(summary, manually_edited, updated_at)``; ONE doc inside
|
||||
``alpha/`` is content-revised and sync #2 settles ``success`` with
|
||||
``summaries_done == summaries_total == 2`` (the AFFECTED count):
|
||||
the ``bravo`` row is BYTE-IDENTICAL — same ``summary`` AND same
|
||||
``updated_at`` (a KB change in folder A does not touch folder B's
|
||||
stored row), while the ``alpha`` AND the source-root rows are
|
||||
re-stamped (``updated_at`` strictly after the captured stamp, the
|
||||
canned text, still AI-written) — asserted through the DB the way
|
||||
the phase-127 suite does.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from playwright.sync_api import Page, expect
|
||||
from sqlalchemy import select, text
|
||||
|
||||
from app.config import Settings as _Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import FolderSummary
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import (
|
||||
ADMIN_PASSWORD,
|
||||
SESSION_SECRET,
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
# Phase 79 (task 04, full inventory): the conftest session app owns its
|
||||
# port in a combined run — this module's app binds its own port instead
|
||||
# (a same-port second uvicorn dies on bind and would drive the wrong
|
||||
# server). Env-overridable.
|
||||
APP_PORT = int(os.environ.get("E2E_APP_PORT_FVOICE", "8158"))
|
||||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fixture constants (deterministic, token-controlled)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: The local source — the temp directory's basename (``kind=local`` →
|
||||
#: the directory's basename is the source name, phase 38).
|
||||
SOURCE = "voice"
|
||||
|
||||
#: The folder every test changes (A) and the unchanged sibling (B —
|
||||
#: the phase-127 regression guard's keeper).
|
||||
FOLDER_A = "alpha"
|
||||
FOLDER_B = "bravo"
|
||||
|
||||
ROOT_NOTE = "root-note.md" # the one root-level doc
|
||||
|
||||
N_FILES = 5 # two folders × 2 docs + the root-level doc
|
||||
N_CANDIDATES = 3 # the source root + the two folders (the ≥ 1-doc rule)
|
||||
|
||||
#: The mock's byte-stable ``FOLDER_SUMMARY_MODE`` lines for this fixture
|
||||
#: (the phase-94 template — the label is the ``FOLDER_HEADER_PREFIX``
|
||||
#: tail: ``<source>`` for the root, ``<source>/<folder>`` for a folder;
|
||||
#: the phase-155 voice: a natural opener naming the folder, never the
|
||||
#: forbidden "This folder contains…" openers).
|
||||
SUMMARY_FOR = "Documentation for {}."
|
||||
EXPECTED: dict[str, str] = {
|
||||
"": SUMMARY_FOR.format(SOURCE),
|
||||
FOLDER_A: SUMMARY_FOR.format(f"{SOURCE}/{FOLDER_A}"),
|
||||
FOLDER_B: SUMMARY_FOR.format(f"{SOURCE}/{FOLDER_B}"),
|
||||
}
|
||||
|
||||
#: The phase-155 shape pin (test 1): the forbidden openers the new
|
||||
#: instruction bans — the generic "This folder contains" opener and the
|
||||
#: "The <folder name> folder contains" form with <folder name> bound to
|
||||
#: the mock's own label (``voice/alpha``) and to the bare folder name
|
||||
#: (``alpha``) — the stored voice must open with none of them.
|
||||
FORBIDDEN_PREFIXES: tuple[str, ...] = (
|
||||
"This folder contains",
|
||||
f"The {SOURCE}/{FOLDER_A} folder contains",
|
||||
f"The {FOLDER_A} folder contains",
|
||||
)
|
||||
|
||||
|
||||
def _forbidden_hits(summary: str) -> list[str]:
|
||||
"""The forbidden openers the text starts with (``[]`` = the shape
|
||||
pin holds — the summary opens in the varied voice)."""
|
||||
return [p for p in FORBIDDEN_PREFIXES if summary.startswith(p)]
|
||||
|
||||
|
||||
#: Test 1's single-file revision — a revised body whose hash differs
|
||||
#: from the fixture's (the importer's sha256 delta reports ``updated``)
|
||||
#: and which no other fixture text contains.
|
||||
ALPHA_A_REVISED_1 = (
|
||||
"Voice alpha fixture note A, REVISED-1: the only file test 1 changed "
|
||||
"— the change-gated regeneration must re-voice exactly this file's "
|
||||
"chain (alpha + the source root) under the new instruction. "
|
||||
"(RESE-FVOICE-01)"
|
||||
)
|
||||
|
||||
#: Test 2's single-file revision (a fresh distinctive body — the temp
|
||||
#: tree persists across the module, so the second revision is a delta
|
||||
#: against the state test 1's revision left behind).
|
||||
ALPHA_A_REVISED_2 = (
|
||||
"Voice alpha fixture note A, REVISED-2: the one changed file of "
|
||||
"test 2's sync — the regenerated folder whose stored voice the RAG "
|
||||
"catalogue must render. (RESE-FVOICE-02)"
|
||||
)
|
||||
|
||||
#: Test 3's single-file revision (``alpha-b.md`` — a different file, so
|
||||
#: the revision is a delta against test 3's own truncated KB).
|
||||
ALPHA_B_REVISED_3 = (
|
||||
"Voice alpha fixture note B, REVISED-3: the one changed file of "
|
||||
"test 3's sync — bravo's row must stay byte-identical through it. "
|
||||
"(RESE-FVOICE-03)"
|
||||
)
|
||||
|
||||
|
||||
def _md(title: str, body: str) -> str:
|
||||
return f"# {title}\n\n{body}\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def fvoice_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||
"""The one-source temp tree (see the module docstring): the app
|
||||
server runs on the same host, so the paths are visible to it. The
|
||||
directory NAME is the source name (``kind=local``, phase 38)."""
|
||||
root = tmp_path_factory.mktemp("bor_fvoice")
|
||||
src = root / SOURCE
|
||||
for folder in (FOLDER_A, FOLDER_B):
|
||||
(src / folder).mkdir(parents=True)
|
||||
for letter in ("a", "b"):
|
||||
(src / folder / f"{folder}-{letter}.md").write_text(
|
||||
_md(
|
||||
f"Voice {folder.title()} {letter.upper()}",
|
||||
f"Voice {folder} fixture note {letter.upper()}: covers "
|
||||
f"topic {letter.upper()} of the {SOURCE} source tree.",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(src / ROOT_NOTE).write_text(
|
||||
_md(
|
||||
"Voice Root Note",
|
||||
f"This file sits directly under the {SOURCE} source, not in "
|
||||
"any folder.",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
assert len(list(src.rglob("*.md"))) == N_FILES
|
||||
return src
|
||||
|
||||
|
||||
def _app_env(
|
||||
base_url: str, tmp_path_factory: pytest.TempPathFactory
|
||||
) -> dict[str, str]:
|
||||
"""The per-module app env (the conftest pattern, cf.
|
||||
``test_incremental_folder_summaries.py``): the ``base_url`` is the
|
||||
mock LLM directly (the fast leg is the only leg — see the module
|
||||
docstring), ``BOR_GIT_SOURCES`` forced empty (the sync source is
|
||||
this suite's own DB-registered local row), and the leak-guarded
|
||||
code defaults."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||
env["BOR_LLM_BASE_URL"] = (
|
||||
"https://aipi.reeseapps.com/v1" if USE_REAL_LLM else base_url
|
||||
)
|
||||
# Mock-calibrated threshold (conftest pattern): this suite never
|
||||
# asks the chat model anything — the gate is never on a path.
|
||||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||
# Phase 67: instant retry waits + the code-default budget (the
|
||||
# conftest leak-guard pattern).
|
||||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||||
env.setdefault(
|
||||
"BOR_DATABASE_URL",
|
||||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||
)
|
||||
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
|
||||
# reads it from cwd) — override it with an EMPTY value (the env var
|
||||
# beats the .env file): the registry must hold EXACTLY the local
|
||||
# directory this suite registers.
|
||||
env["BOR_GIT_SOURCES"] = ""
|
||||
# Leak guards (conftest pattern): an operator's local (gitignored)
|
||||
# .env cannot leak corpus-specific settings into the app under test.
|
||||
env["BOR_DOCS_REPO"] = ""
|
||||
env["BOR_SUGGESTIONS"] = json.dumps(
|
||||
_Settings.model_fields["suggestions"].default
|
||||
)
|
||||
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
|
||||
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
|
||||
env["BOR_SOURCES_DIR"] = str(tmp_path_factory.mktemp("bor_checkouts"))
|
||||
return env
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_server(
|
||||
mock_llm: int,
|
||||
fvoice_dir: Path,
|
||||
tmp_path_factory: pytest.TempPathFactory,
|
||||
) -> Iterator[str]:
|
||||
"""The app under test (all three tests): ``BOR_LLM_BASE_URL`` is
|
||||
the mock directly — the canned ``FOLDER_SUMMARY_MODE`` lines are
|
||||
byte-stable there, and the exact-text and shape assertions key on
|
||||
that exact text (no slow proxy: the assertions are post-terminal,
|
||||
see the module docstring). ``fvoice_dir`` is a dependency only for
|
||||
the fixture ordering (the temp tree exists before the app boots —
|
||||
the sync reads it)."""
|
||||
env = _app_env(f"http://127.0.0.1:{mock_llm}/v1", tmp_path_factory)
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||||
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
||||
cwd=REPO,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
_wait_http(f"{APP_URL}/api/health")
|
||||
yield APP_URL
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url(app_server: str) -> str:
|
||||
return app_server
|
||||
|
||||
|
||||
def _truncate_all() -> None:
|
||||
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
|
||||
share one Postgres, so a leftover git_sources row or document would
|
||||
pollute the candidate count and the counts the syncs and the
|
||||
folder-rows assertions pin exactly."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview, git_sources, folder_summaries"
|
||||
)
|
||||
)
|
||||
db.commit() # without the commit the TRUNCATE rolls back (the house pattern)
|
||||
|
||||
|
||||
def _wait_sync_settled(base_url: str, timeout_s: float = 180.0) -> None:
|
||||
"""No background sync may leak across tests (the run state lives in
|
||||
the app's memory, and a still-running run would keep importing into
|
||||
the NEXT test's truncated KB): wait for the app's sync status to
|
||||
be non-running BEFORE the truncate. Own admin session (the
|
||||
endpoint is admin-only) — usually a no-op: the tests settle only
|
||||
after their run's terminal state."""
|
||||
with httpx.Client(base_url=base_url, timeout=5.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, r.text
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
r = client.get("/api/sync/status")
|
||||
assert r.status_code == 200, r.text
|
||||
if r.json()["state"] != "running":
|
||||
return
|
||||
time.sleep(0.2)
|
||||
raise AssertionError(f"a sync was still running at a test boundary: {base_url}")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
|
||||
"""Per-test KB isolation (the phase-96/127 cleanup pattern): the
|
||||
shared E2E Postgres is truncated before AND after every test —
|
||||
each test seeds its own source and runs its own syncs, so the KB
|
||||
rows + folder_summaries rows of the temp source are exactly this
|
||||
test's own. The pre-wait keeps a leaked run from the previous test
|
||||
(the module app is shared) from racing the truncate."""
|
||||
_wait_sync_settled(app_url)
|
||||
_truncate_all()
|
||||
yield
|
||||
_truncate_all()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _register_source(base_url: str, path: str) -> None:
|
||||
"""Register the temp directory through the authenticated API (the
|
||||
``test_local_directory_sources.py`` / phase-94 pattern — a plain
|
||||
directory is an API/DB-only operation)."""
|
||||
with httpx.Client(base_url=base_url, timeout=30.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, r.text
|
||||
r = client.post(
|
||||
"/api/git-sources", json={"kind": "local", "path": path}
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
|
||||
|
||||
def _run_sync(base_url: str, timeout_s: float = 120.0) -> dict[str, Any]:
|
||||
"""Login + ``POST /api/sync`` + poll the status endpoint until THIS
|
||||
run reaches a terminal state (the ``test_local_directory_sources``
|
||||
pattern, over plain httpx — with the stale-terminal freshness gate:
|
||||
the module app is shared, so the PREVIOUS run's terminal lives in
|
||||
the app's memory until this run supersedes it; a terminal is
|
||||
accepted only after a running tick was observed or its
|
||||
``started_at`` post-dates this POST)."""
|
||||
with httpx.Client(base_url=base_url, timeout=30.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
assert r.status_code == 204, r.text
|
||||
t0 = time.time()
|
||||
r = client.post("/api/sync")
|
||||
assert r.status_code == 202, r.text
|
||||
saw_running = False
|
||||
deadline = time.monotonic() + timeout_s
|
||||
body: dict[str, Any] = {}
|
||||
while time.monotonic() < deadline:
|
||||
r = client.get("/api/sync/status")
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
state = body["state"]
|
||||
if state == "running":
|
||||
saw_running = True
|
||||
elif state in ("success", "failed") and (
|
||||
saw_running
|
||||
or _started_at(body) >= t0 - 1.0 # same-host clock; 1 s tolerance
|
||||
):
|
||||
return body
|
||||
time.sleep(0.1)
|
||||
raise AssertionError(f"sync did not reach a terminal state: {body}")
|
||||
|
||||
|
||||
def _started_at(body: dict[str, Any]) -> float:
|
||||
"""The status body's ``started_at`` as a wall-clock epoch (the
|
||||
server stamps it in UTC; this process runs on the same host —
|
||||
same clock, the phase-64 recorder's freshness convention)."""
|
||||
started = body.get("started_at")
|
||||
if not started:
|
||||
return 0.0
|
||||
try:
|
||||
return datetime.fromisoformat(str(started)).timestamp()
|
||||
except ValueError:
|
||||
return 0.0
|
||||
|
||||
|
||||
def _folder_rows() -> dict[str, tuple[str, bool, datetime]]:
|
||||
"""The source's stored folder summaries
|
||||
``{folder_path: (summary, manually_edited, updated_at)}``
|
||||
(``""`` = the source root) — the test process's direct DB access
|
||||
(the E2E's other established lens)."""
|
||||
with SessionLocal() as db:
|
||||
rows = db.execute(
|
||||
select(
|
||||
FolderSummary.folder_path,
|
||||
FolderSummary.summary,
|
||||
FolderSummary.manually_edited,
|
||||
FolderSummary.updated_at,
|
||||
).where(FolderSummary.source == SOURCE)
|
||||
).all()
|
||||
return {
|
||||
folder: (summary, manually, updated)
|
||||
for folder, summary, manually, updated in rows
|
||||
}
|
||||
|
||||
|
||||
def _first_sync(app_url: str, fvoice_dir: Path) -> dict[str, Any]:
|
||||
"""The suite's precondition (every test): register the temp source
|
||||
and run the first (full) sync under the deterministic mock —
|
||||
settled ``success`` with all 5 docs added and the 3 candidate rows
|
||||
generated (the phase-127 suite's sequence)."""
|
||||
_register_source(app_url, str(fvoice_dir))
|
||||
body = _run_sync(app_url)
|
||||
assert body["state"] == "success", body
|
||||
detail = body["detail"]
|
||||
assert detail["added"] == N_FILES, detail
|
||||
assert detail["updated"] == 0 and detail["unchanged"] == 0, detail
|
||||
assert detail["pruned"] == 0, detail
|
||||
assert detail["overview"] is True, detail
|
||||
return body
|
||||
|
||||
|
||||
def _revise_alpha_a(fvoice_dir: Path, body: str) -> None:
|
||||
"""Test 1's / test 2's KB change: ONE doc inside ``alpha/`` gets
|
||||
new content (the importer's sha256 delta reports ``updated`` — no
|
||||
add, no prune)."""
|
||||
target = fvoice_dir / FOLDER_A / f"{FOLDER_A}-a.md"
|
||||
target.write_text(_md(f"Voice {FOLDER_A.title()} A (revised)", body), encoding="utf-8")
|
||||
|
||||
|
||||
def _assert_change_terminal(body: dict[str, Any]) -> None:
|
||||
"""The second sync's terminal (the keep-final-counts convention):
|
||||
the AFFECTED count — alpha + the source root — the phase-127
|
||||
observable at the wire, plus the single-file detail."""
|
||||
assert body["state"] == "success", body
|
||||
detail = body["detail"]
|
||||
assert detail["added"] == 0, detail
|
||||
assert detail["updated"] == 1, detail
|
||||
assert detail["unchanged"] == N_FILES - 1, detail
|
||||
assert detail["pruned"] == 0, detail
|
||||
assert detail["overview"] is True, detail
|
||||
assert body["summaries_done"] == 2, body
|
||||
assert body["summaries_total"] == 2, body
|
||||
assert body["phase"] is None, body
|
||||
assert body["current_summary"] is None, body
|
||||
|
||||
|
||||
def _drill(page: Page, *names: str) -> None:
|
||||
"""Drill one level at a time (client-side — no fetch, no URL
|
||||
change): each name is the EXACT text of the source/folder link at
|
||||
the current level (the row builders' link text: the source name,
|
||||
or the folder's last path segment)."""
|
||||
for name in names:
|
||||
page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
|
||||
|
||||
|
||||
def _expect_desc_cell(
|
||||
page: Page, row_index: int, full_text: str
|
||||
) -> None:
|
||||
"""The RAG view's Description cell (the phase-97 catalogue idiom —
|
||||
``td:nth-child(4)``; the phase-99 one-line clamp): the cell's text
|
||||
span carries the STORED text (``textContent`` only — the house
|
||||
rule; the clamp is VISUAL only, so the full text stays in the DOM)
|
||||
and its ``title`` carries the FULL unclamped string (the hover
|
||||
escape hatch, ``makeDescCell`` — set whenever the description is
|
||||
non-empty, whether or not the display ellipsizes)."""
|
||||
span = page.locator(f"#folders-tbody tr:nth-child({row_index + 1}) "
|
||||
"td:nth-child(4) span")
|
||||
expect(span).to_have_text(full_text)
|
||||
expect(span).to_have_attribute("title", full_text)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 1. The change-gated regeneration (phase 127's path) stores the mock's
|
||||
# new-voice canned text — exact match, and the shape pin: none of
|
||||
# the forbidden openers
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_regen_summary_avoids_forbidden_openers(
|
||||
app_url: str, fvoice_dir: Path, db_ready: None
|
||||
) -> None:
|
||||
"""A first sync of the temp source settles ``success`` with all 5
|
||||
docs added; ONE doc inside ``alpha/`` is content-revised and the
|
||||
second sync (the CHANGE-GATED path — phase 127: the run's changed
|
||||
document set reaches ``generate_folder_summaries``) settles
|
||||
``success`` with the single-file detail and
|
||||
``summaries_done == summaries_total == 2`` (the AFFECTED count —
|
||||
alpha + the source root): the stored ``folder_summaries`` row for
|
||||
``alpha`` EQUALS the mock's canned reply for that folder (the
|
||||
EXACT string — the mock's line is a pure function of the request,
|
||||
so the match is byte-exact) AND does not start with "This folder
|
||||
contains" or "The voice/alpha folder contains" (the phase-155
|
||||
shape pin); the source-root row — the other affected row —
|
||||
likewise carries its exact canned line."""
|
||||
_first_sync(app_url, fvoice_dir)
|
||||
_revise_alpha_a(fvoice_dir, ALPHA_A_REVISED_1)
|
||||
|
||||
body = _run_sync(app_url)
|
||||
_assert_change_terminal(body)
|
||||
|
||||
rows = _folder_rows()
|
||||
assert set(rows) == {""} | {FOLDER_A, FOLDER_B}, rows
|
||||
# The regenerated folder: the EXACT canned string (deterministic) +
|
||||
# the shape pin (no forbidden opener).
|
||||
summary, manually, _updated_at = rows[FOLDER_A]
|
||||
assert summary == EXPECTED[FOLDER_A], (FOLDER_A, summary)
|
||||
assert _forbidden_hits(summary) == [], (FOLDER_A, summary)
|
||||
assert manually is False, (FOLDER_A, manually)
|
||||
# The other affected row (the source root) — the new voice there too.
|
||||
assert rows[""][0] == EXPECTED[""], rows[""]
|
||||
assert _forbidden_hits(rows[""][0]) == [], rows[""]
|
||||
# The unaffected sibling keeps its canned line (phase 127 — full
|
||||
# guard in test 3; here only the text, the shape pin's context).
|
||||
assert rows[FOLDER_B][0] == EXPECTED[FOLDER_B], rows[FOLDER_B]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 2. The RAG view (the phase-97 catalogue) renders the stored voice in
|
||||
# the Description cells — the exact canned text, title attribute
|
||||
# carrying the full string
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_rag_table_renders_the_stored_voice(
|
||||
page: Page,
|
||||
app_url: str,
|
||||
fvoice_dir: Path,
|
||||
db_ready: None,
|
||||
) -> None:
|
||||
"""The same sync-#1 + alpha-revision + sync-#2 sequence (the
|
||||
regenerated folder's row is re-stamped under the new instruction —
|
||||
the DB state test 1 pins), then the REAL admin login opens the
|
||||
RAG view (the phase-97 catalogue): the top-level source row's
|
||||
Description cell renders the stored source-root voice (the cell
|
||||
text AND its ``title`` — the phase-99 clamp keeps the full text in
|
||||
the DOM and on the hover escape hatch, so the assertion holds
|
||||
whether or not the display ellipsizes); drilling into the source,
|
||||
the ``alpha`` row's Description cell renders the regenerated
|
||||
folder's exact canned line (text + ``title``) and the ``bravo``
|
||||
row its own canned line, and the level block shows the stored
|
||||
source-root voice — the stored voice is what the catalogue
|
||||
renders, end to end."""
|
||||
page.set_default_timeout(30_000)
|
||||
_first_sync(app_url, fvoice_dir)
|
||||
_revise_alpha_a(fvoice_dir, ALPHA_A_REVISED_2)
|
||||
_assert_change_terminal(_run_sync(app_url))
|
||||
# The regenerated folder's DB state (the precondition the view
|
||||
# renders — test 1's exact-text pin, re-asserted here so the view
|
||||
# assertion stands on a verified store).
|
||||
rows = _folder_rows()
|
||||
assert rows[FOLDER_A][0] == EXPECTED[FOLDER_A], rows[FOLDER_A]
|
||||
|
||||
login(page, app_url) # → /sources.html (the RAG view)
|
||||
# The top level: the one source row (the ls() equivalence — the
|
||||
# tree's single fetch settled) and its stored source-root voice in
|
||||
# the Description cell (the exact canned line, text + title).
|
||||
page.wait_for_selector("#folders-tbody tr", timeout=30_000)
|
||||
expect(page.locator("#folders-tbody tr")).to_have_count(1)
|
||||
expect(page.locator("#folders-tbody tr td:nth-child(1) a.folder-link")
|
||||
).to_have_text(SOURCE)
|
||||
_expect_desc_cell(page, 0, EXPECTED[""])
|
||||
|
||||
# Drill into the source: the two folder rows (path order) and the
|
||||
# regenerated folder's stored voice in its Description cell — the
|
||||
# exact canned line (the DB state above is what the cell renders),
|
||||
# plus the sibling's own canned line; the level block carries the
|
||||
# source root's stored voice (the phase-97 ``#kb-level`` idiom).
|
||||
_drill(page, SOURCE)
|
||||
rows_visible = page.locator("#folders-tbody tr")
|
||||
expect(rows_visible).to_have_count(2)
|
||||
expect(rows_visible.nth(0).locator("a.folder-link")).to_have_text(FOLDER_A)
|
||||
_expect_desc_cell(page, 0, EXPECTED[FOLDER_A])
|
||||
expect(rows_visible.nth(1).locator("a.folder-link")).to_have_text(FOLDER_B)
|
||||
_expect_desc_cell(page, 1, EXPECTED[FOLDER_B])
|
||||
expect(page.locator("#kb-level")).to_be_visible()
|
||||
expect(page.locator("#kb-level-summary")).to_have_text(EXPECTED[""])
|
||||
# The rendered voice is the new one: no Description cell in the
|
||||
# level shows a forbidden opener (the shape pin, view-side).
|
||||
for cell in page.locator("#folders-tbody td:nth-child(4) span").all():
|
||||
rendered = cell.text_content() or ""
|
||||
for prefix in FORBIDDEN_PREFIXES:
|
||||
assert not rendered.startswith(prefix), rendered
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 3. The phase-127 regression guard: a KB change in folder A does not
|
||||
# touch folder B's stored row (text AND updated_at byte-identical)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_unchanged_folders_keep_their_rows(
|
||||
app_url: str, fvoice_dir: Path, db_ready: None
|
||||
) -> None:
|
||||
"""Re-seed (the autouse truncate) + sync #1 (full), capturing every
|
||||
row's ``(summary, manually_edited, updated_at)``; then ONE doc
|
||||
inside ``alpha/`` is content-revised (``alpha-b.md`` — a different
|
||||
file, so the revision is a delta against this test's own truncated
|
||||
KB) and sync #2 settles ``success`` with the single-file detail
|
||||
and ``summaries_done == summaries_total == 2`` (the AFFECTED
|
||||
count — alpha + the source root): the ``bravo`` row is
|
||||
BYTE-IDENTICAL — same ``summary`` AND same ``updated_at`` (a KB
|
||||
change in folder A does not touch folder B's stored row — the
|
||||
phase-127 contract), while the ``alpha`` AND the source-root rows
|
||||
are re-stamped (``updated_at`` strictly after the captured stamp,
|
||||
the canned text, still AI-written)."""
|
||||
_first_sync(app_url, fvoice_dir)
|
||||
before = _folder_rows()
|
||||
assert set(before) == {""} | {FOLDER_A, FOLDER_B}, before
|
||||
|
||||
target = fvoice_dir / FOLDER_A / f"{FOLDER_A}-b.md"
|
||||
target.write_text(
|
||||
_md(f"Voice {FOLDER_A.title()} B (revised)", ALPHA_B_REVISED_3),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
body = _run_sync(app_url)
|
||||
_assert_change_terminal(body)
|
||||
|
||||
after = _folder_rows()
|
||||
assert set(after) == {""} | {FOLDER_A, FOLDER_B}, after
|
||||
# The unchanged sibling: BYTE-IDENTICAL — same summary text AND
|
||||
# same updated_at (never re-stamped, never re-generated).
|
||||
assert after[FOLDER_B] == before[FOLDER_B], (
|
||||
FOLDER_B,
|
||||
before[FOLDER_B],
|
||||
after[FOLDER_B],
|
||||
)
|
||||
# The changed chain (alpha + the source root): RE-STAMPED — a new
|
||||
# stamp strictly after the captured one, the canned text, still
|
||||
# AI-written (the mock labels from the folder header only, so the
|
||||
# text is the same deterministic line).
|
||||
for folder in ("", FOLDER_A):
|
||||
summary, manually, updated_at = after[folder]
|
||||
assert summary == EXPECTED[folder], (folder, summary)
|
||||
assert manually is False, (folder, manually)
|
||||
assert updated_at > before[folder][2], (
|
||||
folder,
|
||||
before[folder],
|
||||
after[folder],
|
||||
)
|
||||
@@ -20,7 +20,7 @@ Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
|
||||
deterministic mock (the phase-94 ``FOLDER_SUMMARY_MODE`` canned
|
||||
``Fixture folder summary for <folder>.`` lines): the stored rows are a
|
||||
``Documentation for <folder>.`` lines): the stored rows are a
|
||||
pure function of the request, so the byte-identical assertions are
|
||||
exact only against the mock.
|
||||
|
||||
@@ -135,7 +135,7 @@ N_CANDIDATES = 5 # the source root + the four folders (the ≥ 1-doc rule)
|
||||
#: The mock's byte-stable ``FOLDER_SUMMARY_MODE`` lines for this fixture
|
||||
#: (the phase-94 template — the label is the ``FOLDER_HEADER_PREFIX``
|
||||
#: tail: ``<source>`` for the root, ``<source>/<folder>`` for a folder).
|
||||
SUMMARY_FOR = "Fixture folder summary for {}."
|
||||
SUMMARY_FOR = "Documentation for {}."
|
||||
EXPECTED: dict[str, str] = {
|
||||
"": SUMMARY_FOR.format(SOURCE),
|
||||
FOLDER_A: SUMMARY_FOR.format(f"{SOURCE}/{FOLDER_A}"),
|
||||
|
||||
@@ -29,7 +29,7 @@ stored description is deterministic: the mock's EXISTING
|
||||
stores, per existing folder (the ≥ 1 rule — this fixture holds no
|
||||
single-doc folders, so the pinned rows are unchanged), the canned
|
||||
one-liner naming the folder,
|
||||
``Fixture folder summary for <source>[/<folder>].`` — the ``synced_kb``
|
||||
``Documentation for <source>[/<folder>].`` — the ``synced_kb``
|
||||
module fixture pins those exact rows (all with
|
||||
``manually_edited = false``) after the sync, and the tests assert on
|
||||
that exact text.
|
||||
@@ -159,7 +159,7 @@ TOTAL_DOCS = ALPHA_COUNT + BETA_COUNT
|
||||
#: folder_path)`` order: one row per existing folder (the ≥ 1-doc
|
||||
#: recursive-subtree rule — this fixture holds no single-doc folders)
|
||||
#: — the ``""`` rows are the source roots.
|
||||
SUMMARY_FOR = "Fixture folder summary for {}."
|
||||
SUMMARY_FOR = "Documentation for {}."
|
||||
ALPHA_ROOT_SUM = SUMMARY_FOR.format(ALPHA)
|
||||
ONE_SUM = SUMMARY_FOR.format(f"{ALPHA}/one")
|
||||
TWO_SUM = SUMMARY_FOR.format(f"{ALPHA}/two")
|
||||
|
||||
@@ -39,9 +39,9 @@ Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the clamp test's
|
||||
level-block assertion keys on a MANUAL description (LLM-free), but the
|
||||
seeded KB's stored descriptions at the other levels are the mock's
|
||||
canned ``FOLDER_SUMMARY_MODE`` one-liner ``Fixture folder summary for
|
||||
<folder>.`` (the phase-94 convention — byte-exact only against the
|
||||
mock).
|
||||
canned ``FOLDER_SUMMARY_MODE`` one-liner ``Documentation for
|
||||
<folder>.`` (the phase-155 voice convention — byte-exact only against
|
||||
the mock).
|
||||
|
||||
KB fixture — a host temp dir (``tmp_path_factory``; the app runs on the
|
||||
same host) with ONE registered local source (the
|
||||
@@ -161,7 +161,7 @@ TOTAL_DOCS = 6 # 2 one/ + 2 one/two/ + 2 three/ (no root-level files)
|
||||
#: folder_path)`` order: one row per ≥ 2-doc folder (the
|
||||
#: recursive-subtree rule — ``one/two`` is a distinct nested folder) —
|
||||
#: the ``""`` row is the source root.
|
||||
SUMMARY_FOR = "Fixture folder summary for {}."
|
||||
SUMMARY_FOR = "Documentation for {}."
|
||||
ALPHA_ROOT_SUM = SUMMARY_FOR.format(ALPHA)
|
||||
ONE_SUM = SUMMARY_FOR.format(f"{ALPHA}/one")
|
||||
TWO_SUM = SUMMARY_FOR.format(f"{ALPHA}/one/two")
|
||||
|
||||
@@ -44,7 +44,8 @@ of the mock's cosine distribution, and the ``<tools>`` section is present
|
||||
The mock's canned ``FOLDER_SUMMARY_MODE`` branch (phase 94 task 01)
|
||||
stores, per existing folder (the ≥ 1-doc rule — this fixture holds
|
||||
no single-doc folders), the deterministic one-liner
|
||||
``Fixture folder summary for <source>[/<folder>].`` — the ``synced_kb``
|
||||
``Documentation for <source>[/<folder>].`` (phase 155 task 01's
|
||||
voice) — the ``synced_kb``
|
||||
module fixture pins those exact rows in ``folder_summaries`` after the
|
||||
sync, and the drill answers assert on them in the ``ls`` output.
|
||||
|
||||
@@ -168,12 +169,12 @@ assert "\n" not in TWO_A_CONTENT[:80] # the quote must stay one line
|
||||
#: ≥ 1-doc recursive-subtree rule — this fixture holds no single-doc
|
||||
#: folders) — the ``""`` rows are the source roots.
|
||||
EXPECTED_SUMMARIES: list[tuple[str, str, str]] = [
|
||||
(ALPHA, "", f"Fixture folder summary for {ALPHA}."),
|
||||
(ALPHA, "one", f"Fixture folder summary for {ALPHA}/one."),
|
||||
(ALPHA, "two", f"Fixture folder summary for {ALPHA}/two."),
|
||||
(ALPHA, "wide", f"Fixture folder summary for {ALPHA}/wide."),
|
||||
(BETA, "", f"Fixture folder summary for {BETA}."),
|
||||
(BETA, "gamma", f"Fixture folder summary for {BETA}/gamma."),
|
||||
(ALPHA, "", f"Documentation for {ALPHA}."),
|
||||
(ALPHA, "one", f"Documentation for {ALPHA}/one."),
|
||||
(ALPHA, "two", f"Documentation for {ALPHA}/two."),
|
||||
(ALPHA, "wide", f"Documentation for {ALPHA}/wide."),
|
||||
(BETA, "", f"Documentation for {BETA}."),
|
||||
(BETA, "gamma", f"Documentation for {BETA}/gamma."),
|
||||
]
|
||||
assert [
|
||||
(source, folder) for source, folder, _s in EXPECTED_SUMMARIES
|
||||
|
||||
@@ -54,7 +54,7 @@ Test → phase mapping (Playwright Mapping Rule):
|
||||
NOT — all 4 attempts empty → ``LLMError`` → per-folder fail-soft),
|
||||
a scripted ``ls oneshot`` turn asserts the drill-down listing: the
|
||||
``e2e_empty_once/`` line carries
|
||||
``: Fixture folder summary for oneshot/e2e_empty_once.`` (the retry
|
||||
``: Documentation for oneshot/e2e_empty_once.`` (the retry
|
||||
recovered the row), the ``normal/`` line carries its summary, and
|
||||
the ``e2e_empty_always/`` line is ``e2e_empty_always/ — 2
|
||||
documents`` with NO ``: …`` suffix — while the sync reported
|
||||
@@ -64,7 +64,7 @@ Test → phase mapping (Playwright Mapping Rule):
|
||||
``e2e_empty_once``'s row ``updated_at`` is captured, and the
|
||||
UNCHANGED sync #2 runs the gap gate (task 03): a second scripted
|
||||
``ls oneshot`` turn shows ``normal/`` healed
|
||||
(``: Fixture folder summary for oneshot/normal.`` again) and
|
||||
(``: Documentation for oneshot/normal.`` again) and
|
||||
``e2e_empty_always/`` still absent (the gap-fill attempted it,
|
||||
exhausted, stayed absent — the sync still succeeded), and the DB
|
||||
pins the targeted fill: the ``normal`` row is back with its
|
||||
@@ -141,10 +141,10 @@ def _md(title: str, body: str) -> str:
|
||||
#: fixture (the phase-94 template — the label is the
|
||||
#: ``FOLDER_HEADER_PREFIX`` tail: ``<source>`` for the root,
|
||||
#: ``<source>/<folder>`` for a folder).
|
||||
SUM_ROOT = f"Fixture folder summary for {SOURCE}."
|
||||
SUM_ONCE = f"Fixture folder summary for {SOURCE}/{FOLDER_ONCE}."
|
||||
SUM_ALWAYS = f"Fixture folder summary for {SOURCE}/{FOLDER_ALWAYS}."
|
||||
SUM_NORMAL = f"Fixture folder summary for {SOURCE}/{FOLDER_NORMAL}."
|
||||
SUM_ROOT = f"Documentation for {SOURCE}."
|
||||
SUM_ONCE = f"Documentation for {SOURCE}/{FOLDER_ONCE}."
|
||||
SUM_ALWAYS = f"Documentation for {SOURCE}/{FOLDER_ALWAYS}."
|
||||
SUM_NORMAL = f"Documentation for {SOURCE}/{FOLDER_NORMAL}."
|
||||
|
||||
# --- the pinned ``ls oneshot`` level (app.rag.agent's phase-94 template)
|
||||
|
||||
@@ -560,7 +560,7 @@ def test_deleted_row_self_heals_on_unchanged_sync(
|
||||
other row stays byte-identical (text AND ``updated_at``).
|
||||
|
||||
* ``normal`` — healed: the second ``ls oneshot`` turn carries
|
||||
``: Fixture folder summary for oneshot/normal.`` again, and the
|
||||
``: Documentation for oneshot/normal.`` again, and the
|
||||
DB row is back with its deterministic text;
|
||||
* ``e2e_empty_once`` — the gap-fill NEVER calls it (the row
|
||||
exists): its ``updated_at`` is UNCHANGED (a full regeneration
|
||||
|
||||
@@ -16,7 +16,7 @@ Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
|
||||
deterministic mock (the phase-94 ``FOLDER_SUMMARY_MODE`` canned
|
||||
``Fixture folder summary for <folder>.`` lines) plus the phase-64
|
||||
``Documentation for <folder>.`` lines) plus the phase-64
|
||||
``tests/e2e/slow_llm.py`` delay proxy that stretches the run past both
|
||||
the ~100 ms recorder cadence and the UI's 2 s poll.
|
||||
|
||||
@@ -169,9 +169,9 @@ N_CANDIDATES = 3 # the source root + the two folders (the ≥ 1-doc rule)
|
||||
#: The mock's byte-stable ``FOLDER_SUMMARY_MODE`` lines for this fixture
|
||||
#: (the phase-94 template — the label is the ``FOLDER_HEADER_PREFIX``
|
||||
#: tail: ``<source>`` for the root, ``<source>/<folder>`` for a folder).
|
||||
SUM_ROOT = f"Fixture folder summary for {SOURCE}."
|
||||
SUM_ALPHA = f"Fixture folder summary for {SOURCE}/{FOLDER_A}."
|
||||
SUM_BRavo = f"Fixture folder summary for {SOURCE}/{FOLDER_B}."
|
||||
SUM_ROOT = f"Documentation for {SOURCE}."
|
||||
SUM_ALPHA = f"Documentation for {SOURCE}/{FOLDER_A}."
|
||||
SUM_BRavo = f"Documentation for {SOURCE}/{FOLDER_B}."
|
||||
|
||||
#: The D4 "Summary pending" marker copy (phase 98 task 04 — verbatim
|
||||
#: from the D4 decision): the row-cell text, the row-cell ``title``,
|
||||
|
||||
@@ -0,0 +1,267 @@
|
||||
"""Integration test: ``regen_folder_summaries`` runs the FULL-mode one-shot
|
||||
regeneration (phase 155, task 02).
|
||||
|
||||
Drives ``scripts.regen_folder_summaries.main()`` end to end against the
|
||||
local compose Postgres with a deterministic fake LLM (no live aipi,
|
||||
fresh settings — the phase 31 integration's mocking style), over a
|
||||
fixture catalogue (2 sources, 3 candidate folders: the ``Alpha`` source
|
||||
root, ``Alpha/sub``, the ``Beta`` source root) whose ``folder_summaries``
|
||||
rows are pre-stored with the formulaic pre-nudge voice and a backdated
|
||||
``updated_at``:
|
||||
|
||||
- a full-mode run upserts EVERY candidate row — the text is the
|
||||
fake's deterministic reply for all of them and EVERY row's
|
||||
``updated_at`` advances (a pure regen); the catalogue is
|
||||
unchanged, so the count-driven prune rule fires for nothing
|
||||
(``pruned=0``);
|
||||
- the single cron-safe summary line logs the stats dict
|
||||
(``regen_folder_summaries: folders=… failed=… pruned=…
|
||||
kept_manual=… limit=…``);
|
||||
- one ``lite`` call per candidate folder (a full regeneration burns a
|
||||
call per folder; the ``FOLDER_SUMMARY_MODE`` marker + the phase-155
|
||||
instruction are in the system prompt — the mock contract is intact
|
||||
end to end);
|
||||
- a ``manually_edited`` row is SKIPPED (no ``lite`` burn on owner
|
||||
text, the row stays byte-identical — text AND ``updated_at``) and
|
||||
is named in the stats (``kept_manual=1``);
|
||||
- a full ``lite`` failure is fail-soft PER FOLDER (phase 94): every
|
||||
previous row is kept (text AND ``updated_at``), the run's exit
|
||||
code stays 0, and the failure counts on the summary line
|
||||
(``folders=0 failed=3``) — the exit code is about the run (boot,
|
||||
DB, commit), never about per-folder model failures.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings
|
||||
from app.db import db_available
|
||||
from app.models import Document, FolderSummary
|
||||
from app.rag.llm import LLMError
|
||||
from scripts import regen_folder_summaries as regen
|
||||
from tests.fakes import FakeEmbedder
|
||||
|
||||
#: The fixture catalogue: (source, path, title, summary) rows.
|
||||
CATALOGUE: list[tuple[str, str, str, str]] = [
|
||||
("Alpha", "a.md", "A One", "Summary of A"),
|
||||
("Alpha", "b.md", "B Two", "Summary of B"),
|
||||
("Alpha", "sub/c.md", "C Three", "Summary of C"),
|
||||
("Beta", "x.md", "X One", "Summary of X"),
|
||||
]
|
||||
|
||||
#: The candidate folders (recursive subtree ≥ 1 doc, per source):
|
||||
#: the two source roots + the ``Alpha/sub`` folder.
|
||||
CANDIDATES: list[tuple[str, str]] = [
|
||||
("Alpha", ""),
|
||||
("Alpha", "sub"),
|
||||
("Beta", ""),
|
||||
]
|
||||
|
||||
#: The formulaic pre-nudge voice the pre-stored rows carry (the
|
||||
#: live catalogue's shape — what the one-shot exists to replace).
|
||||
OLD_TEXT = "This folder contains the project's notes and setup guides."
|
||||
|
||||
|
||||
class FailingChatEmbedder(FakeEmbedder):
|
||||
"""A ``lite`` model that always fails (drives the per-folder
|
||||
fail-soft path)."""
|
||||
|
||||
async def chat(self, messages: list[dict[str, str]], model: str | None = None) -> str:
|
||||
self.chat_calls.append(list(messages))
|
||||
raise LLMError("simulated lite-model failure (test sentinel)")
|
||||
|
||||
|
||||
def _seed(db: Session, old: datetime) -> None:
|
||||
"""Seed the fixture catalogue + the pre-stored folder-summary
|
||||
rows (the formulaic voice, backdated ``updated_at``)."""
|
||||
db.add_all(
|
||||
Document(
|
||||
source=source,
|
||||
path=path,
|
||||
full_path=f"/fixture/{source}/{path}",
|
||||
title=title,
|
||||
content=f"# {title}\n\nBody.\n",
|
||||
content_hash=f"{i:064d}"[:64],
|
||||
summary=summary,
|
||||
)
|
||||
for i, (source, path, title, summary) in enumerate(CATALOGUE)
|
||||
)
|
||||
db.add_all(
|
||||
FolderSummary(
|
||||
source=source,
|
||||
folder_path=folder_path,
|
||||
summary=OLD_TEXT,
|
||||
manually_edited=False,
|
||||
updated_at=old,
|
||||
)
|
||||
for source, folder_path in CANDIDATES
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _run_main(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
llm: FakeEmbedder,
|
||||
argv: list[str],
|
||||
) -> tuple[int, list[str]]:
|
||||
"""Run ``regen_folder_summaries.main`` with fresh settings and a
|
||||
fake LLM; returns the exit code + the module logger's summary
|
||||
lines (the single cron-safe line, PLAN §9)."""
|
||||
monkeypatch.setattr(
|
||||
regen, "get_settings", lambda: Settings(_env_file=None) # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
monkeypatch.setattr(regen, "LLMClient", lambda: llm)
|
||||
records: list[logging.LogRecord] = []
|
||||
|
||||
class _Sink(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
records.append(record)
|
||||
|
||||
sink = _Sink()
|
||||
log = logging.getLogger("scripts.regen_folder_summaries")
|
||||
log.addHandler(sink)
|
||||
log.setLevel(logging.INFO)
|
||||
try:
|
||||
rc = regen.main(argv)
|
||||
finally:
|
||||
log.removeHandler(sink)
|
||||
lines = [
|
||||
r.getMessage()
|
||||
for r in records
|
||||
if r.getMessage().startswith("regen_folder_summaries:")
|
||||
]
|
||||
return rc, lines
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_kb(db: Session) -> Iterator[None]:
|
||||
if not db_available():
|
||||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||||
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
|
||||
db.commit()
|
||||
yield
|
||||
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_full_regen_upserts_every_candidate_row(
|
||||
db: Session,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
old = datetime.now(UTC) - timedelta(days=1)
|
||||
_seed(db, old)
|
||||
llm = FakeEmbedder()
|
||||
|
||||
rc, lines = _run_main(monkeypatch, llm, [])
|
||||
|
||||
assert rc == 0
|
||||
# The single cron-safe summary line: the stats dict, one line.
|
||||
assert lines == [
|
||||
"regen_folder_summaries: folders=3 failed=0 pruned=0 "
|
||||
"kept_manual=0 limit=none"
|
||||
]
|
||||
# A full regeneration burns exactly ONE lite call per candidate
|
||||
# folder (nothing else may touch chat).
|
||||
assert len(llm.chat_calls) == 3
|
||||
# The mock contract is intact end to end: the FOLDER_SUMMARY_MODE
|
||||
# marker + the phase-155 instruction (the two forbidden openers as
|
||||
# the constraint) are in every system prompt, and the user
|
||||
# message's first line names the folder (FOLDER_HEADER_PREFIX).
|
||||
expected_user_heads = ["Folder: Alpha", "Folder: Alpha/sub", "Folder: Beta"]
|
||||
for call, head in zip(llm.chat_calls, expected_user_heads, strict=True):
|
||||
by_role = {m["role"]: m["content"] for m in call}
|
||||
assert "FOLDER_SUMMARY_MODE" in by_role["system"]
|
||||
assert "Do NOT begin with 'This folder contains'" in by_role["system"]
|
||||
assert by_role["user"].splitlines()[0] == head
|
||||
# EVERY candidate row was upserted: the text is the fake's
|
||||
# deterministic reply (a pure function of the request — the same
|
||||
# "Summary of Folder:" for all three, the new voice for all of
|
||||
# them), and EVERY row's updated_at advanced past the backdate.
|
||||
db.expire_all()
|
||||
for source, folder_path in CANDIDATES:
|
||||
row = db.get(FolderSummary, (source, folder_path))
|
||||
assert row is not None
|
||||
assert row.summary == "Summary of Folder:"
|
||||
assert row.updated_at > old
|
||||
# No prune: the catalogue is unchanged, so the count-driven prune
|
||||
# rule fires for nothing — the row count is unchanged.
|
||||
assert db.execute(text("SELECT count(*) FROM folder_summaries")).scalar() == 3
|
||||
|
||||
|
||||
def test_manual_row_is_kept_and_named_in_stats(
|
||||
db: Session,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The owner's words are never regenerated (phase 97, task 01):
|
||||
the manual row stays byte-identical (text AND ``updated_at`` —
|
||||
no ``lite`` burn), the other candidate rows advance, and the skip
|
||||
is named in the stats (``kept_manual=1``)."""
|
||||
old = datetime.now(UTC) - timedelta(days=1)
|
||||
_seed(db, old)
|
||||
manual = db.get(FolderSummary, ("Alpha", ""))
|
||||
assert manual is not None
|
||||
manual.manually_edited = True
|
||||
manual.summary = "Owner words — the drill-down picture I wrote."
|
||||
db.commit()
|
||||
# Capture the manual row's stamp AFTER the seed (the column's
|
||||
# onupdate re-stamped it on the flag-setting UPDATE) — the
|
||||
# script must leave it byte-identical.
|
||||
manual = db.get(FolderSummary, ("Alpha", ""))
|
||||
assert manual is not None
|
||||
manual_stamp = manual.updated_at
|
||||
llm = FakeEmbedder()
|
||||
|
||||
rc, lines = _run_main(monkeypatch, llm, [])
|
||||
|
||||
assert rc == 0
|
||||
assert lines == [
|
||||
"regen_folder_summaries: folders=2 failed=0 pruned=0 "
|
||||
"kept_manual=1 limit=none"
|
||||
]
|
||||
# No lite burn on owner text: only the two non-manual candidates.
|
||||
assert len(llm.chat_calls) == 2
|
||||
db.expire_all()
|
||||
row = db.get(FolderSummary, ("Alpha", ""))
|
||||
assert row is not None
|
||||
assert row.summary == "Owner words — the drill-down picture I wrote."
|
||||
assert row.updated_at == manual_stamp # byte-identical — never re-stamped
|
||||
for source, folder_path in [("Alpha", "sub"), ("Beta", "")]:
|
||||
row = db.get(FolderSummary, (source, folder_path))
|
||||
assert row is not None
|
||||
assert row.summary == "Summary of Folder:"
|
||||
assert row.updated_at > old
|
||||
|
||||
|
||||
def test_lite_failure_keeps_previous_rows_and_stays_exit_zero(
|
||||
db: Session,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A full ``lite`` outage is fail-soft PER FOLDER (phase 94): the
|
||||
previous rows are kept (text AND ``updated_at`` — an old summary
|
||||
is better than none), the exit code stays 0 (the run itself was
|
||||
fine — boot, DB, commit all succeeded), and the failure counts on
|
||||
the summary line (``folders=0 failed=3``)."""
|
||||
old = datetime.now(UTC) - timedelta(days=1)
|
||||
_seed(db, old)
|
||||
llm = FailingChatEmbedder()
|
||||
|
||||
rc, lines = _run_main(monkeypatch, llm, [])
|
||||
|
||||
assert rc == 0 # a failed folder batch must not fail the run
|
||||
assert lines == [
|
||||
"regen_folder_summaries: folders=0 failed=3 pruned=0 "
|
||||
"kept_manual=0 limit=none"
|
||||
]
|
||||
assert len(llm.chat_calls) == 3 # one attempt per candidate folder
|
||||
db.expire_all()
|
||||
for source, folder_path in CANDIDATES:
|
||||
row = db.get(FolderSummary, (source, folder_path))
|
||||
assert row is not None
|
||||
assert row.summary == OLD_TEXT # previous row kept
|
||||
assert row.updated_at == old # never re-stamped on failure
|
||||
@@ -272,6 +272,40 @@ def test_system_prompt_has_marker_and_locked_instruction() -> None:
|
||||
assert FOLDER_SUMMARY_MODE in system # the marker the E2E mock keys on
|
||||
|
||||
|
||||
def test_instruction_pins_the_variety_nudge_and_preserved_constraints() -> None:
|
||||
"""Phase 155 task 01: the locked instruction keeps the four phase-94
|
||||
constraints (1–3 sentences; plain text; no markdown; no invention)
|
||||
and ADDS the variety direction — the two forbidden openers are the
|
||||
pin (an explicit NOT, not a mere mention)."""
|
||||
# The four preserved constraints (phase 94 — unchanged).
|
||||
for fragment in (
|
||||
"1-3 sentence",
|
||||
"plain-text",
|
||||
"natural language",
|
||||
"Do not use markdown",
|
||||
"Do not invent anything that is not in the list",
|
||||
):
|
||||
assert fragment in FOLDER_SUMMARY_INSTRUCTION, fragment
|
||||
# The variety direction (phase 155): the "varied, natural way"
|
||||
# direction must survive the grammar polish.
|
||||
assert "Open in a varied, natural way" in FOLDER_SUMMARY_INSTRUCTION
|
||||
assert "state what the folder is for" in FOLDER_SUMMARY_INSTRUCTION
|
||||
# The two forbidden openers, banned by an explicit NOT.
|
||||
assert "Do NOT begin with" in FOLDER_SUMMARY_INSTRUCTION
|
||||
assert "'This folder contains'" in FOLDER_SUMMARY_INSTRUCTION
|
||||
assert "'The <folder name> folder contains'" in FOLDER_SUMMARY_INSTRUCTION
|
||||
|
||||
|
||||
def test_marker_and_header_prefix_stay_byte_identical() -> None:
|
||||
"""Phase 155 task 01: the E2E mock keys on ``FOLDER_SUMMARY_MODE``
|
||||
and the ``FOLDER_HEADER_PREFIX`` line — the instruction nudge must
|
||||
not touch them (the mock-contract pin).``SYSTEM_PROMPT`` keeps the
|
||||
marker-first shape: the nudge lands AFTER the marker."""
|
||||
assert FOLDER_SUMMARY_MODE == "FOLDER_SUMMARY_MODE"
|
||||
assert FOLDER_HEADER_PREFIX == "Folder: "
|
||||
assert SYSTEM_PROMPT.startswith("FOLDER_SUMMARY_MODE: ")
|
||||
|
||||
|
||||
# ---------- build_folder_summary_prompt: user ----------
|
||||
|
||||
|
||||
@@ -369,22 +403,55 @@ def test_mock_returns_canned_folder_summary_naming_the_folder() -> None:
|
||||
marker in the system prompt and returns the canned one-liner naming
|
||||
the folder from the ``Folder: …`` header — driven through the
|
||||
GENERATOR's real prompt, so the two can never drift (the drill-down
|
||||
E2E asserts on this exact template)."""
|
||||
E2E asserts on this exact template). Phase 155 task 01: the canned
|
||||
voice is the natural-opener ``Documentation for <folder>.``."""
|
||||
system, user = build_folder_summary_prompt(
|
||||
"Homelab", "deployments/ansible",
|
||||
[("Homelab", "deployments/ansible/lab-inventory.md", "Lab Inventory", None)],
|
||||
)
|
||||
assert compose_answer(_mock_body(system, user)) == (
|
||||
"Fixture folder summary for Homelab/deployments/ansible."
|
||||
"Documentation for Homelab/deployments/ansible."
|
||||
)
|
||||
# The source-root row names the source itself.
|
||||
system, user = build_folder_summary_prompt("Homelab", "",
|
||||
[("Homelab", "top.md", "Top", None)])
|
||||
assert compose_answer(_mock_body(system, user)) == (
|
||||
"Fixture folder summary for Homelab."
|
||||
"Documentation for Homelab."
|
||||
)
|
||||
|
||||
|
||||
def test_mock_canned_folder_reply_shape_and_determinism() -> None:
|
||||
"""Phase 155 task 01: the canned folder reply is the new voice —
|
||||
it NAMES the folder (from the ``Folder: …`` header), AVOIDS the
|
||||
two forbidden openers, and is a PURE function of the request (the
|
||||
determinism contract: two identical requests → identical replies).
|
||||
"""
|
||||
def reply_for(source: str, folder_path: str) -> str:
|
||||
system, user = build_folder_summary_prompt(
|
||||
source, folder_path, [(source, "x.md", "X", None)]
|
||||
)
|
||||
return compose_answer(_mock_body(system, user))
|
||||
|
||||
first = reply_for("Homelab", "deployments/ansible")
|
||||
second = reply_for("Homelab", "deployments/ansible") # same request → same reply
|
||||
assert first == second, "pure function of the request (determinism contract)"
|
||||
assert "Homelab/deployments/ansible" in first, "the reply names the folder"
|
||||
# The two forbidden openers (phase 155's pin), case-insensitively.
|
||||
assert not first.lower().startswith("this folder contains"), (
|
||||
"the formulaic opener must not survive the canned reply"
|
||||
)
|
||||
assert not first.lower().startswith(
|
||||
"the homelab/deployments/ansible folder contains"
|
||||
), "the named-folder formulaic opener must not survive either"
|
||||
# A different folder yields a different (but same-shape) reply —
|
||||
# still names its own folder, still neither forbidden opener.
|
||||
root = reply_for("Homelab", "")
|
||||
assert root != first, "the reply is a function of the request, not a constant"
|
||||
assert "Homelab" in root
|
||||
assert not root.lower().startswith("this folder contains")
|
||||
assert not root.lower().startswith("the homelab folder contains")
|
||||
|
||||
|
||||
def test_mock_folder_marker_is_not_shadowed_by_the_summary_branch() -> None:
|
||||
"""``FOLDER_SUMMARY_MODE`` contains ``SUMMARY_MODE`` as a substring —
|
||||
the mock must check the folder branch FIRST, or every folder call
|
||||
@@ -394,7 +461,7 @@ def test_mock_folder_marker_is_not_shadowed_by_the_summary_branch() -> None:
|
||||
)
|
||||
assert "SUMMARY_MODE" in system # the shadowing hazard is real
|
||||
answer = compose_answer(_mock_body(system, user))
|
||||
assert answer == "Fixture folder summary for S/a."
|
||||
assert answer == "Documentation for S/a."
|
||||
assert not answer.startswith("This document covers")
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Unit tests: ``scripts/regen_folder_summaries.py`` wiring (phase 155, task 02).
|
||||
|
||||
The one-shot full-regeneration entrypoint is pinned by wiring, not by a
|
||||
live LLM or DB: a fake ``LLMClient``, a fake short-lived session, and a
|
||||
stubbed ``generate_folder_summaries`` that records its call. The pins:
|
||||
|
||||
* the FULL-mode call contract — the stub is called with NO kwargs
|
||||
(``changed_docs=None`` / ``only_missing=False`` / ``skip=False`` are
|
||||
the generator's defaults; the phase-127 change-gated sync path is
|
||||
untouched, the script is a consumer of the full mode);
|
||||
* the stats dict surfaces on the single cron-safe summary line
|
||||
(``regen_folder_summaries: folders=… failed=… pruned=…
|
||||
kept_manual=… limit=…``) on the module logger;
|
||||
* the ``--limit`` debug passthrough (parity with ``import_docs``'s
|
||||
parser) is named on the summary line without disturbing the
|
||||
generator's call contract;
|
||||
* the settings/boot failure exits non-zero with the NAMED stderr line
|
||||
— the app validator idiom (``ensure_admin_configured`` names the
|
||||
missing ``BOR_`` variable; a settings validation error names the
|
||||
field); the LLM/session/generator never run on a boot failure;
|
||||
* a DB/commit outage exits non-zero with the named stderr line (never
|
||||
a silent partial — the session still closes, no summary line);
|
||||
* ``main()`` calls ``configure_debugging()`` (the phase-8 idiom — the
|
||||
``DEBUGPY=1`` listener contract itself is the phase-8 precedent,
|
||||
``tests/unit/test_debugging.py``).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
|
||||
from app.config import Settings
|
||||
from scripts import regen_folder_summaries as regen
|
||||
|
||||
STATS = {"generated": 3, "failed": 1, "pruned": 2, "kept_manual": 1}
|
||||
|
||||
|
||||
class _FakeLLM:
|
||||
"""A duck-typed ``LLMClient`` stand-in (records construction)."""
|
||||
|
||||
|
||||
class _FakeSession:
|
||||
"""A short-lived session stub (phase-53 convention: the script's
|
||||
own session commits; the generator only flushes)."""
|
||||
|
||||
def __init__(self, commit_error: Exception | None = None) -> None:
|
||||
self.commit_error = commit_error
|
||||
self.committed = False
|
||||
self.closed = False
|
||||
|
||||
def commit(self) -> None:
|
||||
if self.commit_error is not None:
|
||||
raise self.commit_error
|
||||
self.committed = True
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class _GenerateStub:
|
||||
"""The stubbed ``generate_folder_summaries``: records the call's
|
||||
positional args and kwargs and returns the fixed stats dict."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.calls: list[tuple[tuple[object, ...], dict[str, object]]] = []
|
||||
self.raised: Exception | None = None
|
||||
|
||||
async def __call__(
|
||||
self, session: object, llm: object, **kwargs: object
|
||||
) -> dict[str, int]:
|
||||
self.calls.append(((session, llm), dict(kwargs)))
|
||||
if self.raised is not None:
|
||||
raise self.raised
|
||||
return dict(STATS)
|
||||
|
||||
|
||||
def _wire(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> tuple[_FakeSession, _GenerateStub, list[logging.LogRecord]]:
|
||||
"""Boot the wiring with a fake LLM, fake session, and generator
|
||||
stub; ``configure_logging``/``configure_debugging`` are spied (the
|
||||
phase-8 idiom is pinned by the call, the listener contract by the
|
||||
phase-8 precedent file)."""
|
||||
session = _FakeSession()
|
||||
stub = _GenerateStub()
|
||||
llm = _FakeLLM()
|
||||
|
||||
monkeypatch.setattr(
|
||||
regen,
|
||||
"get_settings",
|
||||
lambda: Settings(_env_file=None), # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
monkeypatch.setattr(regen, "configure_logging", lambda level: None)
|
||||
debugging_calls: list[int] = []
|
||||
monkeypatch.setattr(
|
||||
regen,
|
||||
"configure_debugging",
|
||||
lambda: debugging_calls.append(1) or False,
|
||||
)
|
||||
monkeypatch.setattr(regen, "LLMClient", lambda: llm)
|
||||
monkeypatch.setattr(regen, "SessionLocal", lambda: session)
|
||||
monkeypatch.setattr(regen, "generate_folder_summaries", stub)
|
||||
|
||||
records: list[logging.LogRecord] = []
|
||||
|
||||
class _Sink(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
records.append(record)
|
||||
|
||||
sink = _Sink()
|
||||
regen_logger = logging.getLogger("scripts.regen_folder_summaries")
|
||||
regen_logger.addHandler(sink)
|
||||
regen_logger.setLevel(logging.INFO)
|
||||
return session, stub, records
|
||||
|
||||
|
||||
def _summary_lines(records: list[logging.LogRecord]) -> list[str]:
|
||||
return [r.getMessage() for r in records if r.getMessage().startswith(
|
||||
"regen_folder_summaries:"
|
||||
)]
|
||||
|
||||
|
||||
# ---------- the parser (the import_docs parity flag) ----------
|
||||
|
||||
|
||||
def test_parser_defaults_to_no_limit() -> None:
|
||||
args = regen.build_parser().parse_args([])
|
||||
assert args.limit is None
|
||||
|
||||
|
||||
def test_parser_limit_parity_flag() -> None:
|
||||
args = regen.build_parser().parse_args(["--limit", "5"])
|
||||
assert args.limit == 5
|
||||
|
||||
|
||||
# ---------- the success path: the FULL-mode call contract ----------
|
||||
|
||||
|
||||
def test_main_full_mode_defaults_and_stats_on_summary_line(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
session, stub, records = _wire(monkeypatch)
|
||||
rc = regen.main([])
|
||||
out = capsys.readouterr()
|
||||
|
||||
assert rc == 0
|
||||
assert out.err == "" # success: no stderr noise
|
||||
# The FULL-mode call contract: exactly one call, (session, llm)
|
||||
# positional, NO kwargs — changed_docs=None / only_missing=False /
|
||||
# skip=False are the generator's defaults (the phase-127 change-
|
||||
# gated sync path is untouched — the script is a consumer of the
|
||||
# default mode, not of changed_docs).
|
||||
assert len(stub.calls) == 1
|
||||
(pos, kwargs), = stub.calls
|
||||
assert kwargs == {}
|
||||
assert isinstance(pos[0], _FakeSession) # the short-lived session
|
||||
assert isinstance(pos[1], _FakeLLM) # the LLMClient stand-in
|
||||
# The phase-53 convention: the script's own session commits + closes.
|
||||
assert session.committed is True
|
||||
assert session.closed is True
|
||||
# The stats dict surfaces on the single cron-safe summary line.
|
||||
lines = _summary_lines(records)
|
||||
assert len(lines) == 1
|
||||
assert lines[0] == (
|
||||
"regen_folder_summaries: folders=3 failed=1 pruned=2 "
|
||||
"kept_manual=1 limit=none"
|
||||
)
|
||||
|
||||
|
||||
def test_main_debug_parsing_runs_full_mode_and_names_limit(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""The --limit debug passthrough (parity with import_docs): named
|
||||
on the summary line WITHOUT disturbing the generator's call
|
||||
contract — the full-mode defaults are still the no-kwargs call."""
|
||||
_session, stub, records = _wire(monkeypatch)
|
||||
rc = regen.main(["--limit", "7"])
|
||||
assert rc == 0
|
||||
assert len(stub.calls) == 1
|
||||
assert stub.calls[0][1] == {} # the full-mode defaults, untouched
|
||||
assert _summary_lines(records) == [
|
||||
"regen_folder_summaries: folders=3 failed=1 pruned=2 "
|
||||
"kept_manual=1 limit=7"
|
||||
]
|
||||
|
||||
|
||||
def test_main_calls_configure_debugging(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""The phase-8 wiring pin: main() calls configure_debugging()
|
||||
(DEBUGPY=1 honored, default off — the listener contract itself is
|
||||
the phase-8 precedent, tests/unit/test_debugging.py)."""
|
||||
called: list[int] = []
|
||||
session, _stub, _records = _wire(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
regen, "configure_debugging", lambda: called.append(1) or False
|
||||
)
|
||||
assert regen.main([]) == 0
|
||||
assert called == [1]
|
||||
assert session.committed is True
|
||||
|
||||
|
||||
# ---------- the fail-loud boot (the app validator idiom) ----------
|
||||
|
||||
|
||||
def test_main_boot_failure_missing_admin_secret_is_named(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""A missing BOR_ADMIN_PASSWORD: the app's fail-loud gate
|
||||
(ensure_admin_configured) names the variable, main exits non-zero
|
||||
with the named stderr line, and the LLM/session/generator NEVER
|
||||
run (never a silent partial)."""
|
||||
monkeypatch.setattr(
|
||||
regen,
|
||||
"get_settings",
|
||||
lambda: Settings(_env_file=None, admin_password=""), # pyright: ignore[reportCallIssue]
|
||||
)
|
||||
monkeypatch.setattr(regen, "configure_logging", lambda level: None)
|
||||
monkeypatch.setattr(regen, "configure_debugging", lambda: False)
|
||||
|
||||
llm_calls: list[int] = []
|
||||
monkeypatch.setattr(
|
||||
regen, "LLMClient", lambda: llm_calls.append(1) or _FakeLLM()
|
||||
)
|
||||
session = _FakeSession()
|
||||
monkeypatch.setattr(regen, "SessionLocal", lambda: session)
|
||||
stub = _GenerateStub()
|
||||
monkeypatch.setattr(regen, "generate_folder_summaries", stub)
|
||||
|
||||
rc = regen.main([])
|
||||
err = capsys.readouterr().err
|
||||
|
||||
assert rc == 1
|
||||
assert "regen_folder_summaries: boot failed:" in err
|
||||
assert "BOR_ADMIN_PASSWORD" in err # the gate names the variable
|
||||
assert llm_calls == [] # the LLM never constructed
|
||||
assert stub.calls == [] # the generator never ran
|
||||
assert session.committed is False and session.closed is False
|
||||
|
||||
|
||||
def test_main_settings_validation_error_is_named(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""A settings validation error (a bad env value — the settings
|
||||
validator naming the offending field) names the failure the same
|
||||
way: non-zero exit, the named line, nothing runs."""
|
||||
def _bad_settings() -> Settings:
|
||||
# The Settings validator rejects a negative recency_boost —
|
||||
# the pydantic ValidationError names the field.
|
||||
return Settings(_env_file=None, recency_boost=-1) # pyright: ignore[reportCallIssue]
|
||||
|
||||
monkeypatch.setattr(regen, "get_settings", _bad_settings)
|
||||
monkeypatch.setattr(regen, "configure_logging", lambda level: None)
|
||||
|
||||
llm_calls: list[int] = []
|
||||
monkeypatch.setattr(
|
||||
regen, "LLMClient", lambda: llm_calls.append(1) or _FakeLLM()
|
||||
)
|
||||
stub = _GenerateStub()
|
||||
monkeypatch.setattr(regen, "generate_folder_summaries", stub)
|
||||
|
||||
rc = regen.main([])
|
||||
err = capsys.readouterr().err
|
||||
|
||||
assert rc == 1
|
||||
assert "regen_folder_summaries: boot failed:" in err
|
||||
assert "recency_boost" in err
|
||||
assert llm_calls == []
|
||||
assert stub.calls == []
|
||||
|
||||
|
||||
# ---------- the fail-loud run (never a silent partial) ----------
|
||||
|
||||
|
||||
def test_main_commit_outage_fails_loud(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||
) -> None:
|
||||
"""A DB/commit outage: the failure is NAMED on stderr, the exit
|
||||
code is non-zero, the session still closes (the uncommitted
|
||||
regeneration rolls back with it — never a silent partial), and NO
|
||||
summary line is logged (a failed run reports nothing generated)."""
|
||||
session, stub, records = _wire(monkeypatch)
|
||||
session.commit_error = RuntimeError("db down (test sentinel)")
|
||||
rc = regen.main([])
|
||||
err = capsys.readouterr().err
|
||||
|
||||
assert rc == 1
|
||||
assert "regen_folder_summaries: regeneration failed:" in err
|
||||
assert "db down (test sentinel)" in err
|
||||
assert len(stub.calls) == 1 # the generator ran…
|
||||
assert session.committed is False # …but the commit failed
|
||||
assert session.closed is True # …and the session closed (rollback)
|
||||
assert _summary_lines(records) == [] # no success line
|
||||
assert "regen_folder_summaries: folders=" not in err
|
||||
Reference in New Issue
Block a user