phase: 155_folder_summary_voice
All verification is complete — the whole project is 100% green. Final report: **Phase 155 final verification pass — complete.** **Verified (all tasks 01–04 done in `complete/`):** - Instruction nudge in `app/rag/folder_summaries.py` (forbidden openers as the pin, 4 constraints preserved, docstring cites the phase) - `scripts/regen_folder_summaries.py` (import_docs idiom, full mode, cron-safe summary line, fail-loud boot) - Unit pins, integration suite, mock voice update, dedicated E2E suite **Test / lint / coverage:** - `uv run pytest --cov=app -q` → 100% passed (exit 0, zero failures); coverage **99%** on `app/` (>90% ✓) - `uv run pytest tests/e2e/test_folder_summary_voice.py -v --no-cov` → **3 passed** in isolation - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings **Completion criteria:** 1. Marker + header prefix byte-identical — ✅ (git vs HEAD + unit pins verbatim, green) 2. Regen script full regeneration — ✅ (live smoke: exit 0, `folders=… limit=7` summary line; misconfig boot → named stderr, exit 1; `updated_at`-advance/no-prune/canned-text semantics proven by the integration suite, green) 3. Sync path untouched — ✅ (0 diffs to `app/api/sync.py` / `app/rag/importer.py`; phase-127 pins green) 4. E2E stored + rendered without forbidden openers — ✅ (3/3, exact canned-string assertions + phase-127 regression guard) 5. Full suite green, E2E isolated green, ruff + pyright clean, coverage >90% — ✅ 6. Atomic commit — deferred to the harness (working tree left intact per protocol) **Documentation:** README.md and AGENTS.md reviewed in full — verified consistent, no changes needed (README already documents the new script in the update workflow; all `.env.example` vars documented; pages table matches routes; 17 colors = 9+8; no stale mock-voice references). **Decisions:** No live-infrastructure changes (dev-DB read + a 0-row script smoke only). The `TODO.md` working-tree edit is the owner's own, left untouched. **Next pending phase:** `156_table_ux_polish`.
This commit is contained in:
@@ -0,0 +1,165 @@
|
||||
"""One-shot full regeneration of the stored folder summaries (phase 155).
|
||||
|
||||
The phase-155 nudge (task 01) re-voices the ``lite`` folder-summary
|
||||
instruction — no more "This folder contains…" catalogue — but the sync
|
||||
path is change-gated (phase 127): it rewrites only the folders a KB
|
||||
change touches and NEVER re-touches unchanged folders. The live
|
||||
catalogue's 15+ source-root rows that still carry the formulaic opener
|
||||
can therefore only be refreshed by this one-shot, which runs the
|
||||
generator's FULL mode (``changed_docs=None`` — the
|
||||
``scripts/import_docs.py`` CLI's default mode) with
|
||||
``only_missing=False``: EVERY stored row is regenerated against the
|
||||
new instruction in one run, not just the gaps::
|
||||
|
||||
uv run python -m scripts.regen_folder_summaries
|
||||
|
||||
The entrypoint mirrors ``scripts/import_docs.py`` exactly (the
|
||||
house entrypoint idiom — one event loop, one ``LLMClient``):
|
||||
|
||||
* ``get_settings()`` + ``configure_logging(settings.log_level)`` +
|
||||
``configure_debugging()`` (the phase-8 debug idiom: ``DEBUGPY=1``
|
||||
honored, default off — AGENTS.md rule 7) + ``ensure_admin_configured
|
||||
(settings)`` — the app's fail-loud boot gate (phase 16): a settings
|
||||
validation error or a missing ``BOR_ADMIN_PASSWORD`` /
|
||||
``BOR_SESSION_SECRET`` exits non-zero NAMING the failure, never a
|
||||
silent partial.
|
||||
* ``llm = LLMClient()`` — the aipi ``lite`` model (no new model
|
||||
management — the ``KB_OVERVIEW_MODE`` / ``FOLDER_SUMMARY_MODE``
|
||||
contract, PLAN §6).
|
||||
* A short-lived ``SessionLocal()``: the generator only flushes — THIS
|
||||
run's own session commits (the phase-53 convention), so a failed
|
||||
commit rolls the whole regeneration back with it (never a silent
|
||||
partial).
|
||||
* One cron-safe summary line (PLAN §9) on the logger, built from the
|
||||
generator's stats dict: ``regen_folder_summaries: folders=<generated>
|
||||
failed=<failed> pruned=<pruned> kept_manual=<kept_manual>
|
||||
limit=<n|none>``.
|
||||
* ``--limit N`` (parity with ``import_docs``'s parser — the same debug
|
||||
passthrough, the summary line names it): the generator's pinned
|
||||
signature (phase 127) has no limit parameter, so the flag does NOT
|
||||
cap the regeneration — it is the log token that names the debug run
|
||||
on the summary line (``limit=<n>``; ``limit=none`` otherwise).
|
||||
|
||||
Per-folder fail-soft is the generator's (phase 94): one folder's
|
||||
``lite`` failure keeps its previous row and counts ``failed`` — a
|
||||
partial failure never flips the exit code (an old summary is better
|
||||
than none). The exit code is about the RUN: 0 on success; non-zero
|
||||
with a named stderr line on a boot failure (settings validation, the
|
||||
admin gate) or a DB/commit outage.
|
||||
|
||||
NO new API surface, NO schema change, NO change to the sync path's
|
||||
change-gated behavior (phase 127's contract is untouched — this script
|
||||
is a consumer of the ``changed_docs=None`` full mode only).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from app.config import get_settings
|
||||
from app.core.auth import ensure_admin_configured
|
||||
from app.core.debugging import configure_debugging
|
||||
from app.core.logging import configure_logging
|
||||
from app.db import SessionLocal
|
||||
from app.rag.folder_summaries import generate_folder_summaries
|
||||
from app.rag.llm import LLMClient
|
||||
|
||||
logger = logging.getLogger("scripts.regen_folder_summaries")
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
p = argparse.ArgumentParser(
|
||||
prog="python -m scripts.regen_folder_summaries",
|
||||
description=(
|
||||
"Regenerate EVERY stored folder summary in one run (full "
|
||||
"mode — changed_docs=None, only_missing=False, skip=False: "
|
||||
"the phase-155 one-shot that re-voices the whole catalogue "
|
||||
"against the varied-opener instruction; the phase-127 "
|
||||
"change-gated sync never re-touches unchanged folders, so "
|
||||
"this is the only way to refresh them)."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--limit",
|
||||
type=int,
|
||||
default=None,
|
||||
metavar="N",
|
||||
help=(
|
||||
"debug passthrough (parity with scripts/import_docs.py's "
|
||||
"--limit; the summary line names it: limit=N) — the "
|
||||
"full-mode generator has no limit parameter, so the "
|
||||
"regeneration itself is not capped"
|
||||
),
|
||||
)
|
||||
return p
|
||||
|
||||
|
||||
async def _run(llm: LLMClient) -> dict[str, int]:
|
||||
"""The full-mode regeneration in one short-lived session.
|
||||
|
||||
``generate_folder_summaries(session, llm)`` with NO kwargs — the
|
||||
FULL-mode defaults (``skip=False``, ``only_missing=False``,
|
||||
``changed_docs=None``): every candidate folder is regenerated, the
|
||||
``scripts/import_docs.py`` CLI's mode. The generator only flushes;
|
||||
this run's own short-lived session commits (the phase-53
|
||||
convention), so a failed commit rolls the whole regeneration back
|
||||
with it.
|
||||
"""
|
||||
session = SessionLocal()
|
||||
try:
|
||||
stats = await generate_folder_summaries(session, llm)
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
return stats
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
args = build_parser().parse_args(argv)
|
||||
# The fail-loud boot (phase 16's app validator idiom, mirrored from
|
||||
# app/main.py): a settings validation error, or a missing
|
||||
# BOR_ADMIN_PASSWORD / BOR_SESSION_SECRET, names the failure and
|
||||
# exits non-zero — a regen that cannot boot must never run.
|
||||
try:
|
||||
settings = get_settings()
|
||||
configure_logging(settings.log_level)
|
||||
configure_debugging()
|
||||
ensure_admin_configured(settings)
|
||||
except Exception as e:
|
||||
print(f"regen_folder_summaries: boot failed: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try:
|
||||
llm = LLMClient()
|
||||
stats = asyncio.run(_run(llm))
|
||||
except Exception as e:
|
||||
# Fail-loud (never a silent partial): a DB outage or a failed
|
||||
# commit propagates here (the short-lived session rolls the
|
||||
# uncommitted regeneration back on close) — the failure is
|
||||
# NAMED on stderr and the exit code is non-zero.
|
||||
print(
|
||||
f"regen_folder_summaries: regeneration failed: {e}", file=sys.stderr
|
||||
)
|
||||
return 1
|
||||
|
||||
# The single cron-safe summary line (PLAN §9): the stats dict, one
|
||||
# greppable line. limit= names the --limit passthrough (parity with
|
||||
# import_docs' parser — the full-mode generator has no limit
|
||||
# parameter, so the token is the log marker, not a cap).
|
||||
limit_token = "none" if args.limit is None else str(args.limit)
|
||||
logger.info(
|
||||
"regen_folder_summaries: folders=%d failed=%d pruned=%d "
|
||||
"kept_manual=%d limit=%s",
|
||||
stats["generated"],
|
||||
stats["failed"],
|
||||
stats["pruned"],
|
||||
stats["kept_manual"],
|
||||
limit_token,
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user