phase: 105_hidden_folders_toggle
All completion criteria verified. Everything is green. **Phase 105 final verification pass — all criteria verified** - Verified the full implementation in the working tree: `git_sources.include_hidden` column + alembic `0019` (dev DB at head, column present), `iter_importable_files`/`import_sources` flag support with `str(root)`-keyed map used by both walk and progress pre-walk, `GitSourcePatchIn` rename with optional fields, sync/CLI pipeline wiring (OR-collision), and the per-row "Hidden" checkbox + tag + error line on the Sources page - Unit + integration: `uv run pytest` → exit 0 (2148 tests collected, all pass; this sandbox occasionally swallows pytest's final status line — exit codes verified) - Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (3879 stmts, 15 miss) — >90% gate ✓ - Dedicated E2E: `uv run pytest tests/e2e/test_hidden_folders_toggle.py -v --no-cov` → **6 passed in 23.20s** (DB up, in isolation) - Regression E2E in isolation: `test_source_ignore_paths` 6 passed, `test_git_sources_admin` 6 passed, `test_local_directory_sources` 3 passed, `test_sync_button` 3 passed, `test_smoke` 3 passed - Lint/types: `uv run ruff check .` + `uv run pyright` → clean (0 errors/warnings) **Completion criteria:** (1) checkbox persists via PATCH 200 → "hidden on" tag + GET round-trips `include_hidden: true`; failure path reverts box + `role="alert"` canned message ✓; (2) flag OFF byte-identical (only `visible.md` indexed), ON indexes `.hidden/note.md` into the KB catalog, `EXCLUDED_DIRS` excluded both states ✓; (3) A2: flag OFF → `detail.pruned==1`, doc gone from catalog ✓; (4) PATCH bool-only/list-only/both/neither no-op, phase-89 fixed 422s unchanged, 404, anonymous 403 (incl. bool-only body) ✓; (5) env-fallback rows render no checkbox, WCAG-clean (aria-label, keyboard focus, visible label, text tag) ✓; (6) full gate green ✓; (7) commit left to the harness per instructions (no `git add`/`commit` run; phase files untouched). **Deviations:** none — no defects found; no code changes were needed on this pass. **Next pending phase:** `.agents/phases/todo/98_sync_summary_visibility`.
This commit is contained in:
+37
-13
@@ -29,7 +29,11 @@ precedence order:
|
||||
Phase 89: resolution also returns each row's ignore paths, keyed by
|
||||
the resolved root string (the importer normalizes them); manual
|
||||
``--source`` dirs and the legacy fallback have no rows, so they import
|
||||
with no ignore.
|
||||
with no ignore. Phase 105 extends the same resolution with each row's
|
||||
``include_hidden`` flag — a second per-root map keyed by the same root
|
||||
strings (the importer reads it per root); manual ``--source`` dirs and
|
||||
the legacy fallback have no rows, so they import with the empty map
|
||||
(hidden paths skipped — A4).
|
||||
|
||||
Imported formats (PLAN anchor A9, revised; phase 56): the A9 family by
|
||||
default — ``md, markdown, txt, yaml, yml, json, py`` plus the quadlet
|
||||
@@ -163,9 +167,10 @@ def repo_name(url: str) -> str:
|
||||
|
||||
def _resolve_sources(
|
||||
cli_sources: list[Path] | None, settings: Settings
|
||||
) -> tuple[list[Path], dict[str, list[str]]]:
|
||||
) -> tuple[list[Path], dict[str, list[str]], dict[str, bool]]:
|
||||
"""Resolve the directories to import (phase 28, extended in phases
|
||||
35 and 38; per-root ignore maps, phase 89).
|
||||
35 and 38; per-root ignore maps, phase 89; per-root hidden-folders
|
||||
flag maps, phase 105).
|
||||
|
||||
Precedence: ``--source`` (explicit manual paths — always wins) >
|
||||
the effective sources — the ``git_sources`` DB rows (git + local),
|
||||
@@ -178,18 +183,23 @@ def _resolve_sources(
|
||||
stored directory, re-verified ``.is_dir()`` at run time) > the
|
||||
legacy ``DEFAULT_SOURCES``.
|
||||
|
||||
Returns ``(sources, ignore_by_root)`` (phase 89): the map is keyed
|
||||
by the resolved root string, exactly as the importer sees it (two
|
||||
rows sharing a root string get the union — extend, not replace);
|
||||
manual ``--source`` dirs and the legacy fallback have no rows, so
|
||||
they import with an empty map (no ignore).
|
||||
Returns ``(sources, ignore_by_root, include_hidden_by_root)``
|
||||
(phase 89; phase 105 adds the per-root flag map — the flag is
|
||||
stored per row, manual ``--source`` dirs and the legacy fallback
|
||||
have no rows and import with the empty map: hidden paths skipped,
|
||||
A4): both maps are keyed by the resolved root string, exactly as
|
||||
the importer sees it (two rows sharing a root string get the union
|
||||
— extend, not replace — for the ignore lists, and the OR of their
|
||||
flags for the hidden map); manual ``--source`` dirs and the legacy
|
||||
fallback have no rows, so they import with empty maps (no ignore,
|
||||
hidden skipped).
|
||||
|
||||
A :class:`GitSyncError` from a failing clone/pull — or a missing
|
||||
local directory (``local source missing: <path>``) — propagates to
|
||||
:func:`main`, which aborts the run before importing anything.
|
||||
"""
|
||||
if cli_sources:
|
||||
return [path.expanduser() for path in cli_sources], {}
|
||||
return [path.expanduser() for path in cli_sources], {}, {}
|
||||
db = SessionLocal()
|
||||
try:
|
||||
rows, origin = effective_sources(db)
|
||||
@@ -204,6 +214,7 @@ def _resolve_sources(
|
||||
sources_root = Path(settings.sources_dir).expanduser()
|
||||
sources: list[Path] = []
|
||||
ignore_by_root: dict[str, list[str]] = {}
|
||||
include_hidden_by_root: dict[str, bool] = {}
|
||||
for row in rows:
|
||||
if row.kind == "git":
|
||||
root = clone_or_pull(row.url, sources_root / repo_name(row.url))
|
||||
@@ -223,8 +234,16 @@ def _resolve_sources(
|
||||
# edge.
|
||||
if row.ignore_paths:
|
||||
ignore_by_root.setdefault(str(root), []).extend(row.ignore_paths)
|
||||
return sources, ignore_by_root
|
||||
return [path.expanduser() for path in DEFAULT_SOURCES], {}
|
||||
# Phase 105 (A1/A4): the row's hidden-folders flag, keyed by
|
||||
# the SAME root string the importer sees; a shared-root
|
||||
# collision ORs — if EITHER row says "index hidden", the
|
||||
# root does (the ignore-map union's boolean mirror).
|
||||
include_hidden_by_root[str(root)] = (
|
||||
include_hidden_by_root.get(str(root), False)
|
||||
or bool(row.include_hidden)
|
||||
)
|
||||
return sources, ignore_by_root, include_hidden_by_root
|
||||
return [path.expanduser() for path in DEFAULT_SOURCES], {}, {}
|
||||
|
||||
|
||||
def _overview_row_exists() -> bool:
|
||||
@@ -264,9 +283,13 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# Git sources resolve (and clone/pull) *before* any import: a failing
|
||||
# repo aborts the run with a non-zero exit, naming the failure — a bad
|
||||
# URL must never silently import partial junk. The second element is
|
||||
# the phase-89 per-root ignore map (empty for manual/fallback paths).
|
||||
# the phase-89 per-root ignore map and the third the phase-105
|
||||
# per-root hidden-folders flag map (both empty for manual/fallback
|
||||
# paths).
|
||||
try:
|
||||
sources, ignore_by_root = _resolve_sources(args.source, settings)
|
||||
sources, ignore_by_root, include_hidden_by_root = _resolve_sources(
|
||||
args.source, settings
|
||||
)
|
||||
except GitSyncError as e:
|
||||
print(f"import_docs: source sync failed: {e}", file=sys.stderr)
|
||||
return 1
|
||||
@@ -332,6 +355,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
summary = await import_sources(
|
||||
sources, llm, prune=args.prune, limit=args.limit,
|
||||
ignore_by_root=ignore_by_root,
|
||||
include_hidden_by_root=include_hidden_by_root,
|
||||
)
|
||||
if args.limit is not None:
|
||||
# An incomplete walk is debug-only — it must never advance
|
||||
|
||||
Reference in New Issue
Block a user