phase: 105_hidden_folders_toggle

All completion criteria verified. Everything is green.

**Phase 105 final verification pass — all criteria verified**

- Verified the full implementation in the working tree: `git_sources.include_hidden` column + alembic `0019` (dev DB at head, column present), `iter_importable_files`/`import_sources` flag support with `str(root)`-keyed map used by both walk and progress pre-walk, `GitSourcePatchIn` rename with optional fields, sync/CLI pipeline wiring (OR-collision), and the per-row "Hidden" checkbox + tag + error line on the Sources page
- Unit + integration: `uv run pytest` → exit 0 (2148 tests collected, all pass; this sandbox occasionally swallows pytest's final status line — exit codes verified)
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (3879 stmts, 15 miss) — >90% gate ✓
- Dedicated E2E: `uv run pytest tests/e2e/test_hidden_folders_toggle.py -v --no-cov` → **6 passed in 23.20s** (DB up, in isolation)
- Regression E2E in isolation: `test_source_ignore_paths` 6 passed, `test_git_sources_admin` 6 passed, `test_local_directory_sources` 3 passed, `test_sync_button` 3 passed, `test_smoke` 3 passed
- Lint/types: `uv run ruff check .` + `uv run pyright` → clean (0 errors/warnings)

**Completion criteria:** (1) checkbox persists via PATCH 200 → "hidden on" tag + GET round-trips `include_hidden: true`; failure path reverts box + `role="alert"` canned message ✓; (2) flag OFF byte-identical (only `visible.md` indexed), ON indexes `.hidden/note.md` into the KB catalog, `EXCLUDED_DIRS` excluded both states ✓; (3) A2: flag OFF → `detail.pruned==1`, doc gone from catalog ✓; (4) PATCH bool-only/list-only/both/neither no-op, phase-89 fixed 422s unchanged, 404, anonymous 403 (incl. bool-only body) ✓; (5) env-fallback rows render no checkbox, WCAG-clean (aria-label, keyboard focus, visible label, text tag) ✓; (6) full gate green ✓; (7) commit left to the harness per instructions (no `git add`/`commit` run; phase files untouched).

**Deviations:** none — no defects found; no code changes were needed on this pass.

**Next pending phase:** `.agents/phases/todo/98_sync_summary_visibility`.

app/api/git_sources.py

@@ -16,15 +16,20 @@ Routes: ``GET`` (DB rows oldest-first, or the env list with
 ``from_env: true`` while the table is empty; rows carry ``kind`` +
 ``path``, git rows — and env rows — report ``path: null``; every row
 reports ``ignore_paths`` — DB rows their stored normalized list, env
-rows ``[]``, phase 89), ``POST``
+rows ``[]``, phase 89 — and ``include_hidden`` — DB rows their stored
+flag, env rows ``False``, phase 105), ``POST``
 (201, validated create; ``kind`` selects the validation: git → exactly
 the phase-35 URL contract, local → an existing absolute directory, else
 422 naming the path; both kinds accept ``ignore_paths`` — optional,
-absent → ``[]``, stored normalized, phase 89),
-``PATCH /{source_id}`` (phase 89, A5 — replace one source's ignore
-list: 404 unknown id, the required body list is normalized + A4-
-validated with fixed-detail 422s and REPLACES the row's list wholesale
-— an empty list clears all; 200 → the ``GitSourceOut`` shape),
+absent → ``[]``, stored normalized, phase 89 — and ``include_hidden``
+— optional, absent → ``False``, stored as sent, phase 105),
+``PATCH /{source_id}`` (phase 89, A5 + phase 105 — the ignore list
+(replace) and/or the hidden-folders flag (phase 105) — each optional,
+present-wins: 404 unknown id; ``ignore_paths`` when present is
+normalized + A4-validated with fixed-detail 422s and REPLACES the
+row's list wholesale — an empty list clears all; ``include_hidden``
+when present sets the flag; both absent → 200 no-op; 200 → the
+``GitSourceOut`` shape),
 ``POST /upload`` (phase 49, backgrounded in phase
 64 task 03, scan deferred in phase 90 — admin archive upload: the
 ``.tar``/``.tar.gz``/``.tgz``/``.zip`` name/format gate + the 1 MiB-
@@ -116,10 +121,10 @@ from app.rag.source_removal import (
 )
 from app.rag.sources_meta import bump_sources_version
 from app.schemas import (
-    GitSourceIgnoreIn,
     GitSourceIn,
     GitSourceList,
     GitSourceOut,
+    GitSourcePatchIn,
     GitSourceRow,
     UploadAccepted,
 )
@@ -208,11 +213,13 @@ def list_git_sources(
     same-timestamp inserts) with ``from_env: false`` — each row carries
     its ``kind``, its ``ignore_paths`` (phase 89 — the stored,
     normalized list; ``or []`` guards a row that predated the column),
-    and, for local rows, the stored ``path`` (git rows and env rows
-    report ``path: null``); while the table is empty, the
+    its ``include_hidden`` (phase 105 — the stored flag), and, for
+    local rows, the stored ``path`` (git rows and env rows report
+    ``path: null``); while the table is empty, the
     ``BOR_GIT_SOURCES`` env URLs as git rows (the env fallback is
-    git-only) with null ``id``/``added_at``, ``ignore_paths: []`` (no
-    DB row to store a list on), and ``from_env: true``.
+    git-only) with null ``id``/``added_at``, ``ignore_paths: []`` and
+    ``include_hidden: False`` (no DB row to store a list or a flag on),
+    and ``from_env: true``.
     """
     rows = db.scalars(
         select(GitSource).order_by(GitSource.added_at.asc(), GitSource.id.asc())
@@ -229,6 +236,7 @@ def list_git_sources(
                     path=row.path,
                     added_at=row.added_at,
                     ignore_paths=row.ignore_paths or [],
+                    include_hidden=row.include_hidden,
                 )
                 for row in rows
             ],
@@ -237,7 +245,13 @@ def list_git_sources(
     return GitSourceList(
         sources=[
             GitSourceRow(
-                id=None, kind="git", url=url, path=None, added_at=None, ignore_paths=[]
+                id=None,
+                kind="git",
+                url=url,
+                path=None,
+                added_at=None,
+                ignore_paths=[],
+                include_hidden=False,
             )
             for url in get_settings().git_source_list
         ],
@@ -272,10 +286,18 @@ def create_git_source(
     lines are normalized + A4-validated (``_validate_ignore_paths`` —
     the fixed-detail 422s) and the normalized list is what is stored and
     reported.
+
+    ``include_hidden`` (phase 105) — optional, both kinds: absent →
+    stored ``False`` (A4), present → stored as sent; the stored flag is
+    what is reported.
     """
     row = _create_git_row(payload, db) if payload.kind == "git" else _create_local_row(payload, db)
     return GitSourceOut(
-        id=row.id, url=row.url, added_at=row.added_at, ignore_paths=row.ignore_paths
+        id=row.id,
+        url=row.url,
+        added_at=row.added_at,
+        ignore_paths=row.ignore_paths,
+        include_hidden=row.include_hidden,
     )
 
 
@@ -331,6 +353,7 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
             url=url,
             kind="git",
             ignore_paths=_validate_ignore_paths(payload.ignore_paths),
+            include_hidden=bool(payload.include_hidden),
         ),
         "a git source with this URL already exists",
         db,
@@ -364,6 +387,7 @@ def _create_local_row(payload: GitSourceIn, db: Session) -> GitSource:
             kind="local",
             path=path,
             ignore_paths=_validate_ignore_paths(payload.ignore_paths),
+            include_hidden=bool(payload.include_hidden),
         ),
         f"a local source with this path already exists: {path}",
         db,
@@ -373,24 +397,33 @@ def _create_local_row(payload: GitSourceIn, db: Session) -> GitSource:
 @router.patch("/{source_id}", response_model=GitSourceOut)
 def patch_git_source(
     source_id: uuid.UUID,
-    payload: GitSourceIgnoreIn,
+    payload: GitSourcePatchIn,
     db: Session = Depends(get_db),  # noqa: B008
 ) -> GitSourceOut:
-    """Replace one source's ignore list (phase 89, A5).
+    """Edit one source's ignore list and/or hidden-folders flag.
 
-    404 unknown id. The body list (required) is normalized +
-    A4-validated (fixed 422 details) and REPLACES the row's list
-    wholesale — an empty list clears all. Returns the updated
-    row's public shape (id, url, added_at, ignore_paths).
+    Phase 89 A5 (ignore list) + phase 105 (the flag): 404 unknown
+    id; each PRESENT body field applies independently —
+    ``ignore_paths`` REPLACES the list (normalized + A4-validated,
+    fixed 422 details); ``include_hidden`` sets the flag. Both
+    absent → 200 no-op. Returns the updated row's public shape
+    (id, url, added_at, ignore_paths, include_hidden).
     """
     row = db.get(GitSource, source_id)
     if row is None:
         raise HTTPException(status_code=404, detail="git source not found")
-    row.ignore_paths = _validate_ignore_paths(payload.ignore_paths)
+    if payload.ignore_paths is not None:
+        row.ignore_paths = _validate_ignore_paths(payload.ignore_paths)
+    if payload.include_hidden is not None:
+        row.include_hidden = payload.include_hidden
     db.commit()
     db.refresh(row)
     return GitSourceOut(
-        id=row.id, url=row.url, added_at=row.added_at, ignore_paths=row.ignore_paths
+        id=row.id,
+        url=row.url,
+        added_at=row.added_at,
+        ignore_paths=row.ignore_paths,
+        include_hidden=row.include_hidden,
     )
 
 

app/api/sync.py

@@ -36,7 +36,9 @@ decisions):
 4. ``import_sources(..., prune=True)`` over the single combined list
    (git checkouts + local dirs), honoring each row's ``ignore_paths``
    (phase 89 — the per-root ignore map is built in the same per-row
-   loop as the source list) — prune so files deleted upstream, out of
+   loop as the source list) and its ``include_hidden`` flag (phase 105
+   — the per-root hidden-folders map, same per-row construction) —
+   prune so files deleted upstream, out of
    a local dir, or newly matching an ignore pattern leave the index
    (pruning covers the union; the CLI's no-prune default is unchanged);
 5. when the import changed the KB (added + updated > 0),
@@ -231,6 +233,7 @@ async def _run_sync() -> None:
         sources_root = Path(settings.sources_dir).expanduser()
         sources: list[Path] = []
         ignore_by_root: dict[str, list[str]] = {}
+        include_hidden_by_root: dict[str, bool] = {}
         for row in rows:
             if row.kind == "git":
                 root = clone_or_pull(row.url, sources_root / repo_name(row.url))
@@ -250,6 +253,14 @@ async def _run_sync() -> None:
             # edge.
             if row.ignore_paths:
                 ignore_by_root.setdefault(str(root), []).extend(row.ignore_paths)
+            # Phase 105 (A1/A4): the row's hidden-folders flag, keyed by
+            # the SAME root string the importer sees; a shared-root
+            # collision ORs — if EITHER row says "index hidden", the
+            # root does (the ignore-map union's boolean mirror).
+            include_hidden_by_root[str(root)] = (
+                include_hidden_by_root.get(str(root), False)
+                or bool(row.include_hidden)
+            )
         # Phase 64 (task 02): the per-file progress hook — the status
         # endpoint reports the file being processed right now. The
         # closure captures the module ``_status`` exactly like the state
@@ -260,7 +271,8 @@ async def _run_sync() -> None:
             _status.files_total = total
 
         summary: ImportSummary = await import_sources(
-            sources, llm, prune=True, progress=_hook, ignore_by_root=ignore_by_root
+            sources, llm, prune=True, progress=_hook, ignore_by_root=ignore_by_root,
+            include_hidden_by_root=include_hidden_by_root,
         )
         overview = False
         if summary.added + summary.updated > 0: