phase: 89_source_ignore_paths
All verification complete — TODO.md was already cleared in the roadmap commit; the two extra unit-test diffs are necessary fake-signature adaptations for the new keywords. Everything is green, no fixes were needed. ## Phase 89 — final verification pass: ALL GREEN **Verified (all 6 task files present in `complete/`):** - `git_sources.ignore_paths` JSONB column + migration 0013; `alembic downgrade -1 && upgrade head` round-trips (head `0013`) - Importer: `normalize_ignore_path`/`is_ignored`/`_ignore_for_root`, `ignore` in walk + progress pre-walk, `ignore_by_root` in `import_sources` - API: GET/POST carry list; admin-only `PATCH` (replace, 404/422 fixed details, anonymous 403) - Pipelines wired: `_run_sync`, `_run_upload` re-upload, `scripts/import_docs.py` - Sources-page box: dialog, §7.4 save lifecycle, `N ignored` tag, a11y; env rows get no box **Test/lint results:** - `uv run pytest` → 1808 passed - `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%) - `uv run pytest tests/e2e/test_source_ignore_paths.py -v --no-cov` → 6 passed (isolated, DB up) - Regressions in isolation: `test_git_sources_admin` 6, `test_archive_upload_sources` 5, `test_sync_button` 3, `test_smoke` 3 — all passed - `uv run ruff check . && uv run pyright` → clean (0 errors) **Completion criteria:** box→PATCH 200→count+GET round-trip ✅ · sync excludes `ignore/` (no docs/chunks/embeddings/summaries) + prunes newly-ignored (pruned==2) ✅ · no-mid-path rule E2E ✅ · PATCH 404/422/replace/clear/403 ✅ · full gate green ✅ · commit + phase move left to harness per rules. **Deviations:** none blocking — E2E pins `files == 4` (overview's "5" was an off-by-one vs its own 6-file tree, documented in-test); `tests/unit/test_importer.py` + `test_sync_button.py` test-double fakes extended for the new keywords (needed for the suite to stay green). **Next pending phase:** none — `todo/` holds only this phase.
This commit is contained in:
+20
-9
@@ -34,9 +34,11 @@ decisions):
|
||||
<path>``; a failing clone or a missing local dir aborts before any
|
||||
import;
|
||||
4. ``import_sources(..., prune=True)`` over the single combined list
|
||||
(git checkouts + local dirs) — prune so files deleted upstream or
|
||||
out of a local dir leave the index (pruning covers the union; the
|
||||
CLI's no-prune default is unchanged);
|
||||
(git checkouts + local dirs), honoring each row's ``ignore_paths``
|
||||
(phase 89 — the per-root ignore map is built in the same per-row
|
||||
loop as the source list) — prune so files deleted upstream, out of
|
||||
a local dir, or newly matching an ignore pattern leave the index
|
||||
(pruning covers the union; the CLI's no-prune default is unchanged);
|
||||
5. when the import changed the KB (added + updated > 0),
|
||||
``regenerate_overview`` refreshes the single ``kb_overview`` row
|
||||
(phase 31 trigger, best-effort inside);
|
||||
@@ -214,19 +216,26 @@ async def _run_sync() -> None:
|
||||
)
|
||||
sources_root = Path(settings.sources_dir).expanduser()
|
||||
sources: list[Path] = []
|
||||
ignore_by_root: dict[str, list[str]] = {}
|
||||
for row in rows:
|
||||
if row.kind == "git":
|
||||
sources.append(clone_or_pull(row.url, sources_root / repo_name(row.url)))
|
||||
root = clone_or_pull(row.url, sources_root / repo_name(row.url))
|
||||
else:
|
||||
# kind=local — the stored expanded path (phase 38 also
|
||||
# mirrors it in the NOT-NULL ``url`` location column, the
|
||||
# ``or`` keeps the type checker honest); re-verified at
|
||||
# sync time because the directory may have moved or been
|
||||
# deleted since add-time.
|
||||
path = Path(row.path or row.url).expanduser()
|
||||
if not path.is_dir():
|
||||
raise GitSyncError(f"local source missing: {path}")
|
||||
sources.append(path)
|
||||
root = Path(row.path or row.url).expanduser()
|
||||
if not root.is_dir():
|
||||
raise GitSyncError(f"local source missing: {root}")
|
||||
sources.append(root)
|
||||
# Phase 89: the row's ignore list, keyed by the SAME root
|
||||
# string the importer sees; two rows sharing a root string
|
||||
# get the union (extend, not replace) — the sibling/repo-name
|
||||
# edge.
|
||||
if row.ignore_paths:
|
||||
ignore_by_root.setdefault(str(root), []).extend(row.ignore_paths)
|
||||
# Phase 64 (task 02): the per-file progress hook — the status
|
||||
# endpoint reports the file being processed right now. The
|
||||
# closure captures the module ``_status`` exactly like the state
|
||||
@@ -236,7 +245,9 @@ async def _run_sync() -> None:
|
||||
_status.files_done = done
|
||||
_status.files_total = total
|
||||
|
||||
summary: ImportSummary = await import_sources(sources, llm, prune=True, progress=_hook)
|
||||
summary: ImportSummary = await import_sources(
|
||||
sources, llm, prune=True, progress=_hook, ignore_by_root=ignore_by_root
|
||||
)
|
||||
overview = False
|
||||
if summary.added + summary.updated > 0:
|
||||
overview = await regenerate_overview(llm)
|
||||
|
||||
Reference in New Issue
Block a user