feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges

An existing, non-git directory is now a first-class source alongside the git repos: one table (git_sources + kind discriminator — A13 reversible migration), one admin page, one Sync button (phase locked decisions; the phase-35 table is extended, not duplicated). The DB is the local-source registry — no env var for local paths; BOR_GIT_SOURCES stays a git-only empty-table fallback. Migration 0007 (reversible, up/down integration-tested): git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind (kind IN ('git','local')); git_sources.path TEXT NULL + uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows read kind='git', path=NULL. API (phase-35 contract extended, git byte-identical): POST kind=local requires path — trimmed, ~-expanded, absolute + an existing server directory, else 422 naming the path (fail loud at add-time); duplicate path 409 (named); wrong field combos 422. GET rows carry kind + path (git and env rows: path null); anonymous still 403 on every route (A10). Sync + import_docs resolve DB git + local rows together: git → clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME (it may have moved/deleted since add-time) — a missing dir raises "local source missing: <path>" (sanitized) before anything imports; one import_sources(..., prune=True) over the single combined list (pruning covers the union). Both-empty fails loudly ("no sources configured (git or local)"); --source still wins; the env fallback stays git-only. Page: second "Add a local directory" form (the same §7.4 never-stale button + inline-error lifecycle as the git form; 422/409 details name the path), Git/Local badges on rows (text + color, never color alone — WCAG), updated hint (git + local together, union prune); the anonymous sign-in gate is unchanged. Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with the git-kind suite green unchanged; the sync pipeline local/git/ mixed/missing against a host temp dir (the KB actually updated); import_docs DB resolution + --source precedence. Story E2E (isolated, deterministic across runs): add (Local badge) → missing path inline 422 naming it / duplicate 409 → the real Sync button imports the fixture file (GET /api/docs + sentinel in its content) → file deleted + sync prunes it (union prune) → row removed; anonymous gate + 403s (phase-35 regression). test_git_sources_admin.py (phase 35) green UNCHANGED — no selector collision with the new form; test_sync_button.py green. Docs: README — the two managed kinds (git = clone/pull mirror; local = direct in-place walk), add-time validation, union pruning, "the DB is the local-source registry (no env var for local paths)"; .env.example — the env fallback is git-only.
2026-08-27 01:04:16 -04:00
parent 15c1272828
commit 94d7228510
22 changed files with 2190 additions and 323 deletions
@@ -1,12 +1,16 @@
-"""Unit: the shared git-source resolver (phase 35, task 03).
+"""Unit: the shared source resolver (phase 35, task 03; local kind,
+phase 38, task 03).

-``effective_git_sources`` is driven with a stubbed session (no Postgres)
+``effective_sources`` is driven with a stubbed session (no Postgres)
 and a fresh ``Settings(_env_file=None)`` env (monkeypatched into the
-resolver's module — the dev ``.env`` never leaks in, same pattern as the
-integration suites): DB rows win in ``(added_at, id)`` order (env
-ignored), the ``BOR_GIT_SOURCES`` list is a fallback only while the
-table is empty (phase-28 CSV parse reused), and both-empty yields
-``([], "env")`` so the callers keep their fail-loud behavior.
+resolver's module — the dev ``.env`` never leaks in, same pattern as
+the integration suites): DB rows of **both kinds** win in
+``(added_at, id)`` order (env ignored), the ``BOR_GIT_SOURCES`` list is
+a git-only fallback while the table is empty (surfaced as synthetic
+``kind='git'`` rows, phase-28 CSV parse reused), and both-empty yields
+``([], "env")`` so the callers keep their fail-loud behavior. The
+phase-35 ``effective_git_sources`` alias is kept covered as well: it
+returns the repo URLs of the git rows only, same origin.
 """
 from __future__ import annotations

@@ -52,33 +56,70 @@ def _stub_env(monkeypatch: pytest.MonkeyPatch, git_sources: str) -> None:
    )


+def _git(url: str) -> GitSource:
+    return GitSource(url=url, kind="git")
+
+
+def _local(path: str) -> GitSource:
+    """A local row as phase-38 task 02 stores it: the expanded path in
+    both ``path`` and the NOT-NULL ``url`` location column."""
+    return GitSource(url=path, kind="local", path=path)
+
+
 # --- the three branches ----------------------------------------------------


 def test_db_rows_win_over_env(monkeypatch: pytest.MonkeyPatch) -> None:
-    """Seeded table + env set → the DB list, origin ``db`` (env ignored)."""
+    """Seeded table + env set → the DB rows, origin ``db`` (env ignored)."""
    _stub_env(monkeypatch, "https://env.example/ignored.git")
-    session = _FakeSession([GitSource(url="https://db.example/a.git"), GitSource(url="https://db.example/b.git")])
+    session = _FakeSession(
+        [GitSource(url="https://db.example/a.git"), GitSource(url="https://db.example/b.git")]
+    )

-    urls, origin = resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+    rows, origin = resolver.effective_sources(session)  # pyright: ignore[reportArgumentType]

-    assert urls == ["https://db.example/a.git", "https://db.example/b.git"]
+    assert [row.url for row in rows] == ["https://db.example/a.git", "https://db.example/b.git"]
    assert origin == "db"


-def test_env_fallback_while_table_empty(monkeypatch: pytest.MonkeyPatch) -> None:
-    """Empty table + env set → the env list, origin ``env``.
+def test_mixed_kinds_returned_in_row_order(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Phase 38: the table holds git **and** local rows — both come back,
+    in ``(added_at, id)`` row order, kinds and paths intact, env ignored."""
+    _stub_env(monkeypatch, "https://env.example/ignored.git")
+    local = "/abs/notes"
+    session = _FakeSession(
+        [_git("https://db.example/a.git"), _local(local), _git("git@db.example:b.git")]
+    )

-    The CSV parse is ``Settings.git_source_list`` itself (phase 28):
+    rows, origin = resolver.effective_sources(session)  # pyright: ignore[reportArgumentType]
+
+    assert origin == "db"
+    assert [row.kind for row in rows] == ["git", "local", "git"]
+    assert [row.url for row in rows] == [
+        "https://db.example/a.git",
+        local,
+        "git@db.example:b.git",
+    ]
+    assert [row.path for row in rows] == [None, local, None]
+
+
+def test_env_fallback_git_only_while_table_empty(monkeypatch: pytest.MonkeyPatch) -> None:
+    """Empty table + env set → synthetic ``kind='git'`` rows, origin
+    ``env``.
+
+    The env fallback is git-only (no local rows can come from it); the
+    CSV parse is ``Settings.git_source_list`` itself (phase 28):
    whitespace-trimmed, empty entries dropped, order preserved.
    """
    _stub_env(monkeypatch, " https://env.example/one.git , ,git@env.example:two.git ")
    session = _FakeSession([])

-    urls, origin = resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+    rows, origin = resolver.effective_sources(session)  # pyright: ignore[reportArgumentType]

-    assert urls == ["https://env.example/one.git", "git@env.example:two.git"]
    assert origin == "env"
+    assert [row.url for row in rows] == ["https://env.example/one.git", "git@env.example:two.git"]
+    assert all(row.kind == "git" for row in rows)
+    assert all(row.path is None for row in rows)


 def test_both_empty_returns_empty_env(monkeypatch: pytest.MonkeyPatch) -> None:
@@ -86,9 +127,9 @@ def test_both_empty_returns_empty_env(monkeypatch: pytest.MonkeyPatch) -> None:
    _stub_env(monkeypatch, " , ")  # whitespace-only is just as unconfigured as empty
    session = _FakeSession([])

-    urls, origin = resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+    rows, origin = resolver.effective_sources(session)  # pyright: ignore[reportArgumentType]

-    assert urls == []
+    assert rows == []
    assert origin == "env"


@@ -99,12 +140,42 @@ def test_db_rows_ordered_by_added_at_then_id(monkeypatch: pytest.MonkeyPatch) ->
    """The statement orders by ``(added_at, id)`` — oldest first, the id
    tie-break deciding same-timestamp inserts (matches the API's GET)."""
    _stub_env(monkeypatch, "")
-    session = _FakeSession([GitSource(url="https://db.example/a.git")])
+    session = _FakeSession([_git("https://db.example/a.git")])

-    resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+    resolver.effective_sources(session)  # pyright: ignore[reportArgumentType]

    assert len(session.statements) == 1
    compiled = str(session.statements[0].compile(compile_kwargs={"literal_binds": True}))
    assert re.search(
        r"ORDER BY\s+git_sources\.added_at ASC,\s+git_sources\.id ASC", compiled
    ), compiled
+
+
+# --- the phase-35 back-compat alias ----------------------------------------
+
+
+def test_alias_returns_git_urls_only(monkeypatch: pytest.MonkeyPatch) -> None:
+    """``effective_git_sources`` (phase-35 name) — repo URLs of the git
+    rows only; local rows are filtered out (they carry no clone URL),
+    origin unchanged."""
+    _stub_env(monkeypatch, "")
+    session = _FakeSession(
+        [_git("https://db.example/a.git"), _local("/abs/notes"), _git("git@db.example:b.git")]
+    )
+
+    urls, origin = resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+
+    assert urls == ["https://db.example/a.git", "git@db.example:b.git"]
+    assert origin == "db"
+
+
+def test_alias_env_fallback_unchanged(monkeypatch: pytest.MonkeyPatch) -> None:
+    """The alias's env fallback is byte-identical to the phase-35 one
+    (git-only CSV list, origin ``env``)."""
+    _stub_env(monkeypatch, " https://env.example/one.git , ")
+    session = _FakeSession([])
+
+    urls, origin = resolver.effective_git_sources(session)  # pyright: ignore[reportArgumentType]
+
+    assert urls == ["https://env.example/one.git"]
+    assert origin == "env"