feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges

An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.

Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.

API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).

Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.

Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.

Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.

Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
This commit is contained in:
2026-08-27 01:04:16 -04:00
parent 15c1272828
commit 94d7228510
22 changed files with 2190 additions and 323 deletions
+49
View File
@@ -0,0 +1,49 @@
"""git_sources.kind + path: local-directory source discriminator
Revision ID: 0007
Revises: 0006
Create Date: 2026-08-26
Phase 38 (local-directory-sources story, A13 — additive, reversible
columns; the phase-35 table is extended, not duplicated):
* ``git_sources.kind`` — TEXT NOT NULL with server default ``'git'`` and
check constraint ``ck_git_sources_kind`` (``kind IN ('git', 'local')``).
Existing rows read as ``kind='git'``; git rows keep ``url``, local
rows carry an existing directory under ``path`` (walked directly —
no clone/pull).
* ``git_sources.path`` — TEXT, nullable (NULL for git rows). Made unique
by the plain unique index ``uq_git_sources_path`` — mirrors 0006's
``uq_git_sources_url`` choice for ``url`` (Postgres treats NULLs as
distinct under a unique index, and the API enforces local-only paths
anyway).
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "0007"
down_revision = "0006"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"git_sources",
sa.Column("kind", sa.Text(), server_default=sa.text("'git'"), nullable=False),
)
op.create_check_constraint(
"ck_git_sources_kind", "git_sources", "kind IN ('git', 'local')"
)
op.add_column("git_sources", sa.Column("path", sa.Text(), nullable=True))
op.create_index("uq_git_sources_path", "git_sources", ["path"], unique=True)
def downgrade() -> None:
op.drop_index("uq_git_sources_path", table_name="git_sources")
op.drop_constraint("ck_git_sources_kind", table_name="git_sources", type_="check")
op.drop_column("git_sources", "path")
op.drop_column("git_sources", "kind")