feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.
Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.
API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).
Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.
Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.
Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.
Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
This commit is contained in:
+51
-12
@@ -181,28 +181,47 @@ class SteeringNoteList(BaseModel):
|
||||
|
||||
|
||||
class GitSourceIn(BaseModel):
|
||||
"""``POST /api/git-sources`` body: one repo URL (phase 35, task 02).
|
||||
"""``POST /api/git-sources`` body (phase 35, task 02; ``kind``, phase 38).
|
||||
|
||||
Mirrors :class:`SteeringNoteIn` — the URL is trimmed *before* the
|
||||
length constraints run, so a whitespace-only body is a 422 and a URL
|
||||
with surrounding spaces is stored clean. Shape validation
|
||||
(``https://``, ``ssh://``, ``git@``) happens in the API layer so the
|
||||
422 detail can be one generic string that never echoes the input.
|
||||
``kind`` selects the source kind and which field carries its location:
|
||||
|
||||
* ``"git"`` (default) — ``url`` is the repo URL. Mirrors the
|
||||
phase-35 contract: trimmed *before* the length constraints run, so a
|
||||
whitespace-only body is a 422 and a URL with surrounding spaces is
|
||||
stored clean. Shape validation (``https://``, ``ssh://``, ``git@``)
|
||||
and the kind-field rules (url present, no path) happen in the API
|
||||
layer so the 422/409 details stay fixed strings that never echo the
|
||||
input (credential safety).
|
||||
* ``"local"`` — ``path`` is an existing directory on the server.
|
||||
Trimmed here; the API layer then ``expanduser()``s it and requires an
|
||||
absolute existing directory (else 422 naming the path — the path is
|
||||
not a secret, unlike a git URL) and no ``url``.
|
||||
"""
|
||||
|
||||
url: str = Field(min_length=1, max_length=500)
|
||||
kind: Literal["git", "local"] = "git"
|
||||
url: str | None = Field(default=None, min_length=1, max_length=500)
|
||||
path: str | None = Field(default=None, min_length=1, max_length=2000)
|
||||
|
||||
@field_validator("url", mode="before")
|
||||
@classmethod
|
||||
def _trim_url(cls, v: object) -> object:
|
||||
return v.strip() if isinstance(v, str) else v
|
||||
|
||||
@field_validator("path", mode="before")
|
||||
@classmethod
|
||||
def _trim_path(cls, v: object) -> object:
|
||||
return v.strip() if isinstance(v, str) else v
|
||||
|
||||
|
||||
class GitSourceOut(BaseModel):
|
||||
"""One git source as returned by the API (phase 35, task 02).
|
||||
"""One created git source as returned by ``POST`` (phase 35, task 02).
|
||||
|
||||
``id`` / ``added_at`` are nullable: env-fallback rows (table empty →
|
||||
the list comes from ``BOR_GIT_SOURCES``) carry neither, only a URL.
|
||||
``id`` / ``added_at`` are non-null for a stored row. ``url`` is the
|
||||
row's location column: the repo URL for ``kind=git`` rows and, for
|
||||
``kind=local`` rows, the stored (expanded) directory path — the
|
||||
phase-35 response shape is unchanged by phase 38, so a local 201
|
||||
reports its path in ``url`` and the full row (``kind`` + ``path``)
|
||||
via ``GET``.
|
||||
"""
|
||||
|
||||
id: uuid.UUID | None
|
||||
@@ -210,14 +229,34 @@ class GitSourceOut(BaseModel):
|
||||
added_at: datetime | None
|
||||
|
||||
|
||||
class GitSourceRow(BaseModel):
|
||||
"""One row of ``GET /api/git-sources`` (phase 35; ``kind``/``path``,
|
||||
phase 38, task 02).
|
||||
|
||||
``kind`` discriminates the row: git rows (and the git-only
|
||||
``BOR_GIT_SOURCES`` env-fallback rows) carry ``url`` and
|
||||
``path: null``; local rows carry ``path`` (the absolute directory,
|
||||
expanded) and the same string in ``url`` (the table's NOT-NULL
|
||||
location column). ``id`` / ``added_at`` are nullable: env-fallback
|
||||
rows (table empty) carry neither.
|
||||
"""
|
||||
|
||||
id: uuid.UUID | None
|
||||
kind: Literal["git", "local"]
|
||||
url: str
|
||||
path: str | None
|
||||
added_at: datetime | None
|
||||
|
||||
|
||||
class GitSourceList(BaseModel):
|
||||
"""``GET /api/git-sources`` response (phase 35, task 02).
|
||||
|
||||
``from_env`` is True only when the ``git_sources`` table is empty and
|
||||
the list comes from ``BOR_GIT_SOURCES`` (the phase's locked fallback);
|
||||
the list comes from ``BOR_GIT_SOURCES`` (the phase's locked fallback
|
||||
— env rows are git-only and report ``kind: "git"``, ``path: null``);
|
||||
once the table has rows the env var is ignored and ``from_env`` is
|
||||
False — the UI is the source of truth.
|
||||
"""
|
||||
|
||||
sources: list[GitSourceOut]
|
||||
sources: list[GitSourceRow]
|
||||
from_env: bool
|
||||
|
||||
Reference in New Issue
Block a user