feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.
Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.
API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).
Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.
Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.
Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.
Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
This commit is contained in:
+39
-12
@@ -133,8 +133,9 @@
|
||||
<div class="page-head">
|
||||
<h1>Git sources</h1>
|
||||
<p class="page-sub">
|
||||
The repositories the Sync button clones and indexes. Add or
|
||||
remove them here — no <code>.env</code>, no restart.
|
||||
The git repositories and local directories the Sync button
|
||||
imports. Add or remove them here — no <code>.env</code>, no
|
||||
restart.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -178,12 +179,36 @@
|
||||
<p class="git-source-error" id="git-source-error" role="alert" hidden></p>
|
||||
</form>
|
||||
|
||||
<div class="table-wrap" id="git-sources-table-wrap" role="region" aria-label="Git sources" tabindex="0">
|
||||
<!-- Phase 38: the second add form — "Local directory": an
|
||||
existing directory on the server (NOT a git repo), walked
|
||||
directly by Sync / import_docs. The SAME never-stale-button
|
||||
+ inline-error pattern as the git form (PLAN §7.4): the
|
||||
button disables + relabels "Adding…" while the POST is out
|
||||
and recovers on success AND failure; on success the input
|
||||
clears and the list re-fetches (the new row lands with the
|
||||
Local badge). A missing/relative path 422s with the path
|
||||
named inline (paths are not secrets, unlike git URLs). -->
|
||||
<form id="local-source-form">
|
||||
<label for="local-source-path">Add a local directory</label>
|
||||
<input
|
||||
id="local-source-path"
|
||||
name="path"
|
||||
type="text"
|
||||
maxlength="2000"
|
||||
autocomplete="off"
|
||||
placeholder="~/Notes"
|
||||
required
|
||||
>
|
||||
<button type="submit" id="local-source-add">Add directory</button>
|
||||
<p class="git-source-error" id="local-source-error" role="alert" hidden></p>
|
||||
</form>
|
||||
|
||||
<div class="table-wrap" id="git-sources-table-wrap" role="region" aria-label="Sources" tabindex="0">
|
||||
<table class="git-sources-table" id="git-sources-table">
|
||||
<caption class="visually-hidden">Git repositories the Sync button clones and indexes</caption>
|
||||
<caption class="visually-hidden">Sources the Sync button imports — git repositories it clones and local directories it walks</caption>
|
||||
<thead>
|
||||
<tr>
|
||||
<th scope="col">URL</th>
|
||||
<th scope="col">Source</th>
|
||||
<th scope="col">Added</th>
|
||||
<th scope="col">Actions</th>
|
||||
</tr>
|
||||
@@ -195,16 +220,18 @@
|
||||
<!-- Empty state — no stored rows AND no env fallback. With
|
||||
from_env, the env note above already explains where the
|
||||
active list comes from. -->
|
||||
<p class="git-sources-empty" id="git-sources-empty" hidden>No git sources stored yet.</p>
|
||||
<p class="git-sources-empty" id="git-sources-empty" hidden>No sources stored yet.</p>
|
||||
|
||||
<!-- Scope boundary (phase locked decision): adding/removing a
|
||||
repo does NOT clone or prune — the Sync button performs
|
||||
that. The hint says so. -->
|
||||
source does NOT clone or prune — the Sync button performs
|
||||
that. The hint says so (phase 38: git + local together,
|
||||
files removed from a source pruned). -->
|
||||
<p class="git-source-hint" id="git-sources-hint" role="note">
|
||||
Use the <strong>Sync sources</strong> button in the header (or on
|
||||
the Sources page) to clone the repos and refresh the index —
|
||||
removing a repository prunes its documents from the index on the
|
||||
next sync.
|
||||
Sync clones/pulls the git repos and imports the local
|
||||
directories together (files removed from a source are
|
||||
pruned). Use the <strong>Sync sources</strong> button in the
|
||||
header (or on the Sources page) to run it — removing a source
|
||||
prunes its documents from the index on the next sync.
|
||||
</p>
|
||||
</div>
|
||||
<!-- Polite live region: the screen-reader confirmation for list
|
||||
|
||||
Reference in New Issue
Block a user