feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.
Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.
API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).
Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.
Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.
Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.
Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
This commit is contained in:
+118
-54
@@ -1,7 +1,10 @@
|
||||
/* Brain of Reese — Git sources admin page (phase 35, task 04).
|
||||
/* Brain of Reese — Git sources admin page (phase 35, task 04;
|
||||
* local directories, phase 38 task 04).
|
||||
*
|
||||
* The page module for /git-sources.html: the admin-only manager for the
|
||||
* stored git source list (git-sources table, phase 35 tasks 01/02).
|
||||
* stored source list (git-sources table, phase 35 tasks 01/02) — git
|
||||
* repo URLs (kind "git") and existing local directories (kind
|
||||
* "local", phase 38).
|
||||
* This module is the single owner of the page's behaviour:
|
||||
*
|
||||
* • boot — initSharedHeader() (one cached whoami, shared with the
|
||||
@@ -11,19 +14,25 @@
|
||||
* #git-sources-content revealed, loadSources().
|
||||
* • loadSources() — GET /api/git-sources → the table rows
|
||||
* (#git-sources-tbody), the env-fallback note's visibility
|
||||
* (from_env), and the empty state. URLs are ALWAYS rendered with
|
||||
* textContent — never innerHTML (they may embed user:pass@
|
||||
* (from_env), and the empty state. Each row leads with its kind
|
||||
* badge (Git/Local — text + color, never color alone) plus the
|
||||
* location in a mono <code>: the git URL, or the full local path
|
||||
* for kind "local" rows (phase 38). Values are ALWAYS rendered
|
||||
* with textContent — never innerHTML (URLs may embed user:pass@
|
||||
* credentials; phase 32's masking discipline). Non-2xx or a
|
||||
* network failure renders the role="alert" load error with a
|
||||
* retry button — never a stuck page.
|
||||
* • add — #git-source-form submit → POST /api/git-sources {url}.
|
||||
* §7.4 never-stale: the button disables + relabels "Adding…"
|
||||
* while the request is out, re-enables ("Add source") on success
|
||||
* AND failure. 201 clears the input, reloads the list, and
|
||||
* focuses the new row's Remove button (a11y); a failure (409
|
||||
* duplicate, 422 shape) shows the server detail inline under the
|
||||
* form (role="alert", 422 shape-aware like the tuning forms) and
|
||||
* keeps the input — the instruction survives.
|
||||
* • add — #git-source-form submit → POST /api/git-sources {url};
|
||||
* #local-source-form submit → POST /api/git-sources
|
||||
* {kind: "local", path} (phase 38). ONE §7.4 never-stale
|
||||
* lifecycle for both (wireAddForm): the button disables +
|
||||
* relabels "Adding…" while the request is out, re-enables on
|
||||
* success AND failure. 201 clears the input, reloads the list,
|
||||
* and focuses the new row's Remove button (a11y); a failure (409
|
||||
* duplicate, 422 validation) shows the server detail inline under
|
||||
* the form (role="alert", 422 shape-aware like the tuning forms)
|
||||
* and keeps the input — the instruction survives. Local 422/409
|
||||
* details name the path (paths are not secrets, unlike URLs).
|
||||
* • remove — a row's Remove button asks window.confirm first
|
||||
* (removal prunes the documents only on the NEXT sync — the
|
||||
* confirm says so). Cancel → nothing; ok → the row button
|
||||
@@ -56,6 +65,12 @@ const formEl = document.querySelector("#git-source-form");
|
||||
const urlInput = document.querySelector("#git-source-url");
|
||||
const addBtn = document.querySelector("#git-source-add");
|
||||
const addError = document.querySelector("#git-source-error");
|
||||
/* Phase 38: the second add form — "Local directory" (same element
|
||||
contract as the git form, own ids). */
|
||||
const localFormEl = document.querySelector("#local-source-form");
|
||||
const pathInput = document.querySelector("#local-source-path");
|
||||
const localAddBtn = document.querySelector("#local-source-add");
|
||||
const localAddError = document.querySelector("#local-source-error");
|
||||
const loadErrorEl = document.querySelector("#git-sources-load-error");
|
||||
const loadErrorText = document.querySelector("#git-sources-load-error-text");
|
||||
const retryBtn = document.querySelector("#git-sources-retry");
|
||||
@@ -125,7 +140,7 @@ async function loadSources() {
|
||||
hideLoadError();
|
||||
const sources = Array.isArray(data.sources) ? data.sources : [];
|
||||
renderSources(sources, data.from_env === true);
|
||||
announce(`${sources.length} git source${sources.length === 1 ? "" : "s"} listed.`);
|
||||
announce(`${sources.length} source${sources.length === 1 ? "" : "s"} listed.`);
|
||||
}
|
||||
|
||||
function showLoadError(message) {
|
||||
@@ -153,11 +168,15 @@ function renderSources(sources, fromEnv) {
|
||||
if (emptyEl) emptyEl.hidden = hasRows;
|
||||
}
|
||||
|
||||
/* One row: the URL in a mono <code> (textContent only — URLs may
|
||||
contain credentials), the localized added date ("—" for env
|
||||
fallback rows), and the per-row Remove button — or the "from .env"
|
||||
tag for env-fallback rows (id null: nothing is stored to remove;
|
||||
the env note says where the active list comes from). */
|
||||
/* One row: the kind badge (Git/Local — phase 38) followed by the
|
||||
location in a mono <code> (textContent only — git URLs may contain
|
||||
credentials, local paths may contain anything), the localized added
|
||||
date ("—" for env fallback rows), and the per-row Remove button —
|
||||
or the "from .env" tag for env-fallback rows (id null: nothing is
|
||||
stored to remove; the env note says where the active list comes
|
||||
from). The value is the git URL for kind "git" rows and the full
|
||||
local path for kind "local" rows (the API reports the path in both
|
||||
`path` and `url`; `path` is the kind-typed field). */
|
||||
const REMOVE_ICON =
|
||||
'<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round"><path d="M5 7h14M10 7V5h4v2M8.5 7l.7 12h5.6l.7-12"/></svg>';
|
||||
|
||||
@@ -165,12 +184,19 @@ function makeRow(s) {
|
||||
const tr = document.createElement("tr");
|
||||
if (s.id) tr.dataset.id = s.id;
|
||||
|
||||
const isLocal = s.kind === "local";
|
||||
const value = isLocal ? (s.path ?? s.url) : s.url;
|
||||
const kindLabel = isLocal ? "local" : "git";
|
||||
|
||||
const urlTd = document.createElement("td");
|
||||
urlTd.className = "git-source-url-cell";
|
||||
urlTd.title = s.url; // full URL on hover (long URLs scroll the wrapper)
|
||||
urlTd.title = value; // full URL/path on hover (long values scroll the wrapper)
|
||||
const badge = document.createElement("span");
|
||||
badge.className = `git-source-kind is-${isLocal ? "local" : "git"}`;
|
||||
badge.textContent = isLocal ? "Local" : "Git"; // text + color, never color alone
|
||||
const code = document.createElement("code");
|
||||
code.textContent = s.url; // rendered as text, never as HTML
|
||||
urlTd.appendChild(code);
|
||||
code.textContent = value; // rendered as text, never as HTML
|
||||
urlTd.append(badge, code);
|
||||
tr.appendChild(urlTd);
|
||||
|
||||
const addedTd = document.createElement("td");
|
||||
@@ -183,13 +209,13 @@ function makeRow(s) {
|
||||
const btn = document.createElement("button");
|
||||
btn.type = "button";
|
||||
btn.className = "git-source-remove";
|
||||
btn.setAttribute("aria-label", `Remove git source: ${s.url}`);
|
||||
btn.setAttribute("aria-label", `Remove ${kindLabel} source: ${value}`);
|
||||
btn.innerHTML = REMOVE_ICON + "<span>Remove</span>";
|
||||
const rowError = document.createElement("span");
|
||||
rowError.className = "git-source-row-error";
|
||||
rowError.setAttribute("role", "alert");
|
||||
rowError.hidden = true;
|
||||
btn.addEventListener("click", () => removeSource(s, btn, rowError));
|
||||
btn.addEventListener("click", () => removeSource(s, btn, rowError, kindLabel));
|
||||
actTd.append(btn, rowError);
|
||||
} else {
|
||||
const tag = document.createElement("span");
|
||||
@@ -206,9 +232,9 @@ function makeRow(s) {
|
||||
* (phase scope boundary), so the confirm says exactly that. Cancel →
|
||||
* nothing; a failed delete → per-row role="alert" error + re-enabled
|
||||
* button (never a stuck row); success → the list reloads. */
|
||||
async function removeSource(s, btn, rowError) {
|
||||
async function removeSource(s, btn, rowError, kindLabel) {
|
||||
const ok = window.confirm(
|
||||
"Remove this git source from the list? Its documents stay indexed until the next sync prunes them.",
|
||||
`Remove this ${kindLabel} source from the list? Its documents stay indexed until the next sync prunes them.`,
|
||||
);
|
||||
if (!ok) return;
|
||||
btn.disabled = true; // one delete per click
|
||||
@@ -216,43 +242,55 @@ async function removeSource(s, btn, rowError) {
|
||||
try {
|
||||
const r = await fetch(`/api/git-sources/${encodeURIComponent(s.id)}`, { method: "DELETE" });
|
||||
if (!r.ok) {
|
||||
rowError.textContent = await apiDetail(r, "Could not remove the git source — try again.");
|
||||
rowError.textContent = await apiDetail(r, `Could not remove the ${kindLabel} source — try again.`);
|
||||
rowError.hidden = false;
|
||||
btn.disabled = false;
|
||||
return;
|
||||
}
|
||||
announce("Git source removed.");
|
||||
announce("Source removed.");
|
||||
await loadSources(); // 204: the server confirmed — the list re-renders
|
||||
} catch {
|
||||
rowError.textContent = "Could not remove the git source — is the app reachable?";
|
||||
rowError.textContent = `Could not remove the ${kindLabel} source — is the app reachable?`;
|
||||
rowError.hidden = false;
|
||||
btn.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
/* ---------- add (POST /api/git-sources) ---------- */
|
||||
|
||||
if (formEl && urlInput && addBtn) {
|
||||
formEl.addEventListener("submit", async (e) => {
|
||||
/* ---------- add (POST /api/git-sources) — both forms, one lifecycle
|
||||
* (the local form is phase 38) ----------
|
||||
* The git form posts {url}; the local form posts {kind:"local",path}.
|
||||
* wireAddForm gives both the §7.4 never-stale lifecycle: while the
|
||||
* request is out the button disables + relabels "Adding…" and
|
||||
* re-enables (idle label restored) on success AND failure. 201 clears
|
||||
* the input, reloads the list, and focuses the new row's Remove button
|
||||
* (a11y); a failure (409 duplicate, 422 validation) shows the server
|
||||
* detail inline under the form (role="alert", 422 shape-aware via
|
||||
* apiDetail) and keeps the input — the fix is one edit, not a re-type.
|
||||
* Git 409/422 details are fixed generic strings (credential safety);
|
||||
* local details name the path (not a secret). */
|
||||
function wireAddForm(opts) {
|
||||
const { form, input, btn, error } = opts;
|
||||
if (!form || !input || !btn) return;
|
||||
form.addEventListener("submit", async (e) => {
|
||||
e.preventDefault();
|
||||
// Client-side non-empty check (the input is `required` too — the
|
||||
// browser's native prompt is the first line, this one the second).
|
||||
const url = urlInput.value.trim();
|
||||
if (!url) {
|
||||
if (addError) {
|
||||
addError.textContent = "Enter a git URL to add.";
|
||||
addError.hidden = false;
|
||||
const value = input.value.trim();
|
||||
if (!value) {
|
||||
if (error) {
|
||||
error.textContent = opts.emptyMessage;
|
||||
error.hidden = false;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (addError) addError.hidden = true; // a new attempt starts clean
|
||||
addBtn.disabled = true; // §7.4: one POST per click
|
||||
addBtn.textContent = "Adding…";
|
||||
if (error) error.hidden = true; // a new attempt starts clean
|
||||
btn.disabled = true; // §7.4: one POST per click
|
||||
btn.textContent = "Adding…";
|
||||
try {
|
||||
const r = await fetch("/api/git-sources", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ url }),
|
||||
body: JSON.stringify(opts.body(value)),
|
||||
});
|
||||
if (r.ok) {
|
||||
let createdId = null;
|
||||
@@ -261,31 +299,57 @@ if (formEl && urlInput && addBtn) {
|
||||
} catch {
|
||||
/* the 201 body is advisory — the reload is the truth */
|
||||
}
|
||||
urlInput.value = ""; // 201: the source is stored
|
||||
announce("Git source added.");
|
||||
input.value = ""; // 201: the source is stored
|
||||
announce(opts.addedMessage);
|
||||
await loadSources(); // the new row lands in the table
|
||||
focusNewRow(createdId); // a11y: land the caret on the new row
|
||||
return;
|
||||
}
|
||||
// 409 duplicate / 422 shape / anything else: the server detail
|
||||
// inline (never echoing a URL the server wouldn't), form kept —
|
||||
// the input survives so the fix is one edit, not a re-type.
|
||||
if (addError) {
|
||||
addError.textContent = await apiDetail(r, "Could not add the git source — try again.");
|
||||
addError.hidden = false;
|
||||
// 409 duplicate / 422 validation / anything else: the server
|
||||
// detail inline, form kept — the input survives so the fix is
|
||||
// one edit, not a re-type.
|
||||
if (error) {
|
||||
error.textContent = await apiDetail(r, opts.failMessage);
|
||||
error.hidden = false;
|
||||
}
|
||||
} catch {
|
||||
if (addError) {
|
||||
addError.textContent = "Could not add the git source — is the app reachable?";
|
||||
addError.hidden = false;
|
||||
if (error) {
|
||||
error.textContent = opts.networkMessage;
|
||||
error.hidden = false;
|
||||
}
|
||||
} finally {
|
||||
addBtn.disabled = false; // never stale — success OR failure
|
||||
addBtn.textContent = "Add source";
|
||||
btn.disabled = false; // never stale — success OR failure
|
||||
btn.textContent = opts.idleLabel;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
wireAddForm({
|
||||
form: formEl,
|
||||
input: urlInput,
|
||||
btn: addBtn,
|
||||
error: addError,
|
||||
body: (url) => ({ url }),
|
||||
emptyMessage: "Enter a git URL to add.",
|
||||
failMessage: "Could not add the git source — try again.",
|
||||
networkMessage: "Could not add the git source — is the app reachable?",
|
||||
addedMessage: "Git source added.",
|
||||
idleLabel: "Add source",
|
||||
});
|
||||
|
||||
wireAddForm({
|
||||
form: localFormEl,
|
||||
input: pathInput,
|
||||
btn: localAddBtn,
|
||||
error: localAddError,
|
||||
body: (path) => ({ kind: "local", path }),
|
||||
emptyMessage: "Enter a directory path to add.",
|
||||
failMessage: "Could not add the local directory — try again.",
|
||||
networkMessage: "Could not add the local directory — is the app reachable?",
|
||||
addedMessage: "Local source added.",
|
||||
idleLabel: "Add directory",
|
||||
});
|
||||
|
||||
/* After a successful add, focus the new row's Remove button so the
|
||||
keyboard/screen-reader user lands where the new data is. The 201
|
||||
body carries the row id (tr[data-id]); without one, the first row
|
||||
|
||||
Reference in New Issue
Block a user