feat(admin): one-click sources sync — admin-only button triggers git clone/pull + re-import + KB overview refresh with polled live status

This commit is contained in:
2026-08-25 21:39:38 -04:00
parent 0654b304e1
commit 52136fe307
13 changed files with 1621 additions and 2 deletions
+38
View File
@@ -287,6 +287,44 @@ BOR_SOURCES_DIR=~/bor-sources # default; each repo lands in <dir>/<repo-name>/
**nothing** (no partial junk). Fix the URL/connectivity and re-run — the
other checkouts stay on disk and are pulled as usual.
### Sync from the UI
The **Sync sources** button on the **Sources** page — visible to the
**admin only** (anonymous visitors never see it) — runs the whole
git-source refresh in one click, in-process:
1. **clone/pull** every `BOR_GIT_SOURCES` repo (the same
`clone_or_pull` the CLI uses — shallow clone on first run,
`git pull --ff-only` afterwards);
2. **re-import with prune** — the `--prune` equivalent, so files deleted
upstream leave the index (the button is the canonical "mirror the
repos" action); the sha256 delta still skips unchanged files, so an
unchanged re-sync re-embeds nothing;
3. **regenerate the KB overview** (the `<knowledge_base>` outline every
chat turn injects) — but only when the import actually changed the
knowledge base.
- **Prerequisites:** `BOR_GIT_SOURCES` must be set — an unset/empty list
fails the sync loudly ("no git sources configured"), because the button
targets the git repos only (manual `--source` directories have no repo
to clone) — and `git` must be on the app's `PATH`.
- **States:** clicking starts the run (`202`) and the button goes
disabled with **Syncing…** (spinning icon) while the page polls
`GET /api/sync/status` every 2 s. There is deliberately **no
client-side timeout** — a clone + embed can legitimately take minutes,
so the poll is the feedback loop and the server state is authoritative.
On success the button settles to **Synced HH:MM** with the last result
in a live region (`1 added`, `0 added · 1 unchanged`, …); on failure it
re-enables (retry-ready) and a red error banner names the failure (git's
stderr, with any embedded credentials masked).
- **One sync at a time:** a second trigger while a run is in flight gets
`409` ("a sync is already running"); the UI adopts the in-flight run
instead of starting a second one, and a page reload mid-sync re-attaches
to it the same way.
- **Idempotent:** re-syncing unchanged repos is a no-op — fast-forward
pull, hash skip, and the overview is left alone (its regeneration is
change-gated).
## Checking retrieval quality
Ask the *real* pipeline (live aipi embeddings + the current KB) whether a