feat(admin): one-click sources sync — admin-only button triggers git clone/pull + re-import + KB overview refresh with polled live status
This commit is contained in:
@@ -287,6 +287,44 @@ BOR_SOURCES_DIR=~/bor-sources # default; each repo lands in <dir>/<repo-name>/
|
||||
**nothing** (no partial junk). Fix the URL/connectivity and re-run — the
|
||||
other checkouts stay on disk and are pulled as usual.
|
||||
|
||||
### Sync from the UI
|
||||
|
||||
The **Sync sources** button on the **Sources** page — visible to the
|
||||
**admin only** (anonymous visitors never see it) — runs the whole
|
||||
git-source refresh in one click, in-process:
|
||||
|
||||
1. **clone/pull** every `BOR_GIT_SOURCES` repo (the same
|
||||
`clone_or_pull` the CLI uses — shallow clone on first run,
|
||||
`git pull --ff-only` afterwards);
|
||||
2. **re-import with prune** — the `--prune` equivalent, so files deleted
|
||||
upstream leave the index (the button is the canonical "mirror the
|
||||
repos" action); the sha256 delta still skips unchanged files, so an
|
||||
unchanged re-sync re-embeds nothing;
|
||||
3. **regenerate the KB overview** (the `<knowledge_base>` outline every
|
||||
chat turn injects) — but only when the import actually changed the
|
||||
knowledge base.
|
||||
|
||||
- **Prerequisites:** `BOR_GIT_SOURCES` must be set — an unset/empty list
|
||||
fails the sync loudly ("no git sources configured"), because the button
|
||||
targets the git repos only (manual `--source` directories have no repo
|
||||
to clone) — and `git` must be on the app's `PATH`.
|
||||
- **States:** clicking starts the run (`202`) and the button goes
|
||||
disabled with **Syncing…** (spinning icon) while the page polls
|
||||
`GET /api/sync/status` every 2 s. There is deliberately **no
|
||||
client-side timeout** — a clone + embed can legitimately take minutes,
|
||||
so the poll is the feedback loop and the server state is authoritative.
|
||||
On success the button settles to **Synced HH:MM** with the last result
|
||||
in a live region (`1 added`, `0 added · 1 unchanged`, …); on failure it
|
||||
re-enables (retry-ready) and a red error banner names the failure (git's
|
||||
stderr, with any embedded credentials masked).
|
||||
- **One sync at a time:** a second trigger while a run is in flight gets
|
||||
`409` ("a sync is already running"); the UI adopts the in-flight run
|
||||
instead of starting a second one, and a page reload mid-sync re-attaches
|
||||
to it the same way.
|
||||
- **Idempotent:** re-syncing unchanged repos is a no-op — fast-forward
|
||||
pull, hash skip, and the overview is left alone (its regeneration is
|
||||
change-gated).
|
||||
|
||||
## Checking retrieval quality
|
||||
|
||||
Ask the *real* pipeline (live aipi embeddings + the current KB) whether a
|
||||
|
||||
Reference in New Issue
Block a user