feat(admin): one-click sources sync — admin-only button triggers git clone/pull + re-import + KB overview refresh with polled live status
This commit is contained in:
@@ -0,0 +1,92 @@
|
|||||||
|
# Story: Admin Sync Button (One-Click Doc Import Sync)
|
||||||
|
|
||||||
|
**Phase:** `32_admin_sync_button.md` · **E2E:** `tests/e2e/test_sync_button.py`
|
||||||
|
|
||||||
|
## Narrative
|
||||||
|
|
||||||
|
As **the admin (owner)**, I don't want to SSH in and hand-run the import
|
||||||
|
script every time my notes repos move. I want **a button that only I can
|
||||||
|
see** — on the Sources page — that **triggers a doc import sync by
|
||||||
|
cloning the relevant repos and then running the import script**, with
|
||||||
|
live feedback so I always know whether it is running, what it changed,
|
||||||
|
or why it failed.
|
||||||
|
|
||||||
|
- **Given** `BOR_GIT_SOURCES` is set (the git repos) and I am signed in
|
||||||
|
as the admin
|
||||||
|
- **When** I click **Sync sources** on the Sources page
|
||||||
|
- **Then** the app clones/pulls each repo, re-imports with prune (the
|
||||||
|
`--prune` equivalent — the canonical "mirror the repos" action), and
|
||||||
|
refreshes the KB overview when anything changed — while the button
|
||||||
|
reports the whole lifecycle (**Syncing…** → **Synced HH:MM** + counts,
|
||||||
|
or an error banner naming the failure) and never sits stale or stuck.
|
||||||
|
Anonymous visitors never see the button, and the sync endpoints answer
|
||||||
|
them with 403.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
1. **Admin-only visibility.** The button ships `hidden` in
|
||||||
|
`sources.html` (anonymous-safe) and `header.js` reveals it for the
|
||||||
|
admin on the SAME cached whoami that reveals `#nav-sources` /
|
||||||
|
`#nav-tuning` (one fetch, no extra whoami call); anonymous users
|
||||||
|
never see it.
|
||||||
|
2. **The sync API (A10 extended, A12 in-process).** `POST /api/sync`
|
||||||
|
(admin only) starts one background task — `clone_or_pull` each
|
||||||
|
`BOR_GIT_SOURCES` repo (phase 28, reused) → `import_sources(prune=True)`
|
||||||
|
→ `regenerate_overview` when the KB changed (phase 31) — and returns
|
||||||
|
`202`. A second trigger while a run is in flight returns
|
||||||
|
`409 {"detail": "a sync is already running"}` (one sync at a time).
|
||||||
|
`GET /api/sync/status` (admin only) reports `idle | running |
|
||||||
|
success | failed` with ISO-8601 timestamps and the run's `detail` /
|
||||||
|
`error`. Both endpoints answer anonymous callers with 403.
|
||||||
|
3. **Loud failure.** An unset/empty `BOR_GIT_SOURCES` fails the sync with
|
||||||
|
"no git sources configured (BOR_GIT_SOURCES)" (manual `--source` dirs
|
||||||
|
have no repo to clone); a `GitSyncError` fails the run with git's
|
||||||
|
stderr (the repo named, credentials masked) before any import.
|
||||||
|
4. **The §7.4 "never stale" lifecycle.** Click → `202` → button disabled
|
||||||
|
with **Syncing…** (spinning icon, `aria-busy`) + a 2 s poll of the
|
||||||
|
status endpoint — the ONLY feedback timer; there is no client-side
|
||||||
|
hard timeout (a sync can legitimately run for minutes; the server
|
||||||
|
state is authoritative). Success → enabled, **Synced HH:MM** (local
|
||||||
|
time of `finished_at`) + the last result in `#sync-result`
|
||||||
|
(`role="status"` / `aria-live="polite"`; "added" always announced,
|
||||||
|
zero terms omitted, a no-op run reads `0 added · 1 unchanged`).
|
||||||
|
Failure → enabled, retry-ready **Sync sources** + the `role="alert"`
|
||||||
|
banner naming the error. A `409` adopts the in-flight run (never a
|
||||||
|
second poll loop); a reload mid-sync re-attaches to the running run;
|
||||||
|
a `403` hides the button (defense in depth).
|
||||||
|
5. **Idempotent.** Re-syncing an unchanged repo is a fast-forward pull +
|
||||||
|
sha256 hash skip — nothing re-embedded, the overview left alone
|
||||||
|
(change-gated), the result `0 added · 1 unchanged`.
|
||||||
|
6. **Quality gates.** Integration (`tests/integration/test_sync_api.py`):
|
||||||
|
anonymous 403s; idle → running → success/failed transitions with git +
|
||||||
|
import + overview mocked; 409 double trigger; `GitSyncError` →
|
||||||
|
`failed` with the repo named and the import never called; `prune=True`
|
||||||
|
asserted. Unit (`tests/unit/test_sync_button.py`, frontend-assertion):
|
||||||
|
the ship-hidden markup, the header reveal, the state machine (2 s
|
||||||
|
poll, 202/409/403 branches, terminal labels, single-poll guard, no
|
||||||
|
client timeout), the CSS states (spin + reduced-motion opt-out,
|
||||||
|
disabled, focus-visible, contrast ≥ 4.5:1, 44 px touch floor).
|
||||||
|
Coverage `app/` > 90 % (`app/api/sync.py` fully covered).
|
||||||
|
7. **E2E (this story's gate).** `tests/e2e/test_sync_button.py`, run in
|
||||||
|
isolation: a real local `file://` git fixture repo (deterministic, no
|
||||||
|
network — git is a documented environment prerequisite, phase 28) with
|
||||||
|
the mock LLM proves the admin-only visibility, the full lifecycle
|
||||||
|
against the REAL clone → import → overview path (including the
|
||||||
|
idempotent second run and the fresh `kb_overview` row), and the 409
|
||||||
|
double trigger.
|
||||||
|
|
||||||
|
## Playwright Mapping Rule
|
||||||
|
`tests/e2e/test_sync_button.py` — run in isolation (Chromium +
|
||||||
|
`podman compose up -d db` + git on PATH; mock LLM, no live aipi). The
|
||||||
|
module overrides the session app fixture with per-module env
|
||||||
|
(`BOR_GIT_SOURCES=file://<fixture repo>`, its own `BOR_SOURCES_DIR`) and
|
||||||
|
truncates the KB tables before each test (the E2E isolation pattern):
|
||||||
|
|
||||||
|
1. `test_anonymous_sees_no_button` → AC 1 + 2 (the button never leaves
|
||||||
|
`hidden`; both endpoints 403).
|
||||||
|
2. `test_admin_sync_lifecycle` → AC 2 + 4 + 5 (button visible for the
|
||||||
|
admin; click → **Syncing…** (disabled) → **Synced HH:MM** +
|
||||||
|
`1 added`; the fixture path `notes/sync-fixture.md` in the Sources
|
||||||
|
table; the `kb_overview` row fresh and non-empty (DB check); the
|
||||||
|
idempotent second run → `0 added · 1 unchanged`).
|
||||||
|
3. `test_double_trigger_409` → AC 2 (second trigger while running → 409
|
||||||
|
with the exact detail; the single in-flight run still completes).
|
||||||
@@ -287,6 +287,44 @@ BOR_SOURCES_DIR=~/bor-sources # default; each repo lands in <dir>/<repo-name>/
|
|||||||
**nothing** (no partial junk). Fix the URL/connectivity and re-run — the
|
**nothing** (no partial junk). Fix the URL/connectivity and re-run — the
|
||||||
other checkouts stay on disk and are pulled as usual.
|
other checkouts stay on disk and are pulled as usual.
|
||||||
|
|
||||||
|
### Sync from the UI
|
||||||
|
|
||||||
|
The **Sync sources** button on the **Sources** page — visible to the
|
||||||
|
**admin only** (anonymous visitors never see it) — runs the whole
|
||||||
|
git-source refresh in one click, in-process:
|
||||||
|
|
||||||
|
1. **clone/pull** every `BOR_GIT_SOURCES` repo (the same
|
||||||
|
`clone_or_pull` the CLI uses — shallow clone on first run,
|
||||||
|
`git pull --ff-only` afterwards);
|
||||||
|
2. **re-import with prune** — the `--prune` equivalent, so files deleted
|
||||||
|
upstream leave the index (the button is the canonical "mirror the
|
||||||
|
repos" action); the sha256 delta still skips unchanged files, so an
|
||||||
|
unchanged re-sync re-embeds nothing;
|
||||||
|
3. **regenerate the KB overview** (the `<knowledge_base>` outline every
|
||||||
|
chat turn injects) — but only when the import actually changed the
|
||||||
|
knowledge base.
|
||||||
|
|
||||||
|
- **Prerequisites:** `BOR_GIT_SOURCES` must be set — an unset/empty list
|
||||||
|
fails the sync loudly ("no git sources configured"), because the button
|
||||||
|
targets the git repos only (manual `--source` directories have no repo
|
||||||
|
to clone) — and `git` must be on the app's `PATH`.
|
||||||
|
- **States:** clicking starts the run (`202`) and the button goes
|
||||||
|
disabled with **Syncing…** (spinning icon) while the page polls
|
||||||
|
`GET /api/sync/status` every 2 s. There is deliberately **no
|
||||||
|
client-side timeout** — a clone + embed can legitimately take minutes,
|
||||||
|
so the poll is the feedback loop and the server state is authoritative.
|
||||||
|
On success the button settles to **Synced HH:MM** with the last result
|
||||||
|
in a live region (`1 added`, `0 added · 1 unchanged`, …); on failure it
|
||||||
|
re-enables (retry-ready) and a red error banner names the failure (git's
|
||||||
|
stderr, with any embedded credentials masked).
|
||||||
|
- **One sync at a time:** a second trigger while a run is in flight gets
|
||||||
|
`409` ("a sync is already running"); the UI adopts the in-flight run
|
||||||
|
instead of starting a second one, and a page reload mid-sync re-attaches
|
||||||
|
to it the same way.
|
||||||
|
- **Idempotent:** re-syncing unchanged repos is a no-op — fast-forward
|
||||||
|
pull, hash skip, and the overview is left alone (its regeneration is
|
||||||
|
change-gated).
|
||||||
|
|
||||||
## Checking retrieval quality
|
## Checking retrieval quality
|
||||||
|
|
||||||
Ask the *real* pipeline (live aipi embeddings + the current KB) whether a
|
Ask the *real* pipeline (live aipi embeddings + the current KB) whether a
|
||||||
|
|||||||
+177
@@ -0,0 +1,177 @@
|
|||||||
|
"""Sources sync API — one-click KB mirror (phase 32, task 01).
|
||||||
|
|
||||||
|
Admin-only ``POST /api/sync`` + ``GET /api/sync/status`` behind the
|
||||||
|
existing :func:`app.core.auth.require_admin` (A10 extended, phase 16
|
||||||
|
pattern — the public API surface stays stateless, the signed cookie
|
||||||
|
remains the only session state, same as ``/api/steering``).
|
||||||
|
|
||||||
|
The button's backend runs the full document sync **in-process** (A12
|
||||||
|
untouched — no queue, no new services): one ``asyncio`` background task
|
||||||
|
plus a module-level :class:`SyncStatus` that the UI polls every 2 s
|
||||||
|
(task 02). One sync at a time — ``POST`` while a run is in flight is
|
||||||
|
409; the status object is authoritative, so the UI can never sit on a
|
||||||
|
stale button state (§7.4 adaptation, phase locked decisions).
|
||||||
|
|
||||||
|
Pipeline (the canonical "mirror the repos" action — phase locked
|
||||||
|
decisions):
|
||||||
|
|
||||||
|
1. resolve the ``BOR_GIT_SOURCES`` URLs — empty/missing fails loudly
|
||||||
|
(``no git sources configured``) instead of silently importing the
|
||||||
|
legacy local directories;
|
||||||
|
2. :func:`scripts.git_sync.clone_or_pull` each repo into
|
||||||
|
``BOR_SOURCES_DIR/<repo-name>/`` (phase 28 — reused, not
|
||||||
|
re-implemented; a failing repo aborts before any import);
|
||||||
|
3. ``import_sources(..., prune=True)`` — prune so files deleted
|
||||||
|
upstream leave the index (the CLI's no-prune default is unchanged);
|
||||||
|
4. when the import changed the KB (added + updated > 0),
|
||||||
|
``regenerate_overview`` refreshes the single ``kb_overview`` row
|
||||||
|
(phase 31 trigger, best-effort inside).
|
||||||
|
|
||||||
|
Status is in memory: a restart mid-sync loses the running state
|
||||||
|
(accepted — the next click re-syncs idempotently).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.core.auth import require_admin
|
||||||
|
from app.rag.importer import ImportSummary, import_sources
|
||||||
|
from app.rag.llm import LLMClient
|
||||||
|
from app.rag.overview import regenerate_overview
|
||||||
|
from scripts.git_sync import GitSyncError, clone_or_pull
|
||||||
|
from scripts.import_docs import repo_name
|
||||||
|
|
||||||
|
logger = logging.getLogger("app.api.sync")
|
||||||
|
|
||||||
|
router = APIRouter(
|
||||||
|
prefix="/sync",
|
||||||
|
tags=["sync"],
|
||||||
|
dependencies=[Depends(require_admin)], # phase 16 pattern: admin-only surface
|
||||||
|
)
|
||||||
|
|
||||||
|
#: ``user:pass@`` inside any error text (git stderr, endpoint URLs) —
|
||||||
|
#: masked so a sync failure can never leak credentials into the UI.
|
||||||
|
_CREDS_RE = re.compile(r"[A-Za-z0-9._~%*-]+:[A-Za-z0-9._~%*-]+@")
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_error(message: str) -> str:
|
||||||
|
"""Mask credentials embedded in an error string (no secrets in the UI).
|
||||||
|
|
||||||
|
Git's stderr is otherwise surfaced verbatim (phase locked decisions) —
|
||||||
|
it names the failing repo and git's reason, which is what the admin
|
||||||
|
needs to fix things.
|
||||||
|
"""
|
||||||
|
return _CREDS_RE.sub("*****@", message)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SyncStatus:
|
||||||
|
"""In-memory state of the (at most one) in-flight sync run.
|
||||||
|
|
||||||
|
``state`` is a four-state machine: ``idle`` (never run / reset),
|
||||||
|
``running``, ``success``, ``failed``. Terminal states carry the run's
|
||||||
|
``detail`` (success) or ``error`` (failure) so the UI can render the
|
||||||
|
last result after a page reload (task 02's re-attach behavior).
|
||||||
|
"""
|
||||||
|
|
||||||
|
state: Literal["idle", "running", "success", "failed"] = "idle"
|
||||||
|
started_at: datetime | None = None
|
||||||
|
finished_at: datetime | None = None
|
||||||
|
detail: dict[str, Any] = field(default_factory=dict)
|
||||||
|
error: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
_status = SyncStatus()
|
||||||
|
_task: asyncio.Task[None] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/status")
|
||||||
|
def sync_status() -> dict[str, Any]:
|
||||||
|
"""Current sync state (the UI polls this every 2 s — task 02).
|
||||||
|
|
||||||
|
``started_at`` / ``finished_at`` are ISO-8601 strings or null.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"state": _status.state,
|
||||||
|
"started_at": _status.started_at.isoformat() if _status.started_at else None,
|
||||||
|
"finished_at": _status.finished_at.isoformat() if _status.finished_at else None,
|
||||||
|
"detail": _status.detail,
|
||||||
|
"error": _status.error,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("", status_code=202)
|
||||||
|
async def start_sync() -> dict[str, str]:
|
||||||
|
"""Start the clone → import → overview sync as a background task.
|
||||||
|
|
||||||
|
202 + ``sync started`` kicks off :func:`_run_sync` on the app's event
|
||||||
|
loop. 409 when a run is already in flight (one sync at a time — the
|
||||||
|
status endpoint is the single source of truth for the run, and the
|
||||||
|
UI re-attaches to it rather than starting a second one).
|
||||||
|
"""
|
||||||
|
global _task
|
||||||
|
if _task is not None and not _task.done():
|
||||||
|
raise HTTPException(status_code=409, detail="a sync is already running")
|
||||||
|
_task = asyncio.create_task(_run_sync())
|
||||||
|
return {"detail": "sync started"}
|
||||||
|
|
||||||
|
|
||||||
|
async def _run_sync() -> None:
|
||||||
|
"""The full sync pipeline, one in-process background task.
|
||||||
|
|
||||||
|
Every failure mode (git, embeddings, anything else) lands in the
|
||||||
|
``failed`` state with a sanitized ``error`` string — a background
|
||||||
|
task must die in state, never as an unobserved exception.
|
||||||
|
``CancelledError`` is deliberately *not* caught: app shutdown
|
||||||
|
cancels the task, and swallowing that would mask a real stop.
|
||||||
|
"""
|
||||||
|
_status.state = "running"
|
||||||
|
_status.started_at = datetime.now(UTC)
|
||||||
|
_status.finished_at = None
|
||||||
|
_status.detail = {}
|
||||||
|
_status.error = None
|
||||||
|
try:
|
||||||
|
settings = get_settings()
|
||||||
|
git_urls = settings.git_source_list
|
||||||
|
if not git_urls:
|
||||||
|
# The button targets BOR_GIT_SOURCES only (manual --source
|
||||||
|
# dirs have no repo to clone) — an empty config fails loudly
|
||||||
|
# instead of silently importing the legacy directories.
|
||||||
|
raise GitSyncError("no git sources configured (BOR_GIT_SOURCES)")
|
||||||
|
logger.info("sync: started repos=%d", len(git_urls))
|
||||||
|
sources_root = Path(settings.sources_dir).expanduser()
|
||||||
|
sources = [clone_or_pull(url, sources_root / repo_name(url)) for url in git_urls]
|
||||||
|
llm = LLMClient()
|
||||||
|
summary: ImportSummary = await import_sources(sources, llm, prune=True)
|
||||||
|
overview = False
|
||||||
|
if summary.added + summary.updated > 0:
|
||||||
|
overview = await regenerate_overview(llm)
|
||||||
|
_status.state = "success"
|
||||||
|
_status.finished_at = datetime.now(UTC)
|
||||||
|
_status.detail = {
|
||||||
|
"files": summary.files,
|
||||||
|
"added": summary.added,
|
||||||
|
"updated": summary.updated,
|
||||||
|
"unchanged": summary.unchanged,
|
||||||
|
"pruned": summary.pruned,
|
||||||
|
"errors": summary.errors,
|
||||||
|
"chunks": summary.chunks,
|
||||||
|
"summaries": summary.summaries,
|
||||||
|
"summary_errors": summary.summary_errors,
|
||||||
|
"overview": overview,
|
||||||
|
}
|
||||||
|
logger.info("sync: done detail=%s", _status.detail)
|
||||||
|
except Exception as e: # noqa: BLE001 — a background task dies in state, see above
|
||||||
|
logger.exception("sync: failed")
|
||||||
|
_status.state = "failed"
|
||||||
|
_status.finished_at = datetime.now(UTC)
|
||||||
|
_status.error = _sanitize_error(str(e))
|
||||||
@@ -25,6 +25,7 @@ from app.api.docs import router as docs_router
|
|||||||
from app.api.health import router as health_router
|
from app.api.health import router as health_router
|
||||||
from app.api.steering import router as steering_router
|
from app.api.steering import router as steering_router
|
||||||
from app.api.suggestions import router as suggestions_router
|
from app.api.suggestions import router as suggestions_router
|
||||||
|
from app.api.sync import router as sync_router
|
||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
from app.core.auth import ensure_admin_configured
|
from app.core.auth import ensure_admin_configured
|
||||||
from app.core.debugging import configure_debugging
|
from app.core.debugging import configure_debugging
|
||||||
@@ -64,6 +65,7 @@ def create_app() -> FastAPI:
|
|||||||
app.include_router(docs_router, prefix="/api")
|
app.include_router(docs_router, prefix="/api")
|
||||||
app.include_router(chat_router, prefix="/api")
|
app.include_router(chat_router, prefix="/api")
|
||||||
app.include_router(steering_router, prefix="/api")
|
app.include_router(steering_router, prefix="/api")
|
||||||
|
app.include_router(sync_router, prefix="/api")
|
||||||
|
|
||||||
static_dir = Path(settings.static_dir).resolve()
|
static_dir = Path(settings.static_dir).resolve()
|
||||||
if static_dir.is_dir():
|
if static_dir.is_dir():
|
||||||
|
|||||||
@@ -12,7 +12,10 @@
|
|||||||
* every page that has a nav (chat, sources, tuning, login),
|
* every page that has a nav (chat, sources, tuning, login),
|
||||||
* revealed for admin. The links SHIP hidden in the HTML
|
* revealed for admin. The links SHIP hidden in the HTML
|
||||||
* (anonymous-safe default — the phase-16 "absent, not hidden"
|
* (anonymous-safe default — the phase-16 "absent, not hidden"
|
||||||
* spirit), so no anonymous user ever sees one for a frame;
|
* spirit), so no anonymous user ever sees one for a frame; and the
|
||||||
|
* Sources page's "Sync sources" button (#sync-btn, phase 32) —
|
||||||
|
* the same ship-hidden / reveal-for-admin contract on the SAME
|
||||||
|
* cached whoami (one fetch, no extra request);
|
||||||
* • the sign-out click binding (POST /api/logout → reload) — moved
|
* • the sign-out click binding (POST /api/logout → reload) — moved
|
||||||
* here from app.js so there is exactly one implementation;
|
* here from app.js so there is exactly one implementation;
|
||||||
* • clearChatStorage() — the phase-14 conversation key, for the
|
* • clearChatStorage() — the phase-14 conversation key, for the
|
||||||
@@ -68,6 +71,10 @@ export async function initSharedHeader() {
|
|||||||
// same ship-hidden / reveal-for-admin contract as the Sources link.
|
// same ship-hidden / reveal-for-admin contract as the Sources link.
|
||||||
const navTuning = document.querySelector("#nav-tuning");
|
const navTuning = document.querySelector("#nav-tuning");
|
||||||
if (navTuning) navTuning.hidden = !admin;
|
if (navTuning) navTuning.hidden = !admin;
|
||||||
|
// Phase 32: the Sources page's "Sync sources" button — admin-only,
|
||||||
|
// revealed on this same cached whoami (anonymous users never see it).
|
||||||
|
const syncBtn = document.querySelector("#sync-btn");
|
||||||
|
if (syncBtn) syncBtn.hidden = !admin;
|
||||||
return admin;
|
return admin;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -146,6 +146,236 @@ function showEmpty() {
|
|||||||
if (tableWrap) tableWrap.hidden = true;
|
if (tableWrap) tableWrap.hidden = true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ---------- Phase 32: the admin "Sync sources" button (§7.4) ----------
|
||||||
|
* The "never stale" lifecycle for a long background job:
|
||||||
|
*
|
||||||
|
* idle → click → POST /api/sync
|
||||||
|
* 202 → "Syncing…" (disabled, aria-busy, spinning icon) + a
|
||||||
|
* 2 s poll of GET /api/sync/status — the feedback loop;
|
||||||
|
* 409 adopts the in-flight run the same way (one poll
|
||||||
|
* loop at a time, never two);
|
||||||
|
* success → "Synced HH:MM" + last-result counts in #sync-result
|
||||||
|
* (aria-live — announced to screen readers) + the
|
||||||
|
* catalog re-fetches live (never a stale table);
|
||||||
|
* failed → "Sync sources" (retry-ready) + the role="alert"
|
||||||
|
* banner naming the error.
|
||||||
|
*
|
||||||
|
* NO client-side hard timeout (phase locked decision): a sync can
|
||||||
|
* legitimately run for minutes (clone + embed), so the 2 s poll is the
|
||||||
|
* feedback loop and the server state is authoritative — the button is
|
||||||
|
* disabled until the run reaches a terminal state, so it can never sit
|
||||||
|
* stale OR stuck. On load (admin only) the page re-attaches: a running
|
||||||
|
* run re-enters the running state (reload mid-sync), a terminal run
|
||||||
|
* renders its last result. A 403 anywhere hides the button (defense in
|
||||||
|
* depth — header.js's whoami reveal is the primary gate).
|
||||||
|
*/
|
||||||
|
const syncBtn = document.querySelector("#sync-btn");
|
||||||
|
const syncLabel = document.querySelector("#sync-label");
|
||||||
|
const syncIcon = syncBtn ? syncBtn.querySelector(".sync-icon") : null;
|
||||||
|
const syncResult = document.querySelector("#sync-result");
|
||||||
|
const syncErrorBanner = document.querySelector("#sync-error-banner");
|
||||||
|
const syncErrorText = document.querySelector("#sync-error-text");
|
||||||
|
|
||||||
|
const SYNC_POLL_MS = 2000; // the 2 s status poll (task 02)
|
||||||
|
let syncPollTimer = null; // at most ONE live poll loop
|
||||||
|
|
||||||
|
function stopSyncPolling() {
|
||||||
|
if (syncPollTimer !== null) {
|
||||||
|
clearTimeout(syncPollTimer);
|
||||||
|
syncPollTimer = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function showSyncError(detail) {
|
||||||
|
if (syncErrorText) syncErrorText.textContent = detail || "The sync failed.";
|
||||||
|
if (syncErrorBanner) syncErrorBanner.hidden = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hideSyncError() {
|
||||||
|
if (syncErrorText) syncErrorText.textContent = "";
|
||||||
|
if (syncErrorBanner) syncErrorBanner.hidden = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* The local HH:MM of finished_at — 24-hour, locale-independent, so the
|
||||||
|
* "Synced 14:32" last-result label is deterministic. */
|
||||||
|
function fmtSyncTime(iso) {
|
||||||
|
const d = new Date(iso);
|
||||||
|
if (Number.isNaN(d.getTime())) return "";
|
||||||
|
const pad = (n) => String(n).padStart(2, "0");
|
||||||
|
return `${pad(d.getHours())}:${pad(d.getMinutes())}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* The last-result line for #sync-result (aria-live). "added" is ALWAYS
|
||||||
|
* announced (the run's headline term); "updated" / "pruned" only when
|
||||||
|
* they happened (zero terms omitted); "unchanged" whenever it is
|
||||||
|
* non-zero — or whenever nothing was added or updated, so a no-op
|
||||||
|
* re-sync reads "0 added · 1 unchanged" instead of an empty live
|
||||||
|
* region (the story gate's idempotency check). */
|
||||||
|
function fmtSyncResult(detail) {
|
||||||
|
const d = detail || {};
|
||||||
|
const added = d.added || 0;
|
||||||
|
const updated = d.updated || 0;
|
||||||
|
const parts = [`${added} added`];
|
||||||
|
if (updated > 0) parts.push(`${updated} updated`);
|
||||||
|
if ((d.unchanged || 0) > 0 || (added === 0 && updated === 0)) {
|
||||||
|
parts.push(`${d.unchanged || 0} unchanged`);
|
||||||
|
}
|
||||||
|
if ((d.pruned || 0) > 0) parts.push(`${d.pruned} pruned`);
|
||||||
|
return parts.join(" · ");
|
||||||
|
}
|
||||||
|
|
||||||
|
function enterRunningState() {
|
||||||
|
syncBtn.disabled = true;
|
||||||
|
syncBtn.setAttribute("aria-busy", "true");
|
||||||
|
if (syncIcon) syncIcon.classList.add("is-spinning");
|
||||||
|
syncLabel.textContent = "Syncing…";
|
||||||
|
if (syncResult) syncResult.textContent = "";
|
||||||
|
hideSyncError();
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Settle the button back to clickable + un-spun with the given label. */
|
||||||
|
function settleSyncButton(label) {
|
||||||
|
syncBtn.disabled = false;
|
||||||
|
syncBtn.removeAttribute("aria-busy");
|
||||||
|
if (syncIcon) syncIcon.classList.remove("is-spinning");
|
||||||
|
syncLabel.textContent = label;
|
||||||
|
}
|
||||||
|
|
||||||
|
function applySyncSuccess(status) {
|
||||||
|
const time = fmtSyncTime(status.finished_at);
|
||||||
|
settleSyncButton(time ? `Synced ${time}` : "Synced");
|
||||||
|
if (syncResult) syncResult.textContent = fmtSyncResult(status.detail);
|
||||||
|
hideSyncError();
|
||||||
|
// The KB just changed — refresh the catalog live so the table, stats,
|
||||||
|
// and empty state never sit stale under the "Synced" label (the sync is
|
||||||
|
// the page's own action; a reload should not be needed to see it).
|
||||||
|
loadDocs();
|
||||||
|
}
|
||||||
|
|
||||||
|
function applySyncFailure(status) {
|
||||||
|
settleSyncButton("Sync sources"); // retry-ready
|
||||||
|
if (syncResult) syncResult.textContent = "";
|
||||||
|
showSyncError(status.error);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* A run can only vanish with a server restart mid-sync (status resets
|
||||||
|
* to idle — the phase-accepted behavior): re-enable retry-ready with no
|
||||||
|
* banner (there is no error to name; the next click re-syncs).
|
||||||
|
* Idempotent — also the post-403 cleanup. */
|
||||||
|
function applySyncIdle() {
|
||||||
|
settleSyncButton("Sync sources");
|
||||||
|
if (syncResult) syncResult.textContent = "";
|
||||||
|
hideSyncError();
|
||||||
|
}
|
||||||
|
|
||||||
|
/* The 2 s poll loop — the ONLY feedback timer (no client-side hard
|
||||||
|
* timeout, phase locked decision). One tick at a time (re-scheduled
|
||||||
|
* only while the run is still live, so an in-flight fetch can never
|
||||||
|
* overlap the next tick), and startSyncPolling refuses to run a second
|
||||||
|
* loop (a 409 adoption or a reload never doubles the polling). */
|
||||||
|
function startSyncPolling() {
|
||||||
|
if (syncPollTimer !== null) return;
|
||||||
|
const tick = async () => {
|
||||||
|
let status = null;
|
||||||
|
let notAdmin = false;
|
||||||
|
try {
|
||||||
|
const r = await fetch("/api/sync/status");
|
||||||
|
if (r.status === 403) notAdmin = true;
|
||||||
|
else if (r.ok) status = await r.json();
|
||||||
|
} catch {
|
||||||
|
/* network blip — the next tick retries (no client timeout to trip) */
|
||||||
|
}
|
||||||
|
if (notAdmin) {
|
||||||
|
// Session lost mid-sync: defense in depth — hide the button.
|
||||||
|
stopSyncPolling();
|
||||||
|
syncBtn.hidden = true;
|
||||||
|
applySyncIdle();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status && status.state === "success") {
|
||||||
|
stopSyncPolling();
|
||||||
|
applySyncSuccess(status);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status && status.state === "failed") {
|
||||||
|
stopSyncPolling();
|
||||||
|
applySyncFailure(status);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status && status.state === "idle") {
|
||||||
|
// The run died with a server restart — retry-ready, no banner.
|
||||||
|
stopSyncPolling();
|
||||||
|
applySyncIdle();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
syncPollTimer = setTimeout(tick, SYNC_POLL_MS);
|
||||||
|
};
|
||||||
|
syncPollTimer = setTimeout(tick, SYNC_POLL_MS);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Click → POST /api/sync. 202 starts the run; 409 adopts the in-flight
|
||||||
|
* one (started elsewhere — e.g. a second tab); 403 hides the button
|
||||||
|
* (defense in depth); anything else names the failure in the banner and
|
||||||
|
* leaves the button retry-ready (the never-stale contract). */
|
||||||
|
async function startSync() {
|
||||||
|
let r;
|
||||||
|
try {
|
||||||
|
r = await fetch("/api/sync", { method: "POST" });
|
||||||
|
} catch {
|
||||||
|
showSyncError("Could not reach the server to start the sync — try again.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (r.status === 403) {
|
||||||
|
stopSyncPolling();
|
||||||
|
syncBtn.hidden = true;
|
||||||
|
applySyncIdle();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (r.status === 202 || r.status === 409) {
|
||||||
|
enterRunningState();
|
||||||
|
startSyncPolling();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let detail = "";
|
||||||
|
try {
|
||||||
|
detail = (await r.json()).detail || "";
|
||||||
|
} catch {
|
||||||
|
/* non-JSON error body */
|
||||||
|
}
|
||||||
|
showSyncError(detail || `The server refused to start the sync (${r.status}).`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Load-time re-attach (admin only — the IIFE runs this after the
|
||||||
|
* whoami gate): a running run re-enters the running state (the user may
|
||||||
|
* have reloaded mid-sync), a terminal run renders its last result, idle
|
||||||
|
* renders nothing. */
|
||||||
|
async function initSyncButton() {
|
||||||
|
if (!syncBtn) return;
|
||||||
|
let status;
|
||||||
|
try {
|
||||||
|
const r = await fetch("/api/sync/status");
|
||||||
|
if (r.status === 403) {
|
||||||
|
syncBtn.hidden = true; // defense in depth
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!r.ok) return;
|
||||||
|
status = await r.json();
|
||||||
|
} catch {
|
||||||
|
return; // network blip — the button stays idle and clickable
|
||||||
|
}
|
||||||
|
if (status.state === "running") {
|
||||||
|
enterRunningState();
|
||||||
|
startSyncPolling();
|
||||||
|
} else if (status.state === "success") {
|
||||||
|
applySyncSuccess(status);
|
||||||
|
} else if (status.state === "failed") {
|
||||||
|
applySyncFailure(status);
|
||||||
|
}
|
||||||
|
/* idle → nothing to render */
|
||||||
|
}
|
||||||
|
|
||||||
|
if (syncBtn) syncBtn.addEventListener("click", startSync);
|
||||||
|
|
||||||
(async () => {
|
(async () => {
|
||||||
await initSharedHeader(); // phase 19: Sign in/out + Sources link in the shared bar
|
await initSharedHeader(); // phase 19: Sign in/out + Sources link in the shared bar
|
||||||
if (!(await isAdmin())) {
|
if (!(await isAdmin())) {
|
||||||
@@ -158,4 +388,5 @@ function showEmpty() {
|
|||||||
}
|
}
|
||||||
if (gateEl) gateEl.hidden = true;
|
if (gateEl) gateEl.hidden = true;
|
||||||
loadDocs();
|
loadDocs();
|
||||||
|
initSyncButton(); // phase 32: re-attach to a running / last sync run
|
||||||
})();
|
})();
|
||||||
|
|||||||
@@ -313,6 +313,48 @@ html::after {
|
|||||||
.auth-link:disabled { opacity: 0.6; cursor: wait; }
|
.auth-link:disabled { opacity: 0.6; cursor: wait; }
|
||||||
.auth-link svg { width: 16px; height: 16px; display: none; }
|
.auth-link svg { width: 16px; height: 16px; display: none; }
|
||||||
|
|
||||||
|
/* Phase 32: the admin-only "Sync sources" pill (Sources header) — the
|
||||||
|
same ghost pill as New chat / the auth links, so the bar keeps one
|
||||||
|
visual language. ink-soft on surface ≈6.9:1 (WCAG AA); hover pair
|
||||||
|
brand-ink/brand-soft ≈6.9:1. The refresh icon is always visible (it
|
||||||
|
doubles as the running-state spinner); icon-only below 640px like
|
||||||
|
the other pills (aria-label keeps the accessible name). ≥44px touch
|
||||||
|
target at every width; :focus-visible via the global rule. */
|
||||||
|
.sync-btn {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 0.4rem;
|
||||||
|
min-height: 44px;
|
||||||
|
padding: 0.5rem 0.9rem;
|
||||||
|
border-radius: 999px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
background: transparent;
|
||||||
|
color: var(--ink-soft);
|
||||||
|
font: inherit;
|
||||||
|
font-weight: 600;
|
||||||
|
font-size: 0.95rem;
|
||||||
|
white-space: nowrap;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
.sync-btn:hover { background: var(--brand-soft); color: var(--brand-ink); }
|
||||||
|
.sync-btn:disabled { opacity: 0.6; cursor: wait; }
|
||||||
|
.sync-icon { width: 16px; height: 16px; display: block; flex: 0 0 auto; }
|
||||||
|
/* Running state: the refresh icon spins (reuses the shared spin
|
||||||
|
keyframes) — the visible half of "Syncing…" while the 2 s poll waits. */
|
||||||
|
.sync-btn .sync-icon.is-spinning { animation: spin 1s linear infinite; }
|
||||||
|
@media (prefers-reduced-motion: reduce) {
|
||||||
|
.sync-btn .sync-icon.is-spinning { animation: none; }
|
||||||
|
}
|
||||||
|
/* The aria-live last-result announcer ("2 added · 1 pruned") — soft ink
|
||||||
|
on the header surface (≈6.9:1), small mono to match the stat cards. */
|
||||||
|
.sync-result {
|
||||||
|
color: var(--ink-soft);
|
||||||
|
font-family: var(--mono);
|
||||||
|
font-size: 0.8rem;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
/* "Tuning" toggle (phase 15): ghost pill like New chat + a mono count
|
/* "Tuning" toggle (phase 15): ghost pill like New chat + a mono count
|
||||||
badge (brand-ink on brand-soft ≈6.9:1). The label is visually-hidden
|
badge (brand-ink on brand-soft ≈6.9:1). The label is visually-hidden
|
||||||
(not removed) below 640px so the accessible name keeps the word.
|
(not removed) below 640px so the accessible name keeps the word.
|
||||||
@@ -1598,6 +1640,23 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
|||||||
.auth-link { padding: 0.4rem 0.55rem; }
|
.auth-link { padding: 0.4rem 0.55rem; }
|
||||||
.auth-label { display: none; }
|
.auth-label { display: none; }
|
||||||
.auth-link svg { display: block; }
|
.auth-link svg { display: block; }
|
||||||
|
/* Phase 32: the sync pill goes icon-only like the other pills (the
|
||||||
|
aria-label keeps the accessible name); the spinning icon is the
|
||||||
|
visible running state on a touch screen. */
|
||||||
|
.sync-btn { padding: 0.4rem 0.55rem; }
|
||||||
|
.sync-label { display: none; }
|
||||||
|
/* The last-result counts stay ANNOUNCED (aria-live is untouched) but
|
||||||
|
go visually hidden — the 58px bar has no room for the text; the
|
||||||
|
icon carries the visible state. Same clip recipe as .steering-label. */
|
||||||
|
.sync-result {
|
||||||
|
position: absolute !important;
|
||||||
|
width: 1px; height: 1px;
|
||||||
|
margin: -1px; padding: 0;
|
||||||
|
overflow: hidden;
|
||||||
|
clip: rect(0 0 0 0);
|
||||||
|
white-space: nowrap;
|
||||||
|
border: 0;
|
||||||
|
}
|
||||||
.steering-toggle { padding: 0.4rem 0.55rem; }
|
.steering-toggle { padding: 0.4rem 0.55rem; }
|
||||||
/* Visually hidden, NOT display:none — the accessible name keeps the
|
/* Visually hidden, NOT display:none — the accessible name keeps the
|
||||||
word "Tuning" next to the count badge. */
|
word "Tuning" next to the count badge. */
|
||||||
|
|||||||
+27
-1
@@ -37,6 +37,20 @@
|
|||||||
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M12 5v14M5 12h14"/></svg>
|
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round"><path d="M12 5v14M5 12h14"/></svg>
|
||||||
<span class="new-chat-label">New chat</span>
|
<span class="new-chat-label">New chat</span>
|
||||||
</button>
|
</button>
|
||||||
|
<!-- Phase 32: the admin-only "Sync sources" button (TODO.md:5) —
|
||||||
|
SHIPS hidden (anonymous-safe), header.js reveals it for the
|
||||||
|
admin on the SAME cached whoami that reveals #nav-sources /
|
||||||
|
#nav-tuning (one fetch, no extra whoami call). sources.js
|
||||||
|
drives the §7.4 "never stale" lifecycle: idle → "Syncing…"
|
||||||
|
(disabled + spinning icon + 2 s GET /api/sync/status poll) →
|
||||||
|
last result ("Synced HH:MM" + counts in #sync-result) or the
|
||||||
|
role="alert" error banner. #sync-result is the aria-live
|
||||||
|
announcer for the last result. -->
|
||||||
|
<button type="button" class="sync-btn" id="sync-btn" hidden aria-label="Sync sources">
|
||||||
|
<svg class="sync-icon" aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12a9 9 0 1 1-9-9c2.52 0 4.93 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/></svg>
|
||||||
|
<span class="sync-label" id="sync-label">Sync sources</span>
|
||||||
|
</button>
|
||||||
|
<span class="sync-result" id="sync-result" role="status" aria-live="polite"></span>
|
||||||
<a href="/login.html?next=/sources.html" class="auth-link" id="sign-in-link" hidden>
|
<a href="/login.html?next=/sources.html" class="auth-link" id="sign-in-link" hidden>
|
||||||
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M10 4h8a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2h-8"/><path d="M4 12h11"/><path d="m12 9 3 3-3 3"/></svg>
|
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round"><path d="M10 4h8a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2h-8"/><path d="M4 12h11"/><path d="m12 9 3 3-3 3"/></svg>
|
||||||
<span class="auth-label">Sign in</span>
|
<span class="auth-label">Sign in</span>
|
||||||
@@ -50,11 +64,23 @@
|
|||||||
|
|
||||||
<main id="main" class="app-main" tabindex="-1">
|
<main id="main" class="app-main" tabindex="-1">
|
||||||
<div class="container sources-shell">
|
<div class="container sources-shell">
|
||||||
|
<!-- Phase 32: the sync failure banner — the chat error-banner
|
||||||
|
markup style (kb-banner + is-error), role="alert" so a failed
|
||||||
|
sync is announced. sources.js fills #sync-error-text and
|
||||||
|
un-hides it on a failed run (the button re-enables,
|
||||||
|
retry-ready); a new sync hides it again. -->
|
||||||
|
<div class="kb-banner is-error" id="sync-error-banner" role="alert" hidden>
|
||||||
|
<svg aria-hidden="true" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 3.6 22.2 20.4H1.8Z"/><path d="M12 9.5v4.6"/><path d="M12 17.4h.01"/></svg>
|
||||||
|
<span id="sync-error-text"></span>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div class="page-head">
|
<div class="page-head">
|
||||||
<h1>Knowledge base</h1>
|
<h1>Knowledge base</h1>
|
||||||
<p class="page-sub">
|
<p class="page-sub">
|
||||||
Every <code>*.md</code> file indexed from <code>~/Homelab</code> and
|
Every <code>*.md</code> file indexed from <code>~/Homelab</code> and
|
||||||
<code>~/Deployments</code>. Re-run the import to refresh.
|
<code>~/Deployments</code>. Re-run the import to refresh — or hit
|
||||||
|
<strong>Sync sources</strong> in the header to clone the repos and
|
||||||
|
re-import.
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,283 @@
|
|||||||
|
"""Phase 32 E2E (Playwright): the admin-only "Sync sources" button.
|
||||||
|
|
||||||
|
Story: ``.agent/user_stories/admin-sync-button.md``
|
||||||
|
Run in isolation (DB must be up: ``podman compose up -d db``; git on
|
||||||
|
PATH — a documented environment prerequisite, phase 28):
|
||||||
|
|
||||||
|
uv run pytest tests/e2e/test_sync_button.py -v --no-cov
|
||||||
|
|
||||||
|
The story gate runs the **real** sync path end to end — a real
|
||||||
|
``git clone`` of a local ``file://`` fixture repo (deterministic, no
|
||||||
|
network), a real import against the mock LLM, a real KB-overview
|
||||||
|
regeneration — plus the admin-only visibility and the full button
|
||||||
|
lifecycle (§7.4: "Syncing…" → "Synced HH:MM" + counts, the idempotent
|
||||||
|
second run, the 409 double trigger).
|
||||||
|
|
||||||
|
Per-module app env (the E2E conftest pattern, module-scoped): this
|
||||||
|
story's app boots with ``BOR_GIT_SOURCES=file://<fixture repo>`` and
|
||||||
|
its own ``BOR_SOURCES_DIR`` — the session app (no git sources) is
|
||||||
|
never started in this isolated run, so no port clash.
|
||||||
|
|
||||||
|
Test → story mapping (Playwright Mapping Rule):
|
||||||
|
1. ``test_anonymous_sees_no_button`` → AC 1 + 3 (hidden button, 403s)
|
||||||
|
2. ``test_admin_sync_lifecycle`` → AC 2 + 4 (full lifecycle + idempotent
|
||||||
|
second run + the fresh ``kb_overview`` row)
|
||||||
|
3. ``test_double_trigger_409`` → AC 2 (one sync at a time)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from playwright.sync_api import Page, expect
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
from app.db import SessionLocal
|
||||||
|
from app.models import KbOverview
|
||||||
|
from e2e.auth_helpers import login
|
||||||
|
from e2e.conftest import (
|
||||||
|
ADMIN_PASSWORD,
|
||||||
|
APP_PORT,
|
||||||
|
SESSION_SECRET,
|
||||||
|
USE_REAL_LLM,
|
||||||
|
_wait_http,
|
||||||
|
)
|
||||||
|
|
||||||
|
REPO = Path(__file__).resolve().parents[2]
|
||||||
|
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||||||
|
|
||||||
|
#: The unique sentinel inside the fixture doc (task 03 step 1) — its
|
||||||
|
#: import into the Sources table proves the REAL clone was indexed.
|
||||||
|
SENTINEL = "RESE-SYNC-SENTINEL-9b2c"
|
||||||
|
FIXTURE_DOC = "notes/sync-fixture.md"
|
||||||
|
|
||||||
|
#: "Synced HH:MM" — the local-time last-result label (sources.js's
|
||||||
|
#: fmtSyncTime), any hour/minute.
|
||||||
|
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")
|
||||||
|
|
||||||
|
#: Real git clone + embed against the mock LLM — generous budget
|
||||||
|
#: (task 03: the sync can legitimately take a while, no client timeout).
|
||||||
|
SYNC_TIMEOUT_MS = 60_000
|
||||||
|
|
||||||
|
|
||||||
|
def _git(cwd: Path, *args: str) -> None:
|
||||||
|
"""Run git in *cwd*; a non-zero exit fails the fixture loudly."""
|
||||||
|
proc = subprocess.run(["git", *args], cwd=cwd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise AssertionError(f"git {' '.join(args)} failed: {proc.stderr.strip()}")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def sync_git_repo(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||||
|
"""A real one-commit git repo the sync must clone (task 03 step 1).
|
||||||
|
|
||||||
|
``tmp_path`` is function-scoped while the module-scoped app fixture
|
||||||
|
needs the repo for the module's lifetime, so it is built under
|
||||||
|
``tmp_path_factory`` (the same pytest-managed temp area, module-safe)
|
||||||
|
via real ``git`` subprocess calls.
|
||||||
|
"""
|
||||||
|
root = tmp_path_factory.mktemp("sync_git")
|
||||||
|
repo = root / "homelab-notes"
|
||||||
|
(repo / "notes").mkdir(parents=True)
|
||||||
|
(repo / "notes" / "sync-fixture.md").write_text(
|
||||||
|
"# Sync fixture note\n"
|
||||||
|
"\n"
|
||||||
|
"One small note that exists only to prove the admin sync button\n"
|
||||||
|
"end to end: a real git clone, a real import, a real KB overview\n"
|
||||||
|
"regeneration.\n"
|
||||||
|
"\n"
|
||||||
|
f"Marker: {SENTINEL}\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
_git(repo, "init", "-q")
|
||||||
|
_git(repo, "add", "-A")
|
||||||
|
_git(
|
||||||
|
repo,
|
||||||
|
"-c", "user.email=e@x", "-c", "user.name=t",
|
||||||
|
"-c", "commit.gpgsign=false", # the fixture commit never signs
|
||||||
|
"commit", "-qm", "one",
|
||||||
|
)
|
||||||
|
assert (repo / ".git").is_dir()
|
||||||
|
return repo
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def app_server(mock_llm: int, sync_git_repo: Path) -> Iterator[str]:
|
||||||
|
"""The real app under test — per-module env: the sync's subject is a
|
||||||
|
real ``file://`` git source with its own checkout dir (the conftest
|
||||||
|
session app boots without ``BOR_GIT_SOURCES`` and is never started
|
||||||
|
in this isolated run)."""
|
||||||
|
env = dict(os.environ)
|
||||||
|
env.pop("DEBUGPY", None)
|
||||||
|
env["BOR_ENVIRONMENT"] = "e2e"
|
||||||
|
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||||
|
env["BOR_LLM_BASE_URL"] = (
|
||||||
|
"https://aipi.reeseapps.com/v1"
|
||||||
|
if USE_REAL_LLM
|
||||||
|
else f"http://127.0.0.1:{mock_llm}/v1"
|
||||||
|
)
|
||||||
|
# Mock-calibrated threshold (conftest pattern) — keeps every story
|
||||||
|
# suite's deterministic gate behavior; production default stays 0.62.
|
||||||
|
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||||
|
env.setdefault(
|
||||||
|
"BOR_DATABASE_URL",
|
||||||
|
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||||
|
)
|
||||||
|
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||||||
|
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||||
|
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||||
|
# Phase 32: the sync button's subject — one real local repo.
|
||||||
|
env["BOR_GIT_SOURCES"] = f"file://{sync_git_repo}"
|
||||||
|
env["BOR_SOURCES_DIR"] = str(sync_git_repo.parent / "checkouts")
|
||||||
|
proc = subprocess.Popen(
|
||||||
|
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||||||
|
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
||||||
|
cwd=REPO,
|
||||||
|
env=env,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
_wait_http(f"{APP_URL}/api/health")
|
||||||
|
yield APP_URL
|
||||||
|
finally:
|
||||||
|
proc.terminate()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=10)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
proc.kill()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def app_url(app_server: str) -> str:
|
||||||
|
return app_server
|
||||||
|
|
||||||
|
|
||||||
|
def _truncate_kb() -> None:
|
||||||
|
"""Fresh KB per test (the E2E isolation pattern): the sync's counts
|
||||||
|
and the ``kb_overview`` row must be the sync's own doing."""
|
||||||
|
with SessionLocal() as db:
|
||||||
|
db.execute(text("TRUNCATE chunks, documents, query_log, kb_overview"))
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _clean_kb(db_ready: None) -> Iterator[None]:
|
||||||
|
_truncate_kb()
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def _overview_row() -> KbOverview | None:
|
||||||
|
with SessionLocal() as db:
|
||||||
|
return db.get(KbOverview, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def _wait_sync_done(page: Page, app_url: str, timeout_s: float = 60.0) -> dict[str, Any]:
|
||||||
|
"""Poll the (cookie-authenticated) status endpoint until the run
|
||||||
|
reaches a terminal state — exactly what the UI's 2 s poll loop
|
||||||
|
observes."""
|
||||||
|
deadline = time.monotonic() + timeout_s
|
||||||
|
body: dict[str, Any] = {}
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
r = page.request.get(f"{app_url}/api/sync/status")
|
||||||
|
assert r.status == 200
|
||||||
|
body = r.json()
|
||||||
|
if body["state"] in ("success", "failed"):
|
||||||
|
return body
|
||||||
|
time.sleep(0.5)
|
||||||
|
raise AssertionError(f"sync did not reach a terminal state: {body}")
|
||||||
|
|
||||||
|
|
||||||
|
# --- 1. Anonymous: no button, locked endpoints ----------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_anonymous_sees_no_button(page: Page, app_url: str, db_ready: None) -> None:
|
||||||
|
"""AC 1 + 3 — anonymous: ``#sync-btn`` never leaves ``hidden`` (the
|
||||||
|
whoami reveal is admin-only, so for a logged-out visitor it is
|
||||||
|
absent from the *revealed* DOM) and both sync endpoints answer
|
||||||
|
403 — the admin-only surface (A10 extended, phase 16 pattern)."""
|
||||||
|
page.goto(f"{app_url}/sources.html")
|
||||||
|
expect(page.locator("#sync-btn")).to_be_hidden()
|
||||||
|
# The same whoami gate that hides the button also hides the admin
|
||||||
|
# nav link and shows the catalog sign-in gate.
|
||||||
|
expect(page.locator("#nav-sources")).to_be_hidden()
|
||||||
|
expect(page.locator("#sources-gate")).to_be_visible()
|
||||||
|
assert page.request.get(f"{app_url}/api/sync/status").status == 403
|
||||||
|
assert page.request.post(f"{app_url}/api/sync").status == 403
|
||||||
|
|
||||||
|
|
||||||
|
# --- 2. Admin: the full lifecycle against the real sync path --------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_admin_sync_lifecycle(page: Page, app_url: str, db_ready: None) -> None:
|
||||||
|
"""AC 2 + 4 — click → "Syncing…" (disabled) → "Synced HH:MM" + the
|
||||||
|
counts, against the REAL pipeline (git clone of the ``file://``
|
||||||
|
fixture → import with prune → KB overview regeneration, mock LLM).
|
||||||
|
Then the idempotent second run: fast-forward pull + sha256 hash
|
||||||
|
skip → "0 added · 1 unchanged"."""
|
||||||
|
login(page, app_url) # lands on /sources.html (the button's home)
|
||||||
|
|
||||||
|
btn = page.locator("#sync-btn")
|
||||||
|
expect(btn).to_be_visible()
|
||||||
|
expect(page.locator("#sync-label")).to_have_text("Sync sources")
|
||||||
|
|
||||||
|
# --- run 1: clone + import + overview --------------------------------
|
||||||
|
btn.click()
|
||||||
|
expect(btn).to_be_disabled()
|
||||||
|
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||||
|
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||||
|
expect(btn).to_be_enabled() # never stale — re-enabled at the terminal state
|
||||||
|
# The fixture doc (one A9-format file) is the only change.
|
||||||
|
expect(page.locator("#sync-result")).to_have_text("1 added")
|
||||||
|
|
||||||
|
# The REAL clone was imported: the fixture path is in the Sources
|
||||||
|
# table (the sentinel lives inside it).
|
||||||
|
expect(page.locator("#docs-tbody tr", has_text=FIXTURE_DOC)).to_have_count(1)
|
||||||
|
|
||||||
|
# Phase-31 regeneration ran (the import changed the KB): the single
|
||||||
|
# kb_overview row is fresh and non-empty (DB check — truncated
|
||||||
|
# before this test, so it is the sync's own doing).
|
||||||
|
row = _overview_row()
|
||||||
|
assert row is not None and row.content.strip(), (
|
||||||
|
"kb_overview must be regenerated by a successful, KB-changing sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- run 2: idempotent pull + hash skip -------------------------------
|
||||||
|
btn.click()
|
||||||
|
expect(btn).to_be_disabled()
|
||||||
|
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||||
|
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||||
|
expect(btn).to_be_enabled()
|
||||||
|
# Nothing re-embedded (sha256 delta) — the no-op run announces the
|
||||||
|
# unchanged count instead of an empty live region.
|
||||||
|
expect(page.locator("#sync-result")).to_have_text("0 added · 1 unchanged")
|
||||||
|
# The doc survived the prune re-import (its file is still in the repo).
|
||||||
|
expect(page.locator("#docs-tbody tr", has_text=FIXTURE_DOC)).to_have_count(1)
|
||||||
|
|
||||||
|
|
||||||
|
# --- 3. Concurrency: one sync at a time ------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_double_trigger_409(page: Page, app_url: str, db_ready: None) -> None:
|
||||||
|
"""AC 2 — a second trigger while a run is in flight gets 409
|
||||||
|
("a sync is already running"); the UI adopts the in-flight run
|
||||||
|
instead of starting a second one. The E2E path adds that the REAL
|
||||||
|
run behind the 409 still completes (the adopted run is the one and
|
||||||
|
only run)."""
|
||||||
|
login(page, app_url)
|
||||||
|
|
||||||
|
first = page.request.post(f"{app_url}/api/sync")
|
||||||
|
assert first.status == 202, first.text
|
||||||
|
second = page.request.post(f"{app_url}/api/sync")
|
||||||
|
assert second.status == 409, second.text
|
||||||
|
assert second.json()["detail"] == "a sync is already running"
|
||||||
|
|
||||||
|
# The adopted (single) run still completes successfully.
|
||||||
|
body = _wait_sync_done(page, app_url)
|
||||||
|
assert body["state"] == "success", body
|
||||||
|
assert body["detail"]["added"] == 1 # the fixture doc, fresh after the truncate
|
||||||
@@ -0,0 +1,379 @@
|
|||||||
|
"""Integration: the admin sources-sync API (phase 32, task 01).
|
||||||
|
|
||||||
|
Covers the in-process sync runner end to end over HTTP: anonymous 403s
|
||||||
|
on both endpoints; admin idle → 202 → ``success`` with the full
|
||||||
|
ImportSummary detail; 409 on a double trigger while a run is in flight;
|
||||||
|
``GitSyncError`` → ``failed`` with the failing repo named and **zero**
|
||||||
|
import attempts; empty ``BOR_GIT_SOURCES`` → ``failed`` loudly; an
|
||||||
|
embedding failure → ``failed`` with any credentials masked; the import
|
||||||
|
always runs with ``prune=True``; and the phase-31 overview trigger is
|
||||||
|
change-gated (no ``lite`` call on an unchanged KB).
|
||||||
|
|
||||||
|
The git / import / overview layers are monkeypatched in ``app.api.sync``
|
||||||
|
(same fake style as ``test_import_docs_git.py``) — no real git, no real
|
||||||
|
DB, no LLM: the runner's state machine and HTTP surface are under test.
|
||||||
|
|
||||||
|
The admin client is used **as a context manager** on purpose: the
|
||||||
|
background sync task lives on the app's event loop, so the loop must
|
||||||
|
survive across requests — exactly how the app runs under uvicorn.
|
||||||
|
(A TestClient without the context manager starts a fresh loop per
|
||||||
|
request and would cancel the task on request exit.)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import time
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api import sync as sync_api
|
||||||
|
from app.config import Settings
|
||||||
|
from app.main import app as fastapi_app
|
||||||
|
from app.rag.importer import ImportSummary
|
||||||
|
from app.rag.llm import EmbeddingError, LLMClient
|
||||||
|
from scripts.git_sync import GitSyncError
|
||||||
|
from tests.conftest import ADMIN_PASSWORD
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _fresh_sync_state() -> Iterator[None]:
|
||||||
|
"""The module-level status object + task are process-global: reset them
|
||||||
|
around every test (both before — a previous test's terminal state
|
||||||
|
would leak into the idle assertion — and after)."""
|
||||||
|
sync_api._status = sync_api.SyncStatus()
|
||||||
|
sync_api._task = None
|
||||||
|
yield
|
||||||
|
sync_api._status = sync_api.SyncStatus()
|
||||||
|
sync_api._task = None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def sync_client() -> Iterator[TestClient]:
|
||||||
|
"""Context-managed TestClient — one app event loop across requests
|
||||||
|
(the background task must survive between the POST and the polls)."""
|
||||||
|
with TestClient(fastapi_app) as client:
|
||||||
|
yield client
|
||||||
|
|
||||||
|
|
||||||
|
def _settings(git_sources: str = "", sources_dir: str = "~/bor-sources") -> Settings:
|
||||||
|
"""Fresh settings (no .env file); explicit kwargs beat any env leaks."""
|
||||||
|
return Settings(_env_file=None, git_sources=git_sources, sources_dir=sources_dir) # pyright: ignore[reportCallIssue]
|
||||||
|
|
||||||
|
|
||||||
|
def _login(client: TestClient) -> None:
|
||||||
|
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||||
|
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
|
||||||
|
|
||||||
|
|
||||||
|
def _poll(client: TestClient, want: str, timeout: float = 5.0) -> dict:
|
||||||
|
"""Poll ``GET /api/sync/status`` until ``state == want`` (terminal).
|
||||||
|
|
||||||
|
Any state other than ``running`` before the deadline fails loudly —
|
||||||
|
an unexpected ``failed`` must never be masked by the wait.
|
||||||
|
"""
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
body = client.get("/api/sync/status").json()
|
||||||
|
if body["state"] == want:
|
||||||
|
return body
|
||||||
|
assert body["state"] == "running", (
|
||||||
|
f"unexpected state {body['state']!r} while waiting for {want!r}: {body}"
|
||||||
|
)
|
||||||
|
time.sleep(0.05)
|
||||||
|
raise AssertionError(f"sync did not reach {want!r} within {timeout}s")
|
||||||
|
|
||||||
|
|
||||||
|
class FakeImportSources:
|
||||||
|
"""Records every ``import_sources`` call; returns a canned summary."""
|
||||||
|
|
||||||
|
def __init__(self, summary: ImportSummary, delay: float = 0.0) -> None:
|
||||||
|
self.summary = summary
|
||||||
|
self.delay = delay
|
||||||
|
self.sources: list[list[Path]] = []
|
||||||
|
self.llms: list[LLMClient] = []
|
||||||
|
self.prune_flags: list[bool] = []
|
||||||
|
|
||||||
|
async def __call__(
|
||||||
|
self,
|
||||||
|
sources: list[Path],
|
||||||
|
llm: LLMClient,
|
||||||
|
*,
|
||||||
|
prune: bool = False,
|
||||||
|
limit: int | None = None,
|
||||||
|
session: Session | None = None,
|
||||||
|
) -> ImportSummary:
|
||||||
|
self.sources.append(list(sources))
|
||||||
|
self.llms.append(llm)
|
||||||
|
self.prune_flags.append(prune)
|
||||||
|
if self.delay:
|
||||||
|
await asyncio.sleep(self.delay)
|
||||||
|
return self.summary
|
||||||
|
|
||||||
|
|
||||||
|
class FakeOverview:
|
||||||
|
"""Records every ``regenerate_overview`` call; canned result."""
|
||||||
|
|
||||||
|
def __init__(self, ok: bool = True) -> None:
|
||||||
|
self.ok = ok
|
||||||
|
self.llms: list[LLMClient] = []
|
||||||
|
|
||||||
|
async def __call__(self, llm: LLMClient, session: Session | None = None) -> bool:
|
||||||
|
self.llms.append(llm)
|
||||||
|
return self.ok
|
||||||
|
|
||||||
|
|
||||||
|
def _fake_clone() -> tuple[list[tuple[str, Path]], object]:
|
||||||
|
"""A ``clone_or_pull`` that materialises a checkout with one .md file."""
|
||||||
|
calls: list[tuple[str, Path]] = []
|
||||||
|
|
||||||
|
def fake_clone_or_pull(url: str, dest: Path | str) -> Path:
|
||||||
|
dest = Path(dest)
|
||||||
|
dest.mkdir(parents=True, exist_ok=True)
|
||||||
|
(dest / "notes.md").write_text(f"# {dest.name}\ncontent for the KB\n", encoding="utf-8")
|
||||||
|
calls.append((url, dest))
|
||||||
|
return dest
|
||||||
|
|
||||||
|
return calls, fake_clone_or_pull
|
||||||
|
|
||||||
|
|
||||||
|
# --- anonymous -------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_anonymous_gets_403_on_both_endpoints(client: TestClient) -> None:
|
||||||
|
r = client.get("/api/sync/status")
|
||||||
|
assert r.status_code == 403
|
||||||
|
assert r.json() == {"detail": "admin only"}
|
||||||
|
r = client.post("/api/sync")
|
||||||
|
assert r.status_code == 403
|
||||||
|
assert r.json() == {"detail": "admin only"}
|
||||||
|
|
||||||
|
|
||||||
|
# --- admin: success --------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_admin_sync_success_reports_full_detail(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
repo_url = f"file://{tmp_path / 'repo.git'}"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
lambda: _settings(git_sources=repo_url, sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
clone_calls, fake_clone = _fake_clone()
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||||||
|
summary = ImportSummary(
|
||||||
|
files=5, added=1, updated=2, unchanged=2, pruned=3, errors=0,
|
||||||
|
chunks=11, embed_batches=4, summaries=1, summary_errors=0,
|
||||||
|
)
|
||||||
|
fake_import = FakeImportSources(summary)
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||||||
|
fake_overview = FakeOverview(ok=True)
|
||||||
|
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.get("/api/sync/status").json() == {
|
||||||
|
"state": "idle",
|
||||||
|
"started_at": None,
|
||||||
|
"finished_at": None,
|
||||||
|
"detail": {},
|
||||||
|
"error": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
r = sync_client.post("/api/sync")
|
||||||
|
assert r.status_code == 202
|
||||||
|
assert r.json() == {"detail": "sync started"}
|
||||||
|
|
||||||
|
body = _poll(sync_client, "success")
|
||||||
|
assert body["error"] is None
|
||||||
|
# ISO-8601 timestamps round-trip; finished after started.
|
||||||
|
started = datetime.fromisoformat(body["started_at"])
|
||||||
|
finished = datetime.fromisoformat(body["finished_at"])
|
||||||
|
assert finished >= started
|
||||||
|
# Every ImportSummary field + the overview flag, verbatim.
|
||||||
|
assert body["detail"] == {
|
||||||
|
"files": 5, "added": 1, "updated": 2, "unchanged": 2, "pruned": 3,
|
||||||
|
"errors": 0, "chunks": 11, "summaries": 1, "summary_errors": 0,
|
||||||
|
"overview": True,
|
||||||
|
}
|
||||||
|
# Git: the configured repo was cloned into BOR_SOURCES_DIR/<repo-name>/.
|
||||||
|
assert clone_calls == [(repo_url, tmp_path / "bor" / "repo")]
|
||||||
|
# Import: exactly the checkouts, with prune=True (the button is the
|
||||||
|
# canonical "mirror the repos" action) and a real LLMClient.
|
||||||
|
assert fake_import.sources == [[tmp_path / "bor" / "repo"]]
|
||||||
|
assert fake_import.prune_flags == [True]
|
||||||
|
assert len(fake_import.llms) == 1
|
||||||
|
assert isinstance(fake_import.llms[0], LLMClient)
|
||||||
|
# Overview: refreshed (added + updated > 0) with the same client.
|
||||||
|
assert fake_overview.llms == [fake_import.llms[0]]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unchanged_kb_skips_overview_refresh(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""Phase-31 trigger is change-gated: added + updated == 0 → no ``lite`` call."""
|
||||||
|
repo_url = f"file://{tmp_path / 'repo.git'}"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
lambda: _settings(git_sources=repo_url, sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
_, fake_clone = _fake_clone()
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||||||
|
summary = ImportSummary(files=7, added=0, updated=0, unchanged=7, pruned=0)
|
||||||
|
fake_import = FakeImportSources(summary)
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||||||
|
fake_overview = FakeOverview(ok=True)
|
||||||
|
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
body = _poll(sync_client, "success")
|
||||||
|
|
||||||
|
assert body["detail"]["overview"] is False
|
||||||
|
assert fake_overview.llms == [] # no wasted model call
|
||||||
|
assert len(fake_import.llms) == 1 # the import itself ran
|
||||||
|
|
||||||
|
|
||||||
|
# --- admin: concurrency ----------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_double_trigger_while_running_returns_409(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
repo_url = f"file://{tmp_path / 'repo.git'}"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
lambda: _settings(git_sources=repo_url, sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
_, fake_clone = _fake_clone()
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||||||
|
# The in-flight run takes a while (asyncio.sleep) so the second POST
|
||||||
|
# lands while it is still running.
|
||||||
|
fake_import = FakeImportSources(ImportSummary(files=1, added=1), delay=0.5)
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||||||
|
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
|
||||||
|
r = sync_client.post("/api/sync") # second trigger while running
|
||||||
|
assert r.status_code == 409
|
||||||
|
assert r.json() == {"detail": "a sync is already running"}
|
||||||
|
|
||||||
|
body = sync_client.get("/api/sync/status").json()
|
||||||
|
assert body["state"] == "running"
|
||||||
|
assert body["started_at"] is not None
|
||||||
|
assert body["finished_at"] is None
|
||||||
|
assert body["error"] is None
|
||||||
|
|
||||||
|
# The (single) run completes; the import ran exactly once.
|
||||||
|
_poll(sync_client, "success")
|
||||||
|
assert len(fake_import.sources) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# --- admin: failures -------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_git_failure_marks_failed_and_skips_import(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
repo_url = f"file://{tmp_path / 'bad.git'}"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
lambda: _settings(git_sources=repo_url, sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
|
||||||
|
def failing_clone(url: str, dest: Path | str) -> Path:
|
||||||
|
raise GitSyncError(
|
||||||
|
f"git clone --depth 1 {url} failed (exit 128): "
|
||||||
|
"fatal: repository not found"
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", failing_clone)
|
||||||
|
fake_import = FakeImportSources(ImportSummary())
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||||||
|
fake_overview = FakeOverview(ok=True)
|
||||||
|
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
|
||||||
|
body = _poll(sync_client, "failed")
|
||||||
|
assert "bad.git" in body["error"] # the failing repo is named
|
||||||
|
assert "fatal: repository not found" in body["error"]
|
||||||
|
assert body["detail"] == {}
|
||||||
|
assert body["finished_at"] is not None
|
||||||
|
assert fake_import.sources == [] # no partial import
|
||||||
|
assert fake_overview.llms == []
|
||||||
|
|
||||||
|
# A failed run leaves the system restartable: a new POST is accepted.
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
_poll(sync_client, "failed")
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_git_sources_configured_fails_loudly(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
# Whitespace-only is just as unconfigured as empty.
|
||||||
|
lambda: _settings(git_sources=" , ", sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
clone_calls, fake_clone = _fake_clone()
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||||||
|
fake_import = FakeImportSources(ImportSummary())
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
|
||||||
|
body = _poll(sync_client, "failed")
|
||||||
|
assert body["error"] == "no git sources configured (BOR_GIT_SOURCES)"
|
||||||
|
assert clone_calls == [] # git is never touched
|
||||||
|
assert fake_import.sources == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_import_error_is_reported_with_credentials_masked(
|
||||||
|
sync_client: TestClient, monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
repo_url = f"file://{tmp_path / 'repo.git'}"
|
||||||
|
monkeypatch.setattr(
|
||||||
|
sync_api,
|
||||||
|
"get_settings",
|
||||||
|
lambda: _settings(git_sources=repo_url, sources_dir=str(tmp_path / "bor")),
|
||||||
|
)
|
||||||
|
_, fake_clone = _fake_clone()
|
||||||
|
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||||||
|
|
||||||
|
async def failing_import(
|
||||||
|
sources: list[Path],
|
||||||
|
llm: LLMClient,
|
||||||
|
*,
|
||||||
|
prune: bool = False,
|
||||||
|
limit: int | None = None,
|
||||||
|
session: Session | None = None,
|
||||||
|
) -> ImportSummary:
|
||||||
|
raise EmbeddingError(
|
||||||
|
"embeddings request to https://user:secret@aipi.reeseapps.com/v1 "
|
||||||
|
"failed: connection refused"
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(sync_api, "import_sources", failing_import)
|
||||||
|
monkeypatch.setattr(sync_api, "regenerate_overview", FakeOverview(ok=True))
|
||||||
|
|
||||||
|
_login(sync_client)
|
||||||
|
assert sync_client.post("/api/sync").status_code == 202
|
||||||
|
|
||||||
|
body = _poll(sync_client, "failed")
|
||||||
|
assert "*****@aipi.reeseapps.com" in body["error"] # credentials masked
|
||||||
|
assert "user:secret" not in body["error"]
|
||||||
|
assert "connection refused" in body["error"] # the reason survives
|
||||||
@@ -0,0 +1,325 @@
|
|||||||
|
"""Unit: the admin "Sync sources" button contract (phase 32, task 02).
|
||||||
|
|
||||||
|
The browser behavior is E2E-covered (tests/e2e/test_sync_button.py,
|
||||||
|
task 03); here we pin the source-level wiring — the anonymous-safe
|
||||||
|
ship-hidden button markup, the header.js admin reveal on the SAME
|
||||||
|
cached whoami (no extra fetch), the sources.js sync state machine
|
||||||
|
(2 s poll, 202 start / 409 adoption / 403 hide, terminal labels,
|
||||||
|
the aria-live result, the single-poll-loop guard, no client-side hard
|
||||||
|
timeout), the §7.4 never-stale CSS (spin + reduced-motion opt-out,
|
||||||
|
disabled state, 44px floor, contrast pair) — so a silent regression is
|
||||||
|
caught without a browser.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||||||
|
ASSETS = FRONTEND / "assets"
|
||||||
|
|
||||||
|
HEADER_JS = ASSETS / "header.js"
|
||||||
|
SOURCES_JS = ASSETS / "sources.js"
|
||||||
|
STYLES_CSS = ASSETS / "styles.css"
|
||||||
|
SOURCES_HTML = FRONTEND / "sources.html"
|
||||||
|
|
||||||
|
|
||||||
|
def _text(path: Path) -> str:
|
||||||
|
assert path.is_file(), f"missing frontend file: {path}"
|
||||||
|
return path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _body(js: str, fn_name: str) -> str:
|
||||||
|
"""The source of the first top-level `function <fn_name>` in js."""
|
||||||
|
fn = js.find(f"function {fn_name}")
|
||||||
|
assert fn != -1, f"{fn_name} must be defined"
|
||||||
|
return js[fn : js.find("\n}", fn)]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- sources.html: anonymous-safe ship-hidden markup ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_button_ships_hidden_and_labeled() -> None:
|
||||||
|
"""#sync-btn SHIPS with the hidden attribute (anonymous-safe —
|
||||||
|
header.js reveals it for the admin), is a real <button type=
|
||||||
|
"button">, and carries aria-label="Sync sources" so the accessible
|
||||||
|
name stays stable across the label states."""
|
||||||
|
tag = re.search(r"<button[^>]*id=\"sync-btn\"[^>]*>", _text(SOURCES_HTML))
|
||||||
|
assert tag, "sources.html must carry the #sync-btn button"
|
||||||
|
attrs = tag.group(0)
|
||||||
|
assert 'class="sync-btn"' in attrs
|
||||||
|
assert 'type="button"' in attrs
|
||||||
|
assert re.search(r"\bhidden\b", attrs), "#sync-btn must ship hidden"
|
||||||
|
assert 'aria-label="Sync sources"' in attrs
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_button_has_icon_and_label_span() -> None:
|
||||||
|
"""The button body is a refresh-cycle svg (aria-hidden — decorative,
|
||||||
|
the spin is the visible running state) + the #sync-label span with
|
||||||
|
the idle text, so the label can be swapped by sources.js."""
|
||||||
|
text = _text(SOURCES_HTML)
|
||||||
|
btn = text[text.find('id="sync-btn"') : text.find("</button>", text.find('id="sync-btn"'))]
|
||||||
|
assert re.search(r'<svg[^>]*class="sync-icon"[^>]*aria-hidden="true"', btn)
|
||||||
|
assert 'id="sync-label"' in btn
|
||||||
|
assert re.search(r'<span[^>]*id="sync-label"[^>]*>Sync sources</span>', btn)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_result_is_the_aria_live_announcer() -> None:
|
||||||
|
"""#sync-result sits right after the button and is a polite live
|
||||||
|
region (role="status" + aria-live="polite") — the last-result /
|
||||||
|
counts announcement for screen readers."""
|
||||||
|
text = _text(SOURCES_HTML)
|
||||||
|
tag = re.search(r'<span[^>]*id="sync-result"[^>]*>', text)
|
||||||
|
assert tag, "sources.html must carry the #sync-result announcer"
|
||||||
|
attrs = tag.group(0)
|
||||||
|
assert 'role="status"' in attrs
|
||||||
|
assert 'aria-live="polite"' in attrs
|
||||||
|
assert text.find('id="sync-result"') > text.find("</button>", text.find('id="sync-btn"'))
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_error_banner_is_a_hidden_alert() -> None:
|
||||||
|
"""The failure banner uses the chat error-banner markup style
|
||||||
|
(kb-banner + is-error) and role="alert", shipping hidden —
|
||||||
|
sources.js un-hides it with the error text on a failed run."""
|
||||||
|
text = _text(SOURCES_HTML)
|
||||||
|
tag = re.search(r'<div[^>]*id="sync-error-banner"[^>]*>', text)
|
||||||
|
assert tag, "sources.html must carry the #sync-error-banner"
|
||||||
|
attrs = tag.group(0)
|
||||||
|
assert "kb-banner" in attrs and "is-error" in attrs
|
||||||
|
assert 'role="alert"' in attrs
|
||||||
|
assert re.search(r"\bhidden\b", attrs)
|
||||||
|
assert 'id="sync-error-text"' in text
|
||||||
|
# The banner lives in the page content, not the 64px header bar.
|
||||||
|
assert text.find('id="sync-error-banner"') > text.find('<main id="main"')
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_sub_copy_mentions_the_button() -> None:
|
||||||
|
"""The page-sub copy still points at the import CLI and now names
|
||||||
|
the header button as the one-click alternative (task 02 step 1)."""
|
||||||
|
sub = re.search(r'<p class="page-sub">(.*?)</p>', _text(SOURCES_HTML), re.DOTALL)
|
||||||
|
assert sub, "sources.html must keep the .page-sub copy"
|
||||||
|
assert "Re-run the import" in sub.group(1)
|
||||||
|
assert "Sync sources" in sub.group(1)
|
||||||
|
assert "header" in sub.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_page_stays_cdn_free() -> None:
|
||||||
|
"""No-CDN rule (PLAN §7.3, A11): the new button markup adds no
|
||||||
|
external references — same-origin assets only (the integration
|
||||||
|
test_index_html_served_locally re-checks this on the served page)."""
|
||||||
|
text = _text(SOURCES_HTML)
|
||||||
|
assert 'src="https://' not in text
|
||||||
|
assert 'href="https://' not in text
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- header.js: the admin reveal ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_header_reveals_sync_btn_on_the_admin_branch() -> None:
|
||||||
|
"""initSharedHeader reveals #sync-btn in the SAME admin branch as
|
||||||
|
#nav-sources (querySelector + hidden = !admin) — one cached whoami,
|
||||||
|
no extra whoami call; anonymous users never leave the hidden
|
||||||
|
default."""
|
||||||
|
js = _text(HEADER_JS)
|
||||||
|
fn = js.find("function initSharedHeader")
|
||||||
|
assert fn != -1
|
||||||
|
body = js[fn : js.find("\n}", fn)]
|
||||||
|
assert 'querySelector("#sync-btn")' in body, "#sync-btn must join the admin reveal"
|
||||||
|
assert "syncBtn.hidden = !admin" in body
|
||||||
|
# The reveal must not introduce a second whoami call site.
|
||||||
|
assert js.count('fetch("/api/whoami")') == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- sources.js: the sync state machine ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_calls_the_sync_api() -> None:
|
||||||
|
"""The click posts to POST /api/sync and the poll loop GETs
|
||||||
|
/api/sync/status — both through the same-origin API (A10)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
assert 'fetch("/api/sync", { method: "POST" })' in js
|
||||||
|
assert 'fetch("/api/sync/status")' in js
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_polls_every_2000ms() -> None:
|
||||||
|
"""The feedback loop is a 2000 ms poll of the status endpoint,
|
||||||
|
re-scheduled one tick at a time (setTimeout, not setInterval — an
|
||||||
|
in-flight fetch can never overlap the next tick)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
assert "SYNC_POLL_MS = 2000" in js
|
||||||
|
assert "setTimeout(tick, SYNC_POLL_MS)" in js
|
||||||
|
assert "setInterval" not in js
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_adopts_409_and_starts_on_202() -> None:
|
||||||
|
"""202 (started) and 409 (a run started elsewhere — e.g. a second
|
||||||
|
tab) both enter the running state and start polling: the UI never
|
||||||
|
starts a second run, it adopts the in-flight one."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
assert "r.status === 202 || r.status === 409" in js
|
||||||
|
idx = js.find("r.status === 202 || r.status === 409")
|
||||||
|
branch = js[idx : idx + 200]
|
||||||
|
assert "enterRunningState()" in branch
|
||||||
|
assert "startSyncPolling()" in branch
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_hides_the_button_on_403() -> None:
|
||||||
|
"""A 403 anywhere (POST or poll) is treated as not-admin: the
|
||||||
|
button hides — defense in depth behind header.js's whoami reveal."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
for occurrence in re.finditer(r"r\.status === 403", js):
|
||||||
|
window = js[occurrence.start() : occurrence.start() + 400]
|
||||||
|
assert "syncBtn.hidden = true" in window, "every 403 branch must hide the button"
|
||||||
|
assert len(re.findall(r"r\.status === 403", js)) >= 3, (
|
||||||
|
"POST, the status poll, and the load re-attach must all handle 403"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_running_state_is_never_stale() -> None:
|
||||||
|
"""Entering the running state disables the button, sets aria-busy,
|
||||||
|
spins the icon, and swaps the label to 'Syncing…' (the §7.4
|
||||||
|
feedback while the poll waits)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
fn = js.find("function enterRunningState")
|
||||||
|
assert fn != -1
|
||||||
|
body = js[fn : js.find("\n}", fn)]
|
||||||
|
assert "syncBtn.disabled = true" in body
|
||||||
|
assert 'syncBtn.setAttribute("aria-busy", "true")' in body
|
||||||
|
assert "syncIcon.classList.add(\"is-spinning\")" in body
|
||||||
|
assert '"Syncing…"' in body
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_terminal_states() -> None:
|
||||||
|
"""Terminal rendering: success → enabled + 'Synced HH:MM' (local
|
||||||
|
time of finished_at) + the last-result counts ('added' always
|
||||||
|
announced, zero terms omitted — a no-op re-sync reads '0 added ·
|
||||||
|
1 unchanged', never an empty live region) + a live catalog refresh
|
||||||
|
(the KB just changed — never a stale table); failed → enabled +
|
||||||
|
retry-ready 'Sync sources' label + the role='alert' banner with
|
||||||
|
the error; the result is cleared on a failure."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
|
||||||
|
success = _body(js, "applySyncSuccess")
|
||||||
|
assert '"Synced"' in success and "fmtSyncTime(status.finished_at)" in success
|
||||||
|
assert "fmtSyncResult(status.detail)" in success
|
||||||
|
# A successful sync just changed the KB: the catalog re-fetches live
|
||||||
|
# (table / stats / empty state never sit stale under "Synced").
|
||||||
|
assert "loadDocs()" in success
|
||||||
|
|
||||||
|
failure = _body(js, "applySyncFailure")
|
||||||
|
assert 'settleSyncButton("Sync sources")' in failure # retry-ready
|
||||||
|
assert "showSyncError(status.error)" in failure
|
||||||
|
|
||||||
|
result = _body(js, "fmtSyncResult")
|
||||||
|
# "added" is the always-announced headline term; "unchanged" covers
|
||||||
|
# the no-op case ("0 added · 1 unchanged"); updated/pruned are
|
||||||
|
# zero-omitted.
|
||||||
|
assert "added" in result and "unchanged" in result
|
||||||
|
assert " · " in result
|
||||||
|
assert "> 0" in result, "zero terms must be omitted"
|
||||||
|
|
||||||
|
time = _body(js, "fmtSyncTime")
|
||||||
|
assert "getHours()" in time and "getMinutes()" in time, "local HH:MM of finished_at"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_settles_the_button_on_terminal() -> None:
|
||||||
|
"""settleSyncButton re-enables the control, drops aria-busy, and
|
||||||
|
un-spins the icon — the button can never sit disabled after a run
|
||||||
|
reaches a terminal state (failed included: retry-ready)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
fn = js.find("function settleSyncButton")
|
||||||
|
assert fn != -1
|
||||||
|
body = js[fn : js.find("\n}", fn)]
|
||||||
|
assert "syncBtn.disabled = false" in body
|
||||||
|
assert 'syncBtn.removeAttribute("aria-busy")' in body
|
||||||
|
assert "syncIcon.classList.remove(\"is-spinning\")" in body
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_never_starts_a_second_poll_loop() -> None:
|
||||||
|
"""startSyncPolling is guarded by the module-level timer: a 409
|
||||||
|
adoption, a reload re-attach, or a stray call can never run two
|
||||||
|
poll loops at once (phase completion criterion)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
fn = js.find("function startSyncPolling")
|
||||||
|
assert fn != -1
|
||||||
|
head = js[fn : js.find("const tick", fn)]
|
||||||
|
assert re.search(r"if\s*\(\s*syncPollTimer\s*!==\s*null\s*\)\s*return", head), (
|
||||||
|
"the single-loop guard must be the first statement"
|
||||||
|
)
|
||||||
|
assert "clearTimeout(syncPollTimer)" in _body(js, "stopSyncPolling")
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_has_no_client_side_hard_timeout() -> None:
|
||||||
|
"""Phase locked decision: a sync can legitimately run for minutes,
|
||||||
|
so there is NO client-side hard timeout — the 2 s poll is the
|
||||||
|
feedback loop and the server state is authoritative (the 120 s
|
||||||
|
LLM-turn guard must not leak into the sync path)."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
assert "TURN_TIMEOUT" not in js
|
||||||
|
assert "120" not in js[js.find("Phase 32") :], (
|
||||||
|
"no turn-timeout constant in the sync section"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sources_js_reattaches_on_load() -> None:
|
||||||
|
"""initSyncButton (run from the IIFE on the admin path, after
|
||||||
|
initSharedHeader) fetches the status once and re-enters the running
|
||||||
|
state on 'running' (reload mid-sync) or renders the last result on
|
||||||
|
a terminal state; the click binding wires startSync to the button."""
|
||||||
|
js = _text(SOURCES_JS)
|
||||||
|
fn = js.find("function initSyncButton")
|
||||||
|
assert fn != -1
|
||||||
|
body = js[fn : js.find("\n}\n", fn)]
|
||||||
|
assert 'fetch("/api/sync/status")' in body
|
||||||
|
assert 'status.state === "running"' in body
|
||||||
|
assert 'status.state === "success"' in body
|
||||||
|
assert 'status.state === "failed"' in body
|
||||||
|
assert "syncBtn.addEventListener(\"click\", startSync)" in js
|
||||||
|
# The IIFE runs it on the admin path only (after the whoami gate).
|
||||||
|
iife = js[js.find("(async () => {") :]
|
||||||
|
admin_idx = iife.find("await isAdmin()")
|
||||||
|
init_idx = iife.find("initSyncButton();")
|
||||||
|
assert -1 < admin_idx < init_idx, "re-attach must run only for the admin"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- styles.css: the §7.4 states ----------
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_button_css_ghost_pill_and_disabled_state() -> None:
|
||||||
|
""".sync-btn is the same ghost pill as .new-chat-btn (contrast pair
|
||||||
|
ink-soft on surface ≈6.9:1 ≥ 4.5:1), with a ≥44px touch floor and a
|
||||||
|
:disabled state (never stale — the busy look is visible)."""
|
||||||
|
css = _text(STYLES_CSS)
|
||||||
|
block = re.search(r"\.sync-btn\s*\{([^}]*)\}", css)
|
||||||
|
assert block, "styles.css must define .sync-btn"
|
||||||
|
body = block.group(1)
|
||||||
|
assert "min-height: 44px" in body
|
||||||
|
assert "color: var(--ink-soft)" in body
|
||||||
|
assert "border-radius: 999px" in body
|
||||||
|
disabled = re.search(r"\.sync-btn:disabled\s*\{([^}]*)\}", css)
|
||||||
|
assert disabled, ".sync-btn:disabled must be styled"
|
||||||
|
assert "cursor: wait" in disabled.group(1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_icon_spins_and_respects_reduced_motion() -> None:
|
||||||
|
"""The running state spins the refresh icon on the shared spin
|
||||||
|
keyframes (1s linear infinite), and prefers-reduced-motion stills
|
||||||
|
it — the existing opt-out pattern."""
|
||||||
|
css = _text(STYLES_CSS)
|
||||||
|
spin = re.search(r"\.sync-btn \.sync-icon\.is-spinning\s*\{([^}]*)\}", css)
|
||||||
|
assert spin, "the .is-spinning state must be styled"
|
||||||
|
assert "animation: spin 1s linear infinite" in spin.group(1)
|
||||||
|
spinner = r"\.sync-btn \.sync-icon\.is-spinning\s*\{([^}]*)\}"
|
||||||
|
reduced = re.search(r"@media \(prefers-reduced-motion: reduce\)\s*\{\s*" + spinner, css)
|
||||||
|
assert reduced, "the spin must opt out under prefers-reduced-motion"
|
||||||
|
assert "animation: none" in reduced.group(1)
|
||||||
|
assert "@keyframes spin" in css, "the spin keyframes are shared (pre-existing)"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_result_is_styled() -> None:
|
||||||
|
"""#sync-result (the aria-live last-result line) is styled in the
|
||||||
|
theme tokens — soft ink, small mono, no wrap in the header bar."""
|
||||||
|
css = _text(STYLES_CSS)
|
||||||
|
block = re.search(r"\.sync-result\s*\{([^}]*)\}", css)
|
||||||
|
assert block, "styles.css must define .sync-result"
|
||||||
|
assert "var(--ink-soft)" in block.group(1)
|
||||||
Reference in New Issue
Block a user