phase: 90_upload_no_scan
All verification is complete and green. Final report:
**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**
- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors
**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)
**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.
**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
This commit is contained in:
@@ -4,77 +4,93 @@ Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_sync_upload_progress.py -v --no-cov
|
||||
|
||||
The story gate for the phase's executable proof (owner-locked A1–A5):
|
||||
both long-running KB jobs report **which file is being processed right
|
||||
now** — not just "Syncing…"/"Uploading…" — and the archive upload is
|
||||
fully **backgrounded**: ``POST /api/git-sources/upload`` answers 202 the
|
||||
moment the archive is on disk (the "Successfully uploaded — <file>"
|
||||
toast fires — the user may navigate away), the unpack/scan continues
|
||||
server-side behind ``GET /api/git-sources/upload/status`` (the phase-32
|
||||
``SyncStatus`` pattern), and the RAG-page sync button
|
||||
(``/sources.html``) animates with the upload's current file while that
|
||||
scan runs.
|
||||
The story gate for the phase's executable proof (owner-locked A1–A5),
|
||||
re-pointed by **phase 90** (the upload no longer scans): the
|
||||
long-running KB job that reports **which file is being processed
|
||||
right now** is the **sync** — and the archive upload is fully
|
||||
**backgrounded but unpack-only**: ``POST /api/git-sources/upload``
|
||||
answers 202 the moment the archive is on disk (the "Successfully
|
||||
uploaded — <file>" toast fires — the user may navigate away), the
|
||||
unpack → swap → row upsert continues server-side behind
|
||||
``GET /api/git-sources/upload/status`` (the phase-32 ``SyncStatus``
|
||||
pattern; the phase-64 key set with ``current_file``/``files_done``/
|
||||
``files_total`` null/0/0 for the whole run — phase 90 A2), and the
|
||||
upload's UI processing state is the BARE "Processing…" (no file, no
|
||||
"(n/m)") until the no-count "Uploaded <name> — press Sync sources to
|
||||
import it." result line lands (phase 90 A3). The scan — with its live
|
||||
file label — is the RAG page's **Sync sources** button's job, and the
|
||||
suite proves the new loop: upload → nothing indexed → **the sync that
|
||||
follows the upload shows its live file label and lands the counts**.
|
||||
|
||||
**Timing fixture (the phase's fixture note):** the mock LLM indexes
|
||||
fast — the in-progress state is real but brief (a 25-file scan against
|
||||
it takes ≈0.4 s, well under the UI's 2 s status poll). This module's
|
||||
app therefore boots behind ``tests/e2e/slow_llm.py`` — a delay-injecting
|
||||
reverse proxy in front of the mock LLM (``SLOW_DELAY_S`` per request →
|
||||
a 25-file scan is 28 LLM requests ≈ 4.2 s), so the scan outlives the
|
||||
2 s poll and the live-file label is asserted at BOTH layers the task
|
||||
pins:
|
||||
**Timing fixture (the phase's fixture note, phase-90 re-pointed):**
|
||||
the mock LLM indexes fast — an in-progress state is real but brief.
|
||||
The UPLOAD run no longer calls the LLM at all (phase 90 removed the
|
||||
model check + import), so it settles in milliseconds: the upload-side
|
||||
assertions lean on (a) the deterministic terminal status shape
|
||||
(null/0/0 progress, the ``{"message": "uploaded"}`` detail — every
|
||||
running tick the recorder catches is asserted bare) and (b) a held
|
||||
first status GET that widens the bare "Processing…" window past the
|
||||
UI's 2 s poll. The SYNC leg still needs ``tests/e2e/slow_llm.py`` —
|
||||
the delay-injecting reverse proxy in front of the mock LLM
|
||||
(``SLOW_DELAY_S`` per request → a 25-file sync is 28 LLM requests
|
||||
≈ 4.2 s) — so the sync outlives the 2 s poll and the live-file label
|
||||
is asserted at BOTH layers the task pins:
|
||||
|
||||
* **deterministic** — the status endpoints (``page.request`` / the
|
||||
concurrent recorder, ~100 ms cadence): ``state == "running"`` with a
|
||||
non-null ``current_file`` (``source/relative/path``) observed at some
|
||||
tick, the counts advancing, and the file-less ticks (unpack/row/
|
||||
model probe — A4) preceding the first file tick;
|
||||
* **UI** — polling the button labels for the ``Importing`` /
|
||||
``Syncing…`` / ``Processing…`` prefix plus a file path (generous
|
||||
timeout), which the pages' own 2 s poll ticks render.
|
||||
non-null ``current_file`` (``source/relative/path``) observed at
|
||||
some tick, the counts advancing, and the file-less ticks (model
|
||||
probe — A4) preceding the first file tick;
|
||||
* **UI** — polling the label for the ``Syncing…`` prefix plus a file
|
||||
path (generous timeout), which the page's own 2 s poll ticks render.
|
||||
|
||||
The upload archive is built in-test with Python's ``tarfile`` from
|
||||
**25 small ``.md`` files** (``e2e-prog.tar.gz`` → source ``e2e-prog``);
|
||||
the sync subject is a host temp dir (``sync-corpus/``, 25 small
|
||||
``.md`` files under ``notes/``) registered as a ``kind=local`` row —
|
||||
the ``test_sync_button.py`` / ``test_local_directory_sources.py``
|
||||
fixture styles. Per-module app env (the conftest pattern):
|
||||
**25 small ``.md`` files** (``e2e-prog.tar.gz`` → source
|
||||
``e2e-prog``); the sync subjects are the uploaded row itself (the new
|
||||
leg) and a host temp dir (``sync-corpus/``, 25 small ``.md`` files
|
||||
under ``notes/``) registered as a ``kind=local`` row — the
|
||||
``test_sync_button.py`` / ``test_local_directory_sources.py`` fixture
|
||||
styles. Per-module app env (the conftest pattern):
|
||||
``BOR_UPLOAD_DIR`` scratch, ``BOR_GIT_SOURCES`` forced empty (the sync
|
||||
sources are this suite's own local row), ``BOR_LLM_BASE_URL`` the slow
|
||||
proxy.
|
||||
sources are this suite's own local rows), ``BOR_LLM_BASE_URL`` the
|
||||
slow proxy.
|
||||
|
||||
Contract under test:
|
||||
|
||||
* **toast → navigate away (A2 + A3)**: on ``/git-sources.html`` the
|
||||
"Successfully uploaded — <source>" toast (``.toast.is-visible``,
|
||||
``role="status"``) appears while the scan is still running;
|
||||
navigating to ``/sources.html`` shows the sync button animating
|
||||
(spinner + ``aria-busy``) with the ``Importing <file>`` label; on
|
||||
completion the button settles to "Sync sources" (no error UI,
|
||||
``#sync-result`` stays empty — the upload's counts never render
|
||||
there, A3) and the catalog shows the uploaded documents (the
|
||||
phase-63 listing, untouched);
|
||||
* **upload progress (A4)**: during the scan the status endpoint
|
||||
reports a non-null ``current_file`` (``source/relative/path`` shape,
|
||||
full denominator, advancing counts) at running ticks, the upload
|
||||
button shows "Processing… <file>" (bare "Processing…" during unpack)
|
||||
before the result line lands, and the toast fired earlier in the run
|
||||
— the result line itself comes from the status ``success``;
|
||||
* **toast → navigate away → the sync does the scan (A1/A2 + phase
|
||||
90)**: on ``/git-sources.html`` the "Successfully uploaded —
|
||||
<source>" toast (``.toast.is-visible``, ``role="status"``) fires at
|
||||
the 202; navigating to ``/sources.html`` shows **zero indexed
|
||||
documents** (the upload unpacked + registered only) and the sync
|
||||
button settled idle with no error UI; clicking **Sync sources**
|
||||
then imports the uploaded row with the LIVE "Syncing… <file> (n/m)"
|
||||
label (both layers) and lands the counts ("N added", the catalog
|
||||
refreshes);
|
||||
* **upload processing (phase 90 A2)**: the upload button shows the
|
||||
BARE "Processing…" for the whole background run (no file, no
|
||||
"(n/m)", no title — proven across a held first status GET); every
|
||||
running tick the recorder catches carries a null ``current_file``
|
||||
and 0/0 counts; the terminal status is ``success`` with the no-count
|
||||
``{"message": "uploaded"}`` detail and null/0/0 progress; the
|
||||
result line points at the Sync button; the KB stays empty;
|
||||
* **sync live file (A4)**: a multi-file local source; clicking
|
||||
**Sync sources** on ``/sources.html`` shows "Syncing…" (bare, the
|
||||
pre-64 click state) then "Syncing… <file> (n/m)" (both layers), then
|
||||
the pre-64 success settle — "Synced HH:MM" + the counts result line —
|
||||
preserved, plus the file in the label;
|
||||
* **reload re-attach (A1 + A2)**: starting an upload and reloading
|
||||
``/git-sources.html`` mid-scan leaves the button in the Processing
|
||||
state (disabled) with no error banner and NO second upload (the
|
||||
status endpoint's single run is still the one from before the
|
||||
reload — pinned on its ``started_at``); it then settles with the
|
||||
result line and the list shows exactly one row for the archive.
|
||||
pre-64 click state) then "Syncing… <file> (n/m)" (both layers),
|
||||
then the pre-64 success settle — "Synced HH:MM" + the counts result
|
||||
line — preserved, plus the file in the label;
|
||||
* **reload re-attach (A1 + A2, phase 90)**: starting an upload and
|
||||
reloading ``/git-sources.html`` (the sub-second run may still be in
|
||||
flight — the boot re-attach enters the bare Processing state — or
|
||||
has settled — the boot re-renders the result line, the NAMELESS
|
||||
variant: the safe name was page-local) leaves the page with no
|
||||
error banner, no toast, and NO second upload (the status
|
||||
endpoint's single run is still the one from before the reload —
|
||||
pinned on its ``started_at``); the list shows exactly one row for
|
||||
the archive (in-place identity preserved).
|
||||
|
||||
Test → story mapping (Playwright Mapping Rule):
|
||||
1. ``test_upload_toast_then_navigate_away``
|
||||
2. ``test_upload_progress_shows_current_file``
|
||||
2. ``test_upload_progress_is_bare``
|
||||
3. ``test_sync_live_file_label``
|
||||
4. ``test_upload_reattach_after_reload``
|
||||
"""
|
||||
@@ -89,6 +105,7 @@ import tarfile
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -124,16 +141,16 @@ SOURCES_URL = "/sources.html"
|
||||
SLOW_PORT = int(os.environ.get("E2E_SLOW_LLM_PORT", "8902"))
|
||||
SLOW_URL = f"http://127.0.0.1:{SLOW_PORT}"
|
||||
|
||||
#: Per-LLM-request delay on the proxy — the scan's duration becomes
|
||||
#: deterministic: an N-file archive scan issues N + 3 LLM requests
|
||||
#: (the check_models embed + chat probe, one embed per file, the
|
||||
#: change-gated overview chat), so a 25-file upload takes ≈ 28 × 0.15 s
|
||||
#: Per-LLM-request delay on the proxy — the SYNC's duration becomes
|
||||
#: deterministic: an N-file sync issues N + 3 LLM requests (the
|
||||
#: check_models embed + chat probe, one embed per file, the
|
||||
#: change-gated overview chat), so a 25-file sync takes ≈ 28 × 0.15 s
|
||||
#: ≈ 4.2 s — long enough to outlive the UI's 2 s status poll (see the
|
||||
#: module docstring's timing-fixture note).
|
||||
#: module docstring's timing-fixture note). The upload run is
|
||||
#: unaffected — phase 90 removed its LLM calls.
|
||||
SLOW_DELAY_S = "0.15"
|
||||
|
||||
#: The uploaded archive: 25 small docs under ``docs/`` (the phase's
|
||||
#: fixture note — 20+ files so the scan outlives the 2 s poll).
|
||||
#: The uploaded archive: 25 small docs under ``docs/``.
|
||||
UPLOAD_NAME = "e2e-prog"
|
||||
UPLOAD_ARCHIVE = f"{UPLOAD_NAME}.tar.gz"
|
||||
N_FILES = 25
|
||||
@@ -155,7 +172,7 @@ SYNC_FILES: dict[str, str] = {
|
||||
#: fmtSyncTime), any hour/minute (test_sync_button.py's pattern).
|
||||
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")
|
||||
|
||||
#: Generous settle budget: a 25-file scan against the slowed LLM is
|
||||
#: Generous settle budget: a 25-file sync against the slowed LLM is
|
||||
#: ≈4.2 s; the UI's 2 s poll settles at most one tick after the
|
||||
#: terminal state lands.
|
||||
SETTLE_TIMEOUT_MS = 45_000
|
||||
@@ -181,9 +198,9 @@ def _build_targz(path: Path, files: dict[str, str]) -> Path:
|
||||
@pytest.fixture(scope="module")
|
||||
def slow_llm(mock_llm: int) -> Iterator[int]:
|
||||
"""The delay-injecting reverse proxy in front of the mock LLM
|
||||
(tests/e2e/slow_llm.py) — this suite's timing fixture: the live-file
|
||||
contract needs the scan to outlive the UI's 2 s poll (see
|
||||
``SLOW_DELAY_S``)."""
|
||||
(tests/e2e/slow_llm.py) — this suite's timing fixture for the SYNC
|
||||
legs: the live-file contract needs the sync to outlive the UI's
|
||||
2 s poll (see ``SLOW_DELAY_S``)."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["SLOW_LLM_DELAY_S"] = SLOW_DELAY_S
|
||||
@@ -239,10 +256,11 @@ def app_server(
|
||||
upload_dir: Path,
|
||||
tmp_path_factory: pytest.TempPathFactory,
|
||||
) -> Iterator[str]:
|
||||
"""The real app under test — per-module env: the LLM base URL is the
|
||||
SLOW PROXY in front of the mock (the timing fixture), uploads unpack
|
||||
into a scratch dir, and the env git list is forced empty (the sync
|
||||
sources are this suite's own ``kind=local`` row, seeded per test)."""
|
||||
"""The real app under test — per-module env: the LLM base URL is
|
||||
the SLOW PROXY in front of the mock (the sync-leg timing fixture;
|
||||
the upload run makes no LLM calls — phase 90), uploads unpack into
|
||||
a scratch dir, and the env git list is forced empty (the sync
|
||||
sources are this suite's own ``kind=local`` rows, seeded per test)."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
@@ -362,22 +380,24 @@ def _status(page: Page, app_url: str, path: str) -> dict[str, Any]:
|
||||
return r.json()
|
||||
|
||||
|
||||
def _wait_running_started_at(
|
||||
page: Page, app_url: str, path: str, timeout_s: float = 15.0
|
||||
) -> str:
|
||||
"""Poll the status endpoint until the run is ``running``; return its
|
||||
``started_at`` (the run's identity — a second run would reset it)."""
|
||||
deadline = time.monotonic() + timeout_s
|
||||
body: dict[str, Any] = {}
|
||||
while time.monotonic() < deadline:
|
||||
body = _status(page, app_url, path)
|
||||
if body["state"] == "running":
|
||||
assert body["started_at"] is not None
|
||||
return str(body["started_at"])
|
||||
if body["state"] in ("success", "failed"):
|
||||
raise AssertionError(f"the run settled too fast to observe: {body}")
|
||||
time.sleep(0.1)
|
||||
raise AssertionError(f"the run never entered running: {body}")
|
||||
def _hold_first_status_fetch(page: Page, hold_s: float) -> None:
|
||||
"""Intercept the upload-status GETs and hold ONLY THE FIRST one for
|
||||
``hold_s`` seconds (later fetches pass straight through). Install
|
||||
AFTER the page's boot re-attach fetch, before the submit. The
|
||||
poll's first tick fires 2 s after the 202; holding its fetch keeps
|
||||
the button in the in-run state long enough to assert the bare
|
||||
"Processing…" label (no file, no "(n/m)") across the whole
|
||||
background run — phase 90's run settles in milliseconds, so without
|
||||
the hold the in-run window is only the 2 s pre-tick gap."""
|
||||
state = {"held": False}
|
||||
|
||||
def handle(route: Any) -> None:
|
||||
if not state["held"]:
|
||||
state["held"] = True
|
||||
time.sleep(hold_s)
|
||||
route.continue_()
|
||||
|
||||
page.route("**/api/git-sources/upload/status", handle)
|
||||
|
||||
|
||||
class _TickRecorder:
|
||||
@@ -387,19 +407,45 @@ class _TickRecorder:
|
||||
endpoint with its OWN admin session (``httpx`` — the browser page
|
||||
drives itself in the meantime; the Playwright sync API is not
|
||||
thread-safe, so the thread never touches it), recording every tick
|
||||
from the first poll: the idle prelude, the running ticks (file-less
|
||||
unpack/row/probe phase, then the per-file ticks), and the terminal
|
||||
body. The thread only READS the same endpoint the UI's 2 s poll
|
||||
reads — it starts no jobs and cannot skew the run."""
|
||||
from the first poll: the idle prelude, the running ticks, and the
|
||||
terminal body. The thread only READS the same endpoint the UI's
|
||||
2 s poll reads — it starts no jobs and cannot skew the run.
|
||||
|
||||
def __init__(self, app_url: str, path: str) -> None:
|
||||
``require_running`` (phase 90): the unpack-only upload run settles
|
||||
in milliseconds, so a fast machine can miss every running tick —
|
||||
with the flag off, a terminal is accepted when the run it names
|
||||
started at or after this recorder's start (the stale-terminal
|
||||
guard: the run state lives in the app's memory, so a previous
|
||||
test's terminal must not be mistaken for this run's)."""
|
||||
|
||||
def __init__(self, app_url: str, path: str, require_running: bool = True) -> None:
|
||||
self._url = f"{app_url}{path}"
|
||||
self._login_url = f"{app_url}/api/login"
|
||||
self._require_running = require_running
|
||||
self._t0 = time.time()
|
||||
self._ticks: list[dict[str, Any]] = []
|
||||
self._terminal: dict[str, Any] | None = None
|
||||
self._stop = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
def _terminal_is_this_run(self, body: dict[str, Any], saw_running: bool) -> bool:
|
||||
"""See the class docstring — a terminal is accepted when the
|
||||
run it names is THIS recorder's: a running tick was observed,
|
||||
or (``require_running=False``) its ``started_at`` is no earlier
|
||||
than the recorder's start."""
|
||||
if saw_running:
|
||||
return True
|
||||
if self._require_running:
|
||||
return False
|
||||
started = body.get("started_at")
|
||||
if not started:
|
||||
return False
|
||||
try:
|
||||
started_at = datetime.fromisoformat(str(started)).timestamp()
|
||||
except ValueError:
|
||||
return False
|
||||
return started_at >= self._t0 - 2.0 # tolerance for the pre-submit gap
|
||||
|
||||
def start(self) -> None:
|
||||
def run() -> None:
|
||||
with httpx.Client(timeout=5.0) as client:
|
||||
@@ -413,15 +459,12 @@ class _TickRecorder:
|
||||
if r.status_code == 200:
|
||||
body = r.json()
|
||||
self._ticks.append(body)
|
||||
# The run status is in the app's memory and
|
||||
# SURVIVES across this module's tests: a
|
||||
# residual terminal state (a previous test's
|
||||
# run) must not be mistaken for this test's
|
||||
# own terminal — accept it only AFTER this
|
||||
# run's "running" has been observed.
|
||||
if body["state"] == "running":
|
||||
saw_running = True
|
||||
elif body["state"] in ("success", "failed") and saw_running:
|
||||
elif (
|
||||
body["state"] in ("success", "failed")
|
||||
and self._terminal_is_this_run(body, saw_running)
|
||||
):
|
||||
self._terminal = body
|
||||
return
|
||||
except Exception: # noqa: BLE001 — blip: retry next tick
|
||||
@@ -457,8 +500,8 @@ def _assert_live_file_ticks(
|
||||
) -> None:
|
||||
"""A4 against the recorded running ticks (the deterministic layer):
|
||||
|
||||
* the file-less ticks (unpack/row/model probe — before any file is
|
||||
indexed) come FIRST (the label is the bare prefix then);
|
||||
* the file-less ticks (model probe — before any file is indexed)
|
||||
come FIRST (the label is the bare prefix then);
|
||||
* SOME tick reports a non-null ``current_file`` in the
|
||||
``source/relative/path`` shape;
|
||||
* ``files_total`` is the full pre-walk count from the first file
|
||||
@@ -480,15 +523,16 @@ def _assert_live_file_ticks(
|
||||
# body instead of a recorded tick (100 ms cadence vs ≈160 ms file).
|
||||
assert max(dones) >= n_files - 1, f"files_done never advanced: {dones}"
|
||||
pre = [t for t in ticks if t["current_file"] is None]
|
||||
assert pre, f"no file-less running tick (the unpack phase): {ticks[:6]}"
|
||||
assert pre, f"no file-less running tick (the probe phase): {ticks[:6]}"
|
||||
assert ticks.index(pre[0]) < ticks.index(first_with), (
|
||||
"a file tick preceded the file-less unpack ticks"
|
||||
"a file tick preceded the file-less probe ticks"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Toast on 202 → navigate away → sync button animates with the
|
||||
# upload's current file → settle + catalog refresh (A2 + A3)
|
||||
# 1. Toast on 202 → navigate away → nothing indexed → the sync that
|
||||
# follows shows its live file label and lands the counts (A1/A2 +
|
||||
# the phase-90 leg)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -496,17 +540,18 @@ def test_upload_toast_then_navigate_away(
|
||||
page: Page, app_url: str, db_ready: None, upload_archive: Path
|
||||
) -> None:
|
||||
"""On ``/git-sources.html``: pick the multi-file archive, submit →
|
||||
the "Successfully uploaded — <source>" toast appears WHILE the scan
|
||||
is still running; immediately navigate to ``/sources.html`` → the
|
||||
sync button is present, animating (icon ``is-spinning``,
|
||||
``aria-busy``) with the ``Importing`` label; wait for the settle →
|
||||
button idle ("Sync sources"), no error UI, and the catalog table
|
||||
shows the uploaded documents (the phase-63 listing, untouched)."""
|
||||
the "Successfully uploaded — <source>" toast fires at the 202
|
||||
(A2); immediately navigate to ``/sources.html`` → **zero indexed
|
||||
documents** (phase 90 A1: the upload unpacked + registered only)
|
||||
and the sync button settled idle with no error UI; then click
|
||||
**Sync sources** (the new leg, phase 90) → the LIVE
|
||||
"Syncing… <file> (n/m)" label while the sync imports the uploaded
|
||||
row (both layers), then the success settle with the counts and the
|
||||
catalog refresh."""
|
||||
page.set_default_timeout(30_000)
|
||||
_admin_git_sources_page(page, app_url)
|
||||
# (A previous test's terminal run may re-render its result line at
|
||||
# boot — the task-05 re-attach contract; the submit below clears
|
||||
# it, and that is the "clean start" asserted after the click.)
|
||||
# boot — the re-attach contract; the submit below clears it.)
|
||||
|
||||
page.set_input_files("#archive-upload-file", str(upload_archive))
|
||||
page.click("#archive-upload-btn")
|
||||
@@ -518,38 +563,66 @@ def test_upload_toast_then_navigate_away(
|
||||
expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
|
||||
assert toast.get_attribute("role") == "status"
|
||||
expect(toast).to_have_text(f"Successfully uploaded — {UPLOAD_NAME}")
|
||||
# …and the scan is still running — the result line is not up yet
|
||||
# (the toast precedes the scan's completion, A2).
|
||||
expect(page.locator("#archive-upload-result")).to_be_hidden()
|
||||
started_at = _wait_running_started_at(page, app_url, "/api/git-sources/upload/status")
|
||||
assert started_at is not None
|
||||
|
||||
# Navigate away immediately (A1: the scan no longer dies with the
|
||||
# Navigate away immediately (A1: the run no longer dies with the
|
||||
# page).
|
||||
page.goto(app_url + SOURCES_URL)
|
||||
|
||||
# The RAG-page sync button re-attaches to the in-flight upload scan
|
||||
# (A3): present, animating (spinner + aria-busy), disabled, with
|
||||
# the live "Importing <file>" label — no error UI on this page
|
||||
# (the upload's failure UI lives on the Sources page, A3).
|
||||
# Phase 90 A1: the upload indexed NOTHING — the catalog is empty…
|
||||
expect(page.locator("#docs-tbody tr")).to_have_count(0)
|
||||
# …and the sync button settles idle with no error UI (the
|
||||
# sub-second upload run is over by the time this page's 2 s poll
|
||||
# first ticks; the upload's counts never render here — A3).
|
||||
btn = page.locator("#sync-btn")
|
||||
expect(btn).to_be_visible(timeout=30_000)
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
"Sync sources", timeout=SETTLE_TIMEOUT_MS
|
||||
)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).not_to_have_attribute("aria-busy")
|
||||
|
||||
# The new leg (phase 90): the Sync button does the scan the upload
|
||||
# deferred — live file label at both layers, counts + catalog on
|
||||
# the settle.
|
||||
recorder = _TickRecorder(app_url, "/api/sync/status")
|
||||
recorder.start()
|
||||
btn.click()
|
||||
|
||||
# The click's immediate state (A4 — the bare prefix until the
|
||||
# import's first file): disabled, aria-busy, spinning icon, no
|
||||
# error…
|
||||
expect(btn).to_be_disabled()
|
||||
expect(btn).to_have_attribute("aria-busy", "true")
|
||||
expect(btn.locator(".sync-icon")).to_have_class(re.compile(r"\bis-spinning\b"))
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
re.compile(rf"Importing {re.escape(UPLOAD_NAME)}/"), timeout=SETTLE_TIMEOUT_MS
|
||||
)
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
|
||||
# Settle: the button returns to idle, the sync-result line never
|
||||
# rendered the upload's counts (A3), and the catalog refreshes with
|
||||
# the uploaded documents — the phase-63 listing, untouched.
|
||||
expect(page.locator("#sync-label")).to_have_text("Sync sources", timeout=SETTLE_TIMEOUT_MS)
|
||||
# UI layer: the label gains the live file at the page's 2 s poll
|
||||
# tick ("Syncing… <source/relative/path> (n/m)").
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
re.compile(rf"Syncing… {re.escape(UPLOAD_NAME)}/.+\.md \(\d+/{N_FILES}\)"),
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
|
||||
# Deterministic layer: the recorder's full tick series — file-less
|
||||
# model-check ticks first, then the per-file ticks (full
|
||||
# denominator, advancing counts).
|
||||
terminal = recorder.stop()
|
||||
assert terminal["state"] == "success", terminal
|
||||
_assert_live_file_ticks(recorder.running_ticks, UPLOAD_NAME, N_FILES)
|
||||
assert terminal["current_file"] is None
|
||||
assert terminal["files_done"] == N_FILES
|
||||
assert terminal["files_total"] == N_FILES
|
||||
assert terminal["detail"]["added"] == N_FILES
|
||||
|
||||
# The success settle: "Synced HH:MM" + the counts result line, the
|
||||
# button re-enabled, no error — and the catalog lists the
|
||||
# imported docs (the upload's row, scanned by the sync).
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).not_to_have_attribute("aria-busy")
|
||||
expect(btn.locator(".sync-icon")).not_to_have_class(re.compile(r"\bis-spinning\b"))
|
||||
expect(page.locator("#sync-result")).to_have_text("")
|
||||
expect(page.locator("#sync-result")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#docs-tbody tr")).to_have_count(N_FILES, timeout=30_000)
|
||||
expect(page.locator("#docs-tbody tr", has_text="docs/00.md")).to_have_count(1)
|
||||
expect(page.locator("#docs-tbody tr", has_text="docs/24.md")).to_have_count(1)
|
||||
@@ -557,77 +630,90 @@ def test_upload_toast_then_navigate_away(
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Upload progress: live current file at BOTH layers; the toast fired
|
||||
# earlier than the result (A4)
|
||||
# 2. Upload processing: BARE "Processing…" for the whole run, the
|
||||
# no-count terminal shape, nothing indexed (phase 90 A2)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_upload_progress_shows_current_file(
|
||||
def test_upload_progress_is_bare(
|
||||
page: Page, app_url: str, db_ready: None, upload_archive: Path
|
||||
) -> None:
|
||||
"""During the scan: the status endpoint reports a non-null
|
||||
``current_file`` (``source/relative/path`` shape) at some running
|
||||
tick; the upload button label shows "Processing…" with a file path
|
||||
(UI layer) BEFORE the result line lands; the toast fired earlier in
|
||||
the run (not after the result). The result line + the settle come
|
||||
from the status ``success``."""
|
||||
"""Phase 90 A2: the upload's background run has NO file-level
|
||||
progress. The button shows the BARE "Processing…" for the whole
|
||||
run (no file, no "(n/m)", no title — proven across a held first
|
||||
status GET); every running tick the recorder catches carries a
|
||||
null ``current_file`` and 0/0 counts; the terminal status is
|
||||
``success`` with the no-count ``{"message": "uploaded"}`` detail
|
||||
and null/0/0 progress; the settled result line points at the Sync
|
||||
button; and the KB stays empty (no scan)."""
|
||||
page.set_default_timeout(30_000)
|
||||
_admin_git_sources_page(page, app_url)
|
||||
|
||||
recorder = _TickRecorder(app_url, "/api/git-sources/upload/status")
|
||||
# require_running=False — phase 90: the unpack-only run settles in
|
||||
# milliseconds, so a fast machine can miss every running tick; the
|
||||
# stale-terminal guard (started_at vs. the recorder's start) keeps
|
||||
# a previous test's terminal from being mistaken for this run's.
|
||||
recorder = _TickRecorder(
|
||||
app_url, "/api/git-sources/upload/status", require_running=False
|
||||
)
|
||||
recorder.start()
|
||||
# Hold the FIRST status GET (installed after the boot re-attach
|
||||
# fetch, before the submit): the bare in-run label gets a window
|
||||
# wider than the 2 s pre-tick gap.
|
||||
_hold_first_status_fetch(page, hold_s=4.5)
|
||||
page.set_input_files("#archive-upload-file", str(upload_archive))
|
||||
page.click("#archive-upload-btn")
|
||||
# A new attempt starts clean (the submit handler hides the result
|
||||
# line — any previous run's re-rendered line is gone by now).
|
||||
expect(page.locator("#archive-upload-result")).to_be_hidden()
|
||||
|
||||
# The toast fires at the 202 — earlier in the run, NOT after the
|
||||
# result (the result line is still down when the toast is up).
|
||||
# The toast fires at the 202 — NOT after the result (the result
|
||||
# line is still down when the toast is up).
|
||||
toast = page.locator(".toast")
|
||||
expect(toast).to_have_count(1, timeout=SETTLE_TIMEOUT_MS)
|
||||
expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
|
||||
expect(toast).to_have_text(f"Successfully uploaded — {UPLOAD_NAME}")
|
||||
expect(page.locator("#archive-upload-result")).to_be_hidden()
|
||||
|
||||
# UI layer: the button hands over to the scan — bare "Processing…"
|
||||
# at the 202 (A4: no file yet during the unpack phase)…
|
||||
# The button hands over to the run — the BARE "Processing…" (phase
|
||||
# 90 A2: no file, no counts, no title)…
|
||||
btn = page.locator("#archive-upload-btn")
|
||||
expect(btn).to_be_disabled()
|
||||
expect(btn).to_have_text("Processing…", timeout=5_000)
|
||||
# …then the live file label ("Processing… <file> (n/m)") at the
|
||||
# page's next 2 s poll tick, still before the result line lands.
|
||||
expect(btn).to_have_text(
|
||||
re.compile(rf"Processing… {re.escape(UPLOAD_NAME)}/.+\.md \(\d+/{N_FILES}\)"),
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
expect(page.locator("#archive-upload-result")).to_be_hidden()
|
||||
expect(btn).to_have_attribute("title", "")
|
||||
# …and it STAYS bare across the whole background run: the first
|
||||
# status GET is held, so the settling tick is in flight — no file
|
||||
# and no "(n/m)" can have rendered.
|
||||
time.sleep(2.5)
|
||||
expect(btn).to_have_text("Processing…")
|
||||
|
||||
# Deterministic layer: the recorder's full tick series (the same
|
||||
# endpoint the UI's 2 s poll reads) — file-less unpack ticks first,
|
||||
# then the per-file ticks with the full denominator.
|
||||
# Deterministic layer: every running tick the recorder caught is
|
||||
# bare (null file, 0/0 counts — phase 90 A2).
|
||||
for t in recorder.running_ticks:
|
||||
assert t["current_file"] is None, t
|
||||
assert t["files_done"] == 0 and t["files_total"] == 0, t
|
||||
terminal = recorder.stop()
|
||||
assert terminal["state"] == "success", terminal
|
||||
_assert_live_file_ticks(recorder.running_ticks, UPLOAD_NAME, N_FILES)
|
||||
# Phase 64: current_file is null in terminal states (the final
|
||||
# counts survive).
|
||||
assert terminal["detail"] == {"message": "uploaded"}, terminal
|
||||
assert terminal["current_file"] is None
|
||||
assert terminal["files_done"] == N_FILES
|
||||
assert terminal["files_total"] == N_FILES
|
||||
# The result line is rendered from the status success (the
|
||||
# UploadOut-shaped detail, counts unchanged in shape).
|
||||
assert terminal["detail"]["files"] == N_FILES
|
||||
assert terminal["detail"]["added"] == N_FILES
|
||||
assert terminal["detail"]["source"] == UPLOAD_NAME
|
||||
assert terminal["files_done"] == 0
|
||||
assert terminal["files_total"] == 0
|
||||
|
||||
# Settle: the result line lands, the button restores, the input
|
||||
# cleared, and the list has exactly one row for the archive.
|
||||
# Settle (the held GET released): the ready-for-sync result line,
|
||||
# the button restored ("Upload"), the input cleared, one row for
|
||||
# the archive — and the KB empty (no scan, phase 90 A1).
|
||||
result = page.locator("#archive-upload-result")
|
||||
expect(result).to_have_text(f"{N_FILES} added", timeout=30_000)
|
||||
expect(result).to_have_text(
|
||||
f"Uploaded {UPLOAD_NAME} — press Sync sources to import it.",
|
||||
timeout=30_000,
|
||||
)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).to_have_text("Upload & scan")
|
||||
expect(btn).to_have_text("Upload")
|
||||
expect(page.locator("#archive-upload-file")).to_have_value("")
|
||||
expect(page.locator("#git-sources-tbody tr", has_text=UPLOAD_NAME)).to_have_count(1)
|
||||
r = page.request.get(f"{app_url}/api/docs")
|
||||
assert r.status == 200, r.text
|
||||
assert r.json()["documents"] == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -698,20 +784,23 @@ def test_sync_live_file_label(
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Reload mid-scan → re-attach: no error, no second upload (A1 + A2)
|
||||
# 4. Reload → re-attach: no dead-end, no error, no toast, no second
|
||||
# upload (A1 + A2, phase 90)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_upload_reattach_after_reload(
|
||||
page: Page, app_url: str, db_ready: None, upload_archive: Path
|
||||
) -> None:
|
||||
"""Start the upload and, DURING the scan, reload
|
||||
``/git-sources.html`` → the button is in the Processing state
|
||||
(disabled) with no error banner and NO second upload (the status
|
||||
"""Start the upload and, while the sub-second run is in flight (or
|
||||
has just settled), reload ``/git-sources.html`` → the page never
|
||||
dead-ends: the boot re-attach either enters the bare Processing
|
||||
state + poll (run still in flight) or re-renders the settled
|
||||
result line (the NAMELESS variant — the safe name was page-local),
|
||||
with no error banner, no toast, and NO second upload (the status
|
||||
endpoint's single run is still the one from before the reload —
|
||||
pinned on its ``started_at``); it then settles with the result line
|
||||
and the list shows exactly one row for the archive (in-place
|
||||
identity preserved)."""
|
||||
pinned on its ``started_at``); the list shows exactly one row for
|
||||
the archive (in-place identity preserved)."""
|
||||
page.set_default_timeout(30_000)
|
||||
_admin_git_sources_page(page, app_url)
|
||||
|
||||
@@ -724,29 +813,28 @@ def test_upload_reattach_after_reload(
|
||||
expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
|
||||
# …and the run's identity: the status's started_at (a second run
|
||||
# would reset it — the single-run claim is pinned on it).
|
||||
started_at = _wait_running_started_at(page, app_url, "/api/git-sources/upload/status")
|
||||
started_at = _status(page, app_url, "/api/git-sources/upload/status")["started_at"]
|
||||
|
||||
# Reload mid-scan — the page must re-attach, not dead-end.
|
||||
# Reload — the run may be in flight (the boot re-attach enters the
|
||||
# bare Processing state + poll) or settled (the boot re-renders
|
||||
# the result line); either way the page must not dead-end.
|
||||
page.reload()
|
||||
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
|
||||
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
|
||||
expect(page.locator("#git-sources-content")).to_be_visible()
|
||||
|
||||
# The boot re-attach (task 05): the button is in the Processing
|
||||
# state (disabled, "Processing…") with no error banner and no
|
||||
# result line yet…
|
||||
btn = page.locator("#archive-upload-btn")
|
||||
expect(btn).to_be_disabled(timeout=15_000)
|
||||
expect(btn).to_have_text(re.compile(r"Processing…"), timeout=15_000)
|
||||
# No error banner, no toast at boot (A2 — the toast fired at the
|
||||
# 202, on the previous document life)…
|
||||
expect(page.locator("#archive-upload-error")).to_be_hidden()
|
||||
expect(page.locator("#archive-upload-result")).to_be_hidden()
|
||||
|
||||
# …and it settles with the result line from the status success…
|
||||
expect(page.locator(".toast")).to_have_count(0)
|
||||
# …and the settled result line: the nameless variant (the safe
|
||||
# name was page-local — lastUploadName is null after a reload).
|
||||
expect(page.locator("#archive-upload-result")).to_have_text(
|
||||
f"{N_FILES} added", timeout=SETTLE_TIMEOUT_MS
|
||||
"Uploaded — press Sync sources to import it.",
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).to_have_text("Upload & scan")
|
||||
expect(page.locator("#archive-upload-btn")).to_be_enabled()
|
||||
expect(page.locator("#archive-upload-btn")).to_have_text("Upload")
|
||||
# …and the list shows exactly one row for the archive.
|
||||
expect(page.locator("#git-sources-tbody tr")).to_have_count(1)
|
||||
row = page.locator("#git-sources-tbody tr", has_text=UPLOAD_NAME)
|
||||
@@ -757,7 +845,8 @@ def test_upload_reattach_after_reload(
|
||||
# started_at is the one from before the reload.
|
||||
terminal = _status(page, app_url, "/api/git-sources/upload/status")
|
||||
assert terminal["state"] == "success", terminal
|
||||
assert str(terminal["started_at"]) == started_at, (
|
||||
assert str(terminal["started_at"]) == str(started_at), (
|
||||
f"the run's identity changed (a second upload ran): {terminal['started_at']}"
|
||||
)
|
||||
assert terminal["current_file"] is None
|
||||
assert terminal["detail"] == {"message": "uploaded"}
|
||||
|
||||
Reference in New Issue
Block a user