brain-of-reese/tests/e2e/test_sync_upload_progress.py

"""Phase 64 story E2E (Playwright): real-time progress for sync + upload.

Run in isolation (DB must be up: ``podman compose up -d db``):

    uv run pytest tests/e2e/test_sync_upload_progress.py -v --no-cov

The story gate for the phase's executable proof (owner-locked A1–A5),
re-pointed by **phase 90** (the upload no longer scans): the
long-running KB job that reports **which file is being processed
right now** is the **sync** — and the archive upload is fully
**backgrounded but unpack-only**: ``POST /api/git-sources/upload``
answers 202 the moment the archive is on disk (the "Successfully
uploaded — <file>" toast fires — the user may navigate away), the
unpack → swap → row upsert continues server-side behind
``GET /api/git-sources/upload/status`` (the phase-32 ``SyncStatus``
pattern; the phase-64 key set with ``current_file``/``files_done``/
``files_total`` null/0/0 for the whole run — phase 90 A2), and the
upload's UI processing state is the BARE "Processing…" (no file, no
`N remaining` count) until the no-count "Uploaded <name> (<stamp>) — press Sync
sources to import it." result line lands (phase 90 A3 — the phase-154
completion stamp). The scan — with its live
file label — is the RAG page's **Sync sources** button's job, and the
suite proves the new loop: upload → nothing indexed → **the sync that
follows the upload shows its live file label and lands the counts**.

**Timing fixture (the phase's fixture note, phase-90 re-pointed):**
the mock LLM indexes fast — an in-progress state is real but brief.
The UPLOAD run no longer calls the LLM at all (phase 90 removed the
model check + import), so it settles in milliseconds: the upload-side
assertions lean on (a) the deterministic terminal status shape
(null/0/0 progress, the ``{"message": "uploaded"}`` detail — every
running tick the recorder catches is asserted bare) and (b) a held
first status GET that widens the bare "Processing…" window past the
UI's 2 s poll. The SYNC leg still needs ``tests/e2e/slow_llm.py`` —
the delay-injecting reverse proxy in front of the mock LLM
(``SLOW_DELAY_S`` per request → a 25-file sync is 28 LLM requests
≈ 4.2 s) — so the sync outlives the 2 s poll and the live-file label
is asserted at BOTH layers the task pins:

* **deterministic** — the status endpoints (``page.request`` / the
  concurrent recorder, ~100 ms cadence): ``state == "running"`` with a
  non-null ``current_file`` (``source/relative/path``) observed at
  some tick, the counts advancing, and the file-less ticks (model
  probe — A4) preceding the first file tick;
* **UI** — polling the detail box (``#sync-progress`` — phase 149:
  the running label rides the box under the button row, the button is
  the spinner) for the ``Syncing…`` prefix plus a file path (generous
  timeout), which the page's own 2 s poll ticks render.

Phase 193 (task 02) moves the running file-count part to the distance
to the end — ``N remaining`` with ``N = total - done`` (the owner's
"how many files are LEFT, not how many files out of the total"), the
upload job sharing the one builder and the shape. The box/title count
pins below therefore make the part optional: it is hidden on the FINAL
file, and this suite's 2 s poll can land exactly there — the pattern
still rejects the pre-phase ``(n/m)`` tail.

The upload archive is built in-test with Python's ``tarfile`` from
**25 small ``.md`` files** (``e2e-prog.tar.gz`` → source
``e2e-prog``); the sync subjects are the uploaded row itself (the new
leg) and a host temp dir (``sync-corpus/``, 25 small ``.md`` files
under ``notes/``) registered as a ``kind=local`` row — the
``test_sync_button.py`` / ``test_local_directory_sources.py`` fixture
styles. Per-module app env (the conftest pattern):
``BOR_UPLOAD_DIR`` scratch, ``BOR_GIT_SOURCES`` forced empty (the sync
sources are this suite's own local rows), ``BOR_LLM_BASE_URL`` the
slow proxy.

Contract under test:

* **toast → navigate away → the sync does the scan (A1/A2 + phase
  90)**: on ``/git-sources.html`` the "Successfully uploaded —
  <source>" toast (``.toast.is-visible``, ``role="status"``) fires at
  the 202; navigating to ``/sources.html`` shows **zero indexed
  documents** (the upload unpacked + registered only) and the sync
  button settled idle with no error UI; clicking **Sync sources**
  then imports the uploaded row with the LIVE "Syncing… <file>
  N remaining" label (phase 193 — the count part counts DOWN to the
  end; both layers — the phase-149 detail box carries the full
  string, the phase-129 pinned chunk part "· chunk n/m" rides the box
  + title while a file's embed batch is in flight) and lands the
  counts ("N added", the catalog refreshes);
* **upload processing (phase 90 A2)**: the upload button shows the
  BARE "Processing…" for the whole background run (no file, no
  `N remaining` count, no title — proven across a held first status
  GET); every
  running tick the recorder catches carries a null ``current_file``
  and 0/0 counts; the terminal status is ``success`` with the no-count
  ``{"message": "uploaded"}`` detail and null/0/0 progress; the
  result line points at the Sync button; the KB stays empty;
* **sync live file (A4)**: a multi-file local source; clicking
  **Sync sources** on ``/sources.html`` shows "Syncing…" (bare, the
  pre-64 click state) in the detail box (phase 149 — the button is
  the spinner) then "Syncing… <file> N remaining" (phase 193; both
  layers — the phase-
  129 pinned chunk part "· chunk n/m" rides the box + title while a
  file's embed batch is in flight), then the pre-64 success settle —
  "Synced HH:MM" + the counts result line — preserved, plus the file
  in the label (the box hides at settle);
* **reload re-attach (A1 + A2, phase 90)**: starting an upload and
  reloading ``/git-sources.html`` (the sub-second run may still be in
  flight — the boot re-attach enters the bare Processing state — or
  has settled — the boot re-renders the result line, the NAMELESS
  variant: the safe name was page-local) leaves the page with no
  error banner, no toast, and NO second upload (the status
  endpoint's single run is still the one from before the reload —
  pinned on its ``started_at``); the list shows exactly one row for
  the archive (in-place identity preserved).

Test → story mapping (Playwright Mapping Rule):
1. ``test_upload_toast_then_navigate_away``
2. ``test_upload_progress_is_bare``
3. ``test_sync_live_file_label``
4. ``test_upload_reattach_after_reload``
"""
from __future__ import annotations

import io
import os
import re
import subprocess
import sys
import tarfile
import threading
import time
from collections.abc import Iterator
from datetime import datetime
from pathlib import Path
from typing import Any

import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text

from app.db import SessionLocal
from app.models import GitSource
from e2e.auth_helpers import login
from e2e.conftest import (
    ADMIN_PASSWORD,
    MOCK_PORT,
    SESSION_SECRET,
    USE_REAL_LLM,
    _wait_http,
)
from e2e.sync_helpers import counts_line

REPO = Path(__file__).resolve().parents[2]

# Phase 79 (task 04, full inventory): the conftest session app owns its
# port in a combined run — this module app binds its own port instead
# (a same-port second uvicorn dies on bind and would drive the wrong
# server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_SYNCUP", "8133"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"

GIT_SOURCES_URL = "/git-sources.html"
SOURCES_URL = "/sources.html"

#: The slow-LLM proxy's port (the conftest's mock LLM stays on MOCK_PORT).
SLOW_PORT = int(os.environ.get("E2E_SLOW_LLM_PORT", "8902"))
SLOW_URL = f"http://127.0.0.1:{SLOW_PORT}"

#: Per-LLM-request delay on the proxy — the SYNC's duration becomes
#: deterministic: an N-file sync issues N + 3 LLM requests (the
#: check_models embed + chat probe, one embed per file, the
#: change-gated overview chat), so a 25-file sync takes ≈ 28 × 0.15 s
#: ≈ 4.2 s — long enough to outlive the UI's 2 s status poll (see the
#: module docstring's timing-fixture note). The upload run is
#: unaffected — phase 90 removed its LLM calls.
SLOW_DELAY_S = "0.15"

#: The uploaded archive: 25 small docs under ``docs/``.
UPLOAD_NAME = "e2e-prog"


def _ready_for_sync_line(name: str) -> re.Pattern[str]:
    """Phase 154 (task 02): the ready-for-sync line carrying the
    completion stamp — "Uploaded <name> (<stamp>) — press Sync sources
    to import it." (the fresh 202 settle, the page-local name). In
    E2E the stamp is ALWAYS the same-day ``HH:MM`` form — the run
    settles within seconds (the date-qualified forms are unit-pinned
    on a fixed clock — E2E never depends on the wall clock)."""
    return re.compile(
        rf"^Uploaded {re.escape(name)} \(\d{{1,2}}:\d{{2}}\)"
        r" — press Sync sources to import it\.$"
    )


def _nameless_ready_for_sync_line() -> re.Pattern[str]:
    """Phase 154 (task 02): the boot re-render line — nameless (the
    safe name was page-local — ``lastUploadName`` is null after a
    reload, phase 90) but SELF-DATING through the completion stamp
    (``fmtUploadStamp`` from the server payload): "Uploaded <stamp> —
    press Sync sources to import it." (the same-day ``HH:MM`` form in
    E2E)."""
    return re.compile(
        r"^Uploaded \d{1,2}:\d{2} — press Sync sources to import it\.$"
    )
UPLOAD_ARCHIVE = f"{UPLOAD_NAME}.tar.gz"
N_FILES = 25
UPLOAD_FILES: dict[str, str] = {
    f"docs/{i:02d}.md": f"# Doc {i:02d}\n\nDeterministic upload content {i:02d}.\n"
    for i in range(N_FILES)
}

#: The local sync source (host temp dir — the app runs on the same
#: machine): 25 small docs under ``notes/`` (the current_file's
#: source/relative/path shape has a directory level).
SYNC_SOURCE_DIR = "sync-corpus"
SYNC_FILES: dict[str, str] = {
    f"notes/{i:02d}.md": f"# Sync doc {i:02d}\n\nDeterministic sync content {i:02d}.\n"
    for i in range(N_FILES)
}

#: "Synced HH:MM" — the local-time last-result label (sources.js's
#: fmtSyncTime), any hour/minute (test_sync_button.py's pattern).
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")

#: Generous settle budget: a 25-file sync against the slowed LLM is
#: ≈4.2 s; the UI's 2 s poll settles at most one tick after the
#: terminal state lands.
SETTLE_TIMEOUT_MS = 45_000


# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------


def _build_targz(path: Path, files: dict[str, str]) -> Path:
    """A deterministic ``.tar.gz`` (mtime 0) over the given files."""
    with tarfile.open(path, "w:gz") as tf:
        for rel, content in files.items():
            data = content.encode("utf-8")
            info = tarfile.TarInfo(rel)
            info.size = len(data)
            info.mtime = 0
            tf.addfile(info, io.BytesIO(data))
    return path


@pytest.fixture(scope="module")
def slow_llm(mock_llm: int) -> Iterator[int]:
    """The delay-injecting reverse proxy in front of the mock LLM
    (tests/e2e/slow_llm.py) — this suite's timing fixture for the SYNC
    legs: the live-file contract needs the sync to outlive the UI's
    2 s poll (see ``SLOW_DELAY_S``)."""
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["SLOW_LLM_DELAY_S"] = SLOW_DELAY_S
    env["E2E_MOCK_PORT"] = str(MOCK_PORT)
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "tests.e2e.slow_llm:app",
         "--host", "127.0.0.1", "--port", str(SLOW_PORT), "--log-level", "warning"],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{SLOW_URL}/v1/models")
        yield SLOW_PORT
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def upload_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The app's ``BOR_UPLOAD_DIR`` for this suite — a scratch dir the
    host-side assertions inspect (the app server runs on the same
    machine)."""
    return tmp_path_factory.mktemp("bor_uploads") / "uploads"


@pytest.fixture(scope="module")
def upload_archive(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The 25-file upload archive (``e2e-prog.tar.gz``)."""
    root = tmp_path_factory.mktemp("bor_archive")
    return _build_targz(root / UPLOAD_ARCHIVE, UPLOAD_FILES)


@pytest.fixture(scope="module")
def sync_local_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The host temp dir the sync test registers as a ``kind=local``
    source — ``sync-corpus/notes/NN.md`` (25 files)."""
    root = tmp_path_factory.mktemp("bor_sync_local") / SYNC_SOURCE_DIR
    for rel, content in SYNC_FILES.items():
        path = root / rel
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text(content, encoding="utf-8")
    return root


@pytest.fixture(scope="module")
def app_server(
    mock_llm: int,
    slow_llm: int,
    upload_dir: Path,
    tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[str]:
    """The real app under test — per-module env: the LLM base URL is
    the SLOW PROXY in front of the mock (the sync-leg timing fixture;
    the upload run makes no LLM calls — phase 90), uploads unpack into
    a scratch dir, and the env git list is forced empty (the sync
    sources are this suite's own ``kind=local`` rows, seeded per test)."""
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["BOR_ENVIRONMENT"] = "e2e"
    env["BOR_STATIC_DIR"] = str(REPO / "frontend")
    env["BOR_LLM_BASE_URL"] = (
        "https://aipi.reeseapps.com/v1"
        if USE_REAL_LLM
        else f"{SLOW_URL}/v1"
    )
    # Mock-calibrated threshold (conftest pattern) — no chat turn is
    # ever sent in this suite, but the app boots with the same env shape.
    env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
    env.setdefault(
        "BOR_DATABASE_URL",
        "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
    )
    # Phase 16: admin auth must be set or create_app() refuses to boot.
    env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
    env["BOR_SESSION_SECRET"] = SESSION_SECRET
    env["BOR_GIT_SOURCES"] = ""
    env["BOR_UPLOAD_DIR"] = str(upload_dir)
    env["BOR_SOURCES_DIR"] = str(tmp_path_factory.mktemp("bor_checkouts"))
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "app.main:app",
         "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{APP_URL}/api/health")
        yield APP_URL
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
    return app_server


def _truncate_all() -> None:
    """Fresh registry + KB per test (the E2E isolation pattern): the
    upload's/sync's counts and every ``/api/docs`` assertion must be
    this test's own doing. The E2E suites share one Postgres, and a
    leftover git_sources row or document would corrupt the row-count
    and doc-list assertions (and a leftover row would be a SECOND sync
    source, skewing the per-file counts)."""
    with SessionLocal() as db:
        db.execute(text("TRUNCATE chunks, documents, query_log, kb_overview, git_sources"))
        db.commit()


def _wait_no_running_jobs(app_url: str) -> None:
    """No background job may leak across tests (the run states live in
    the app's memory, and a still-running scan would keep importing
    into the NEXT test's truncated KB): wait for both status endpoints
    to be non-running BEFORE the truncate. Own admin session (the
    endpoints are admin-only) — usually a no-op: the tests settle only
    after their run's terminal state."""
    with httpx.Client(timeout=5.0) as client:
        client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
        for path in ("/api/sync/status", "/api/git-sources/upload/status"):
            body: dict[str, Any] = {}
            deadline = time.monotonic() + 90
            while time.monotonic() < deadline:
                r = client.get(f"{app_url}{path}")
                if r.status_code == 200:
                    body = r.json()
                    if body["state"] != "running":
                        break
                time.sleep(0.2)
            assert body["state"] != "running", (
                f"a background job was still running at test boundary: {path} {body}"
            )


@pytest.fixture(autouse=True)
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
    _wait_no_running_jobs(app_url)
    _truncate_all()
    yield
    _truncate_all()


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------


def _seed_local_source(path: str) -> None:
    """Register a ``kind=local`` source row directly (deterministic —
    the phase-49 page form is upload-only; a plain directory is an
    API/DB-only operation, the test_local_directory_sources.py note)."""
    with SessionLocal() as db:
        db.add(GitSource(url=path, kind="local", path=path))
        db.commit()


def _admin_git_sources_page(page: Page, app_url: str) -> None:
    """Real form login landing on the git sources page (admin settled:
    Sign out visible, the manager revealed by the page module)."""
    login(page, app_url, next=GIT_SOURCES_URL)
    expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
    expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
    expect(page.locator("#git-sources-gate")).to_be_hidden()
    expect(page.locator("#git-sources-content")).to_be_visible()


def _status(page: Page, app_url: str, path: str) -> dict[str, Any]:
    """One (cookie-authenticated) status-endpoint fetch."""
    r = page.request.get(f"{app_url}{path}")
    assert r.status == 200, r.text
    return r.json()


def _hold_first_status_fetch(page: Page, hold_s: float) -> None:
    """Intercept the upload-status GETs and hold ONLY THE FIRST one for
    ``hold_s`` seconds (later fetches pass straight through). Install
    AFTER the page's boot re-attach fetch, before the submit. The
    poll's first tick fires 2 s after the 202; holding its fetch keeps
    the button in the in-run state long enough to assert the bare
    "Processing…" label (no file, no `N remaining` count) across the whole
    background run — phase 90's run settles in milliseconds, so without
    the hold the in-run window is only the 2 s pre-tick gap."""
    state = {"held": False}

    def handle(route: Any) -> None:
        if not state["held"]:
            state["held"] = True
            time.sleep(hold_s)
        route.continue_()

    page.route("**/api/git-sources/upload/status", handle)


class _TickRecorder:
    """The deterministic layer, concurrent with the UI assertions.

    A daemon thread that tight-polls (~100 ms cadence) the status
    endpoint with its OWN admin session (``httpx`` — the browser page
    drives itself in the meantime; the Playwright sync API is not
    thread-safe, so the thread never touches it), recording every tick
    from the first poll: the idle prelude, the running ticks, and the
    terminal body. The thread only READS the same endpoint the UI's
    2 s poll reads — it starts no jobs and cannot skew the run.

    ``require_running`` (phase 90): the unpack-only upload run settles
    in milliseconds, so a fast machine can miss every running tick —
    with the flag off, a terminal is accepted when the run it names
    started at or after this recorder's start (the stale-terminal
    guard: the run state lives in the app's memory, so a previous
    test's terminal must not be mistaken for this run's)."""

    def __init__(self, app_url: str, path: str, require_running: bool = True) -> None:
        self._url = f"{app_url}{path}"
        self._login_url = f"{app_url}/api/login"
        self._require_running = require_running
        self._t0 = time.time()
        self._ticks: list[dict[str, Any]] = []
        self._terminal: dict[str, Any] | None = None
        self._stop = threading.Event()
        self._thread: threading.Thread | None = None

    def _terminal_is_this_run(self, body: dict[str, Any], saw_running: bool) -> bool:
        """See the class docstring — a terminal is accepted when the
        run it names is THIS recorder's: a running tick was observed,
        or (``require_running=False``) its ``started_at`` is no earlier
        than the recorder's start."""
        if saw_running:
            return True
        if self._require_running:
            return False
        started = body.get("started_at")
        if not started:
            return False
        try:
            started_at = datetime.fromisoformat(str(started)).timestamp()
        except ValueError:
            return False
        return started_at >= self._t0 - 2.0  # tolerance for the pre-submit gap

    def start(self) -> None:
        def run() -> None:
            with httpx.Client(timeout=5.0) as client:
                # Its own admin session — the recorder's reads must not
                # depend on (or disturb) the browser context's cookie.
                client.post(self._login_url, json={"password": ADMIN_PASSWORD})
                saw_running = False
                while not self._stop.is_set():
                    try:
                        r = client.get(self._url)
                        if r.status_code == 200:
                            body = r.json()
                            self._ticks.append(body)
                            if body["state"] == "running":
                                saw_running = True
                            elif (
                                body["state"] in ("success", "failed")
                                and self._terminal_is_this_run(body, saw_running)
                            ):
                                self._terminal = body
                                return
                    except Exception:  # noqa: BLE001 — blip: retry next tick
                        pass
                    self._stop.wait(0.1)

        self._thread = threading.Thread(target=run, daemon=True)
        self._thread.start()

    def stop(self, timeout_s: float = 60.0) -> dict[str, Any]:
        """Join until a terminal tick is recorded (or fail the test)."""
        deadline = time.monotonic() + timeout_s
        while time.monotonic() < deadline:
            if self._terminal is not None:
                break
            time.sleep(0.05)
        self._stop.set()
        if self._thread is not None:
            self._thread.join(timeout=5)
        assert self._terminal is not None, (
            "the recorder saw no terminal state "
            f"(last ticks: {self._ticks[-3:] if self._ticks else 'none'})"
        )
        return self._terminal

    @property
    def running_ticks(self) -> list[dict[str, Any]]:
        return [t for t in self._ticks if t["state"] == "running"]


def _assert_live_file_ticks(
    ticks: list[dict[str, Any]], source: str, n_files: int
) -> None:
    """A4 against the recorded running ticks (the deterministic layer):

    * the file-less ticks (model probe — before any file is indexed)
      come FIRST (the label is the bare prefix then);
    * SOME tick reports a non-null ``current_file`` in the
      ``source/relative/path`` shape;
    * ``files_total`` is the full pre-walk count from the first file
      tick, and ``files_done`` advances monotonically to it.
    """
    with_file = [t for t in ticks if t["current_file"]]
    assert with_file, f"no running tick carried a current_file: {ticks[:6]}"
    first_with = with_file[0]
    assert first_with["current_file"].startswith(f"{source}/"), first_with
    assert re.fullmatch(
        rf"{re.escape(source)}/.+\.(md|markdown)", first_with["current_file"]
    ), first_with
    assert first_with["files_total"] == n_files, first_with
    assert 1 <= first_with["files_done"] <= n_files, first_with
    dones = [t["files_done"] for t in with_file]
    assert dones == sorted(dones), f"files_done not monotonic: {dones}"
    # The last file tick is (n-1, n): the hook fires before the final
    # file's index, so the final count itself can land in the terminal
    # body instead of a recorded tick (100 ms cadence vs ≈160 ms file).
    assert max(dones) >= n_files - 1, f"files_done never advanced: {dones}"
    pre = [t for t in ticks if t["current_file"] is None]
    assert pre, f"no file-less running tick (the probe phase): {ticks[:6]}"
    assert ticks.index(pre[0]) < ticks.index(first_with), (
        "a file tick preceded the file-less probe ticks"
    )


# ---------------------------------------------------------------------------
# 1. Toast on 202 → navigate away → nothing indexed → the sync that
#    follows shows its live file label and lands the counts (A1/A2 +
#    the phase-90 leg)
# ---------------------------------------------------------------------------


def test_upload_toast_then_navigate_away(
    page: Page, app_url: str, db_ready: None, upload_archive: Path
) -> None:
    """On ``/git-sources.html``: pick the multi-file archive, submit →
    the "Successfully uploaded — <source>" toast fires at the 202
    (A2); immediately navigate to ``/sources.html`` → **zero indexed
    documents** (phase 90 A1: the upload unpacked + registered only)
    and the sync button settled idle with no error UI; then click
    **Sync sources** (the new leg, phase 90) → the LIVE
    "Syncing… <file> N remaining" detail while the sync imports the uploaded
    row (phase 149: the box under the button row — the button is the
    spinner — at both layers), then the success settle with the counts
    and the catalog refresh (the box hides)."""
    page.set_default_timeout(30_000)
    _admin_git_sources_page(page, app_url)
    # (A previous test's terminal run may re-render its result line at
    # boot — the re-attach contract; the submit below clears it.)

    page.set_input_files("#archive-upload-file", str(upload_archive))
    page.click("#archive-upload-btn")

    # The 202 moment (A2): a single .toast node, visible, role=status,
    # naming the safe source name…
    toast = page.locator(".toast")
    expect(toast).to_have_count(1, timeout=SETTLE_TIMEOUT_MS)
    expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
    assert toast.get_attribute("role") == "status"
    expect(toast).to_have_text(f"Successfully uploaded — {UPLOAD_NAME}")

    # Navigate away immediately (A1: the run no longer dies with the
    # page).
    page.goto(app_url + SOURCES_URL)

    # Phase 90 A1: the upload indexed NOTHING — the catalog shows the
    # registered source with 0 documents (phase 97: the source row IS
    # the load-settled signal; the file table stays hidden/empty)…
    row = page.locator("#folders-tbody tr", has_text=UPLOAD_NAME)
    expect(row).to_have_count(1, timeout=30_000)
    expect(row.locator("td:nth-child(2)")).to_have_text("0")
    expect(page.locator("#docs-tbody tr")).to_have_count(0)
    # …and the sync button settles idle with no error UI (the
    # sub-second upload run is over by the time this page's 2 s poll
    # first ticks; the upload's counts never render here — A3).
    btn = page.locator("#sync-btn")
    expect(btn).to_be_visible(timeout=30_000)
    expect(page.locator("#sync-error-banner")).to_be_hidden()
    expect(page.locator("#sync-label")).to_have_text(
        "Sync sources", timeout=SETTLE_TIMEOUT_MS
    )
    expect(btn).to_be_enabled()
    expect(btn).not_to_have_attribute("aria-busy")

    # The new leg (phase 90): the Sync button does the scan the upload
    # deferred — live file label at both layers, counts + catalog on
    # the settle.
    recorder = _TickRecorder(app_url, "/api/sync/status")
    recorder.start()
    btn.click()

    # The click's immediate state (A4 — the bare prefix until the
    # import's first file): disabled, aria-busy, spinning icon, no
    # error, and the detail box revealed with the bare label
    # (phase 149 — the running label rides the box)…
    expect(btn).to_be_disabled()
    expect(btn).to_have_attribute("aria-busy", "true")
    expect(btn.locator(".sync-icon")).to_have_class(re.compile(r"\bis-spinning\b"))
    expect(page.locator("#sync-progress")).to_have_text("Syncing…")
    expect(page.locator("#sync-error-banner")).to_be_hidden()

    # UI layer (phase 149 task 01): the RUNNING detail rides the
    # detail box — the page's 2 s poll tick reveals the live file in
    # #sync-progress as the FULL untruncated string: "Syncing…
    # <source/rel/path> N remaining" (phase 193 — the count part
    # counts DOWN and hides on the FINAL file, so the part is optional
    # in this wait; the pre-phase "(n/m)" tail can no longer match)
    # (+ " · chunk n/m" while a file's embed batch is in flight —
    # phase 129, batch granularity). The box is the visible running
    # surface — nothing is ellipsized away, the count is IN the full
    # string (the never-clipped pin), and the button title carries the
    # same full string (A4 — one source, two surfaces).
    expect(page.locator("#sync-progress-text")).to_have_text(counts_line(
        re.compile(
            rf"^Syncing… {re.escape(UPLOAD_NAME)}/.+\.md"
            r"(?: [0-9]+ remaining)?(?: · chunk \d+/\d+)?$"
        )),
        timeout=SETTLE_TIMEOUT_MS,
    )
    # The box is the VISIBLE running surface (the count's old
    # "never clipped" pin now lives in its full string — the
    # "N remaining" part is in the text above).
    expect(page.locator("#sync-progress")).to_be_visible(
        timeout=SETTLE_TIMEOUT_MS
    )
    expect(btn).to_have_attribute(
        "title",
        re.compile(
            rf"^Syncing… {re.escape(UPLOAD_NAME)}/.+\.md"
            r"(?: [0-9]+ remaining)?(?: · chunk \d+/\d+)?$"
        ),
        timeout=SETTLE_TIMEOUT_MS,
    )

    # Deterministic layer: the recorder's full tick series — file-less
    # model-check ticks first, then the per-file ticks (full
    # denominator, advancing counts).
    terminal = recorder.stop()
    assert terminal["state"] == "success", terminal
    _assert_live_file_ticks(recorder.running_ticks, UPLOAD_NAME, N_FILES)
    assert terminal["current_file"] is None
    assert terminal["files_done"] == N_FILES
    assert terminal["files_total"] == N_FILES
    assert terminal["detail"]["added"] == N_FILES

    # The success settle: "Synced HH:MM" + the counts result line, the
    # button re-enabled, no error — and the catalog lists the
    # imported docs (the upload's row, scanned by the sync).
    expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
    expect(btn).to_be_enabled()
    expect(btn).not_to_have_attribute("aria-busy")
    # The counts line rides the box (phase 150 — the sole sync surface).
    expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
    # Phase 97: the catalog is the drill-down tree — drill source →
    # folder `docs` before the per-level row asserts (the flat total
    # no longer exists in one tbody).
    page.click(f'#folders-tbody a.folder-link:text-is("{UPLOAD_NAME}")')
    page.click('#folders-tbody a.folder-link:text-is("docs")')
    expect(page.locator("#docs-tbody tr")).to_have_count(N_FILES, timeout=30_000)
    expect(page.locator("#docs-tbody tr", has_text="docs/00.md")).to_have_count(1)
    expect(page.locator("#docs-tbody tr", has_text="docs/24.md")).to_have_count(1)
    expect(page.locator("#docs-tbody tr", has_text=UPLOAD_NAME)).to_have_count(N_FILES)


# ---------------------------------------------------------------------------
# 2. Upload processing: BARE "Processing…" for the whole run, the
#    no-count terminal shape, nothing indexed (phase 90 A2)
# ---------------------------------------------------------------------------


def test_upload_progress_is_bare(
    page: Page, app_url: str, db_ready: None, upload_archive: Path
) -> None:
    """Phase 90 A2: the upload's background run has NO file-level
    progress. The button shows the BARE "Processing…" for the whole
    run (no file, no `N remaining` count, no title — proven across a held first
    status GET); every running tick the recorder catches carries a
    null ``current_file`` and 0/0 counts; the terminal status is
    ``success`` with the no-count ``{"message": "uploaded"}`` detail
    and null/0/0 progress; the settled result line points at the Sync
    button; and the KB stays empty (no scan)."""
    page.set_default_timeout(30_000)
    _admin_git_sources_page(page, app_url)

    # require_running=False — phase 90: the unpack-only run settles in
    # milliseconds, so a fast machine can miss every running tick; the
    # stale-terminal guard (started_at vs. the recorder's start) keeps
    # a previous test's terminal from being mistaken for this run's.
    recorder = _TickRecorder(
        app_url, "/api/git-sources/upload/status", require_running=False
    )
    recorder.start()
    # Hold the FIRST status GET (installed after the boot re-attach
    # fetch, before the submit): the bare in-run label gets a window
    # wider than the 2 s pre-tick gap.
    _hold_first_status_fetch(page, hold_s=4.5)
    page.set_input_files("#archive-upload-file", str(upload_archive))
    page.click("#archive-upload-btn")
    # A new attempt starts clean (the submit handler hides the result
    # line — any previous run's re-rendered line is gone by now).
    expect(page.locator("#archive-upload-result")).to_be_hidden()

    # The toast fires at the 202 — NOT after the result (the result
    # line is still down when the toast is up).
    toast = page.locator(".toast")
    expect(toast).to_have_count(1, timeout=SETTLE_TIMEOUT_MS)
    expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
    expect(toast).to_have_text(f"Successfully uploaded — {UPLOAD_NAME}")
    expect(page.locator("#archive-upload-result")).to_be_hidden()

    # The button hands over to the run — the BARE "Processing…" (phase
    # 90 A2: no file, no counts, no title)…
    btn = page.locator("#archive-upload-btn")
    expect(btn).to_be_disabled()
    expect(btn).to_have_text("Processing…", timeout=5_000)
    expect(btn).to_have_attribute("title", "")
    # …and it STAYS bare across the whole background run: the first
    # status GET is held, so the settling tick is in flight — no file
    # and no `N remaining` count can have rendered.
    time.sleep(2.5)
    expect(btn).to_have_text("Processing…")

    # Deterministic layer: every running tick the recorder caught is
    # bare (null file, 0/0 counts — phase 90 A2).
    for t in recorder.running_ticks:
        assert t["current_file"] is None, t
        assert t["files_done"] == 0 and t["files_total"] == 0, t
    terminal = recorder.stop()
    assert terminal["state"] == "success", terminal
    assert terminal["detail"] == {"message": "uploaded"}, terminal
    assert terminal["current_file"] is None
    assert terminal["files_done"] == 0
    assert terminal["files_total"] == 0

    # Settle (the held GET released): the ready-for-sync result line,
    # the button restored ("Upload"), the input cleared, one row for
    # the archive — and the KB empty (no scan, phase 90 A1).
    result = page.locator("#archive-upload-result")
    expect(result).to_have_text(
        _ready_for_sync_line(UPLOAD_NAME),
        timeout=30_000,
    )
    expect(btn).to_be_enabled()
    expect(btn).to_have_text("Upload")
    expect(page.locator("#archive-upload-file")).to_have_value("")
    expect(page.locator("#git-sources-tbody tr", has_text=UPLOAD_NAME)).to_have_count(1)
    r = page.request.get(f"{app_url}/api/docs")
    assert r.status == 200, r.text
    assert r.json()["documents"] == []


# ---------------------------------------------------------------------------
# 3. Sync live file label: both layers, then the preserved pre-64
#    success settle (A4)
# ---------------------------------------------------------------------------


def test_sync_live_file_label(
    page: Page, app_url: str, db_ready: None, sync_local_dir: Path
) -> None:
    """A multi-file local source (the ``test_sync_button.py`` fixture
    style): on ``/sources.html`` click **Sync sources** → the detail
    box (phase 149 — the running label left the pill; the button is
    the spinner) shows "Syncing…" (bare, the pre-64 click state) then
    "Syncing… <file> N remaining" while running (endpoint layer:
    ``current_file`` non-null at running ticks; UI layer: the box
    poll), then the success settle with the counts result line (the
    box hides) — the pre-phase-64 sync UX's settle is preserved, plus
    the file in the label while running."""
    page.set_default_timeout(30_000)
    _seed_local_source(str(sync_local_dir))
    login(page, app_url)  # lands on /sources.html (the button's home)

    btn = page.locator("#sync-btn")
    expect(btn).to_be_visible(timeout=30_000)
    # The button is retry-ready at boot: idle ("Sync sources") — or the
    # RE-ATTACHED terminal from a prior test's sync (the module app is
    # shared, its in-memory sync state survives the test boundary, and
    # the phase-32 boot re-attach renders the last result — "Synced
    # HH:MM" after test 1's successful run). Either way the button is
    # enabled and a click starts THIS test's run (the assertions below
    # are the run's own).
    expect(page.locator("#sync-label")).to_have_text(
        re.compile(r"^(Sync sources|Synced \d{1,2}:\d{2})$")
    )
    expect(btn).to_be_enabled()

    recorder = _TickRecorder(app_url, "/api/sync/status")
    recorder.start()
    btn.click()

    # The click's immediate state is the pre-64 one (A4 — the bare
    # prefix until the import's first file): disabled, aria-busy,
    # spinning icon, no error, and the detail box revealed with the
    # bare label (phase 149 — the running label rides the box)…
    expect(btn).to_be_disabled()
    expect(btn).to_have_attribute("aria-busy", "true")
    expect(btn.locator(".sync-icon")).to_have_class(re.compile(r"\bis-spinning\b"))
    expect(page.locator("#sync-progress")).to_have_text("Syncing…")
    expect(page.locator("#sync-error-banner")).to_be_hidden()

    # UI layer (phase 149 task 01): the RUNNING detail rides the
    # detail box — the page's 2 s poll tick reveals the live file in
    # #sync-progress as the FULL untruncated string: "Syncing…
    # <source/rel/path> N remaining" (phase 193 — the count part
    # counts DOWN and hides on the FINAL file, so the part is optional
    # in this wait; the pre-phase "(n/m)" tail can no longer match)
    # (+ " · chunk n/m" while a file's embed batch is in flight —
    # phase 129, batch granularity). The box is the visible running
    # surface — nothing is ellipsized away, the count is IN the full
    # string (the never-clipped pin), and the button title carries the
    # same full string (A4 — one source, two surfaces).
    expect(page.locator("#sync-progress-text")).to_have_text(counts_line(
        re.compile(
            rf"^Syncing… {re.escape(SYNC_SOURCE_DIR)}/.+\.md"
            r"(?: [0-9]+ remaining)?(?: · chunk \d+/\d+)?$"
        )),
        timeout=SETTLE_TIMEOUT_MS,
    )
    # The box is the VISIBLE running surface (the count's old
    # "never clipped" pin now lives in its full string — the
    # "N remaining" part is in the text above).
    expect(page.locator("#sync-progress")).to_be_visible(
        timeout=SETTLE_TIMEOUT_MS
    )
    expect(btn).to_have_attribute(
        "title",
        re.compile(
            "^Syncing… "
            rf"{re.escape(SYNC_SOURCE_DIR)}/.+\.md"
            r"(?: [0-9]+ remaining)?(?: · chunk \d+/\d+)?$"
        ),
        timeout=SETTLE_TIMEOUT_MS,
    )

    # Deterministic layer: the recorder's full tick series — file-less
    # model-check ticks first, then the per-file ticks (full
    # denominator, advancing counts).
    terminal = recorder.stop()
    assert terminal["state"] == "success", terminal
    _assert_live_file_ticks(recorder.running_ticks, SYNC_SOURCE_DIR, N_FILES)
    assert terminal["current_file"] is None
    assert terminal["files_done"] == N_FILES
    assert terminal["files_total"] == N_FILES
    assert terminal["detail"]["added"] == N_FILES

    # The pre-64 success settle, preserved: "Synced HH:MM" + the counts
    # result line, button re-enabled, no error — and the catalog lists
    # the imported docs.
    expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
    expect(btn).to_be_enabled()
    expect(btn).not_to_have_attribute("aria-busy")
    # The counts line rides the box (phase 150 — the sole sync surface).
    expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
    # Phase 97: the catalog is the drill-down tree — drill source →
    # folder `notes` before the per-level row asserts (the flat total
    # no longer exists in one tbody).
    page.click(f'#folders-tbody a.folder-link:text-is("{SYNC_SOURCE_DIR}")')
    page.click('#folders-tbody a.folder-link:text-is("notes")')
    expect(page.locator("#docs-tbody tr")).to_have_count(N_FILES, timeout=30_000)
    expect(page.locator("#docs-tbody tr", has_text="notes/00.md")).to_have_count(1)
    expect(page.locator("#docs-tbody tr", has_text=SYNC_SOURCE_DIR)).to_have_count(N_FILES)


# ---------------------------------------------------------------------------
# 4. Reload → re-attach: no dead-end, no error, no toast, no second
#    upload (A1 + A2, phase 90)
# ---------------------------------------------------------------------------


def test_upload_reattach_after_reload(
    page: Page, app_url: str, db_ready: None, upload_archive: Path
) -> None:
    """Start the upload and, while the sub-second run is in flight (or
    has just settled), reload ``/git-sources.html`` → the page never
    dead-ends: the boot re-attach either enters the bare Processing
    state + poll (run still in flight) or re-renders the settled
    result line (the NAMELESS variant — the safe name was page-local),
    with no error banner, no toast, and NO second upload (the status
    endpoint's single run is still the one from before the reload —
    pinned on its ``started_at``); the list shows exactly one row for
    the archive (in-place identity preserved)."""
    page.set_default_timeout(30_000)
    _admin_git_sources_page(page, app_url)

    page.set_input_files("#archive-upload-file", str(upload_archive))
    page.click("#archive-upload-btn")

    # The 202 toast (the run is in flight)…
    toast = page.locator(".toast")
    expect(toast).to_have_count(1, timeout=SETTLE_TIMEOUT_MS)
    expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
    # …and the run's identity: the status's started_at (a second run
    # would reset it — the single-run claim is pinned on it).
    started_at = _status(page, app_url, "/api/git-sources/upload/status")["started_at"]

    # Reload — the run may be in flight (the boot re-attach enters the
    # bare Processing state + poll) or settled (the boot re-renders
    # the result line); either way the page must not dead-end.
    page.reload()
    expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
    expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
    expect(page.locator("#git-sources-content")).to_be_visible()

    # No error banner, no toast at boot (A2 — the toast fired at the
    # 202, on the previous document life)…
    expect(page.locator("#archive-upload-error")).to_be_hidden()
    expect(page.locator(".toast")).to_have_count(0)
    # …and the settled result line: the nameless variant (the safe
    # name was page-local — lastUploadName is null after a reload),
    # self-dating through the completion stamp (phase 154 — the stamp
    # comes from the server payload, so the line is still self-dating
    # with the name unknown).
    expect(page.locator("#archive-upload-result")).to_have_text(
        _nameless_ready_for_sync_line(),
        timeout=SETTLE_TIMEOUT_MS,
    )
    expect(page.locator("#archive-upload-btn")).to_be_enabled()
    expect(page.locator("#archive-upload-btn")).to_have_text("Upload")
    # …and the list shows exactly one row for the archive.
    expect(page.locator("#git-sources-tbody tr")).to_have_count(1)
    row = page.locator("#git-sources-tbody tr", has_text=UPLOAD_NAME)
    expect(row).to_have_count(1)
    expect(row.locator("span.git-source-kind")).to_have_text("Local")

    # No second upload: the terminal status is the SAME run — its
    # started_at is the one from before the reload.
    terminal = _status(page, app_url, "/api/git-sources/upload/status")
    assert terminal["state"] == "success", terminal
    assert str(terminal["started_at"]) == str(started_at), (
        f"the run's identity changed (a second upload ran): {terminal['started_at']}"
    )
    assert terminal["current_file"] is None
    assert terminal["detail"] == {"message": "uploaded"}