brain-of-reese/tests/e2e/test_sync_remaining_count.py

"""Phase 193 story E2E (Playwright): the sync label counts DOWN.

Run in isolation (DB must be up: ``podman compose up -d db``):

    uv run pytest tests/e2e/test_sync_remaining_count.py -v --no-cov

**Source (``TODO.md:7``):** "The sync status should show how many files
are left to sync, not how many files out of the total." The pre-phase
running label said ``Syncing… notes/a.md (1234/27000)`` — the owner's
position *in* the whole; the owner reads the distance *to the end*.
Phase 193 changes exactly ONE label part: ``fmtSyncLabel``'s count is
now ``N remaining`` with ``N = total − done``.

**Server untouched.** No endpoint, no payload field, no poll changed:
``GET /api/sync/status`` still answers with the phase-64 key set
(``current_file`` / ``files_done`` / ``files_total`` — the 1-based
position of the in-flight file over the pre-walk count), and the suite
pins exactly that (test 1's cross-layer invariant): the payload still
carries the ``(done, total)`` position while the label speaks the
subtraction of the very same pair. The change is one string built by
``frontend/assets/sources.js``.

**A1 (owner-confirmed):** ``N remaining`` shows while ``N ≥ 1`` and is
HIDDEN at the final file (``N == 0``) — the main part names that file;
``total === 0`` (the pre-walk, the phase-64 contract) also renders no
count part. **A2 (owner-confirmed):** the upload job shares the ONE
builder, so it shares the shape (``Importing <file> N remaining``).

Fixture: a real git repo (the phase-28/32/107 ``file://`` transport-true
convention — ``BOR_GIT_SOURCES=file://<fixture repo>``, so the env
fallback IS the effective source under the autouse truncate) holding
``N_FILES = 5`` markdown docs — the first AND the last
multi-chunk (the phase-129 chunk-part climb rides a real embed span, and
the last one is the window whose label must carry NO count part), the
three in between one chunk each. ``tests/e2e/slow_llm.py`` delays every
LLM call ``SLOW_DELAY_S``, so a file's import span (content-embed
batches + the summary chat + the summary-chunk embed) is ≫ the UI's 2 s
status poll: every file gets its own rendered label, deterministically,
instead of racing it.

What the owner sees is captured, not sampled: a MutationObserver
installed before the click records EVERY value the single label writer
lands — the ``#sync-progress-text`` box (the phase-150 sole sync text
surface + sole ``aria-live`` region), the button ``title`` and
``aria-label`` (the phase-64 A4 untruncated placement) and the
``.sync-label-count`` span's visibility — so the assertions read the
whole rendered sequence, not the few instants a Python poll happens to
hit.

Arms (the phase's three, in test order):

1. ``test_running_label_counts_down_across_a_real_sync`` — **(a) the
   count-down**: every rendered import label matches
   ``^Syncing… <file> [1-N] remaining(?: · chunk n/m)?$``; the sequence
   of rendered ``N``s is strictly DECREASING (each new value smaller),
   never ``0``, never ``≥ total``; the ``· chunk n/m`` part coexists
   with it (its ``m`` equals the in-flight file's real
   ``chunk_document`` count — phase 129 unbroken); at the FINAL file no
   label carries a count part at all while the main part still names
   that file; no ``Syncing…`` label anywhere carries the pre-phase
   ``(n/m)`` tail; the ``(n/m)`` shape survives ONLY in the summaries
   phase (A1's exception) and the overview label still paints; the
   terminal result line is byte-identical to the phase-64 counts line
   (the ``counts_line`` helper); and the endpoint's own tick series
   still reports the untouched ``(done, total)`` position whose
   subtraction equals every rendered ``N``.
2. ``test_prewalk_and_final_file_labels_have_no_count_part`` — **(c)
   the pre-walk edge**, pinned on the exact payload the guard reads:
   the page's own 2 s poll fed ``phase: null`` / ``files_total: 0``
   renders the bare ``Syncing…`` (no count part, the count span hidden,
   the title equal), the first file renders ``N − 1 remaining``, a
   mid-run tick renders ``1 remaining`` *beside* ``· chunk 2/5``, the
   final-file tick (``done == total``) renders the file with the count
   part GONE and never brings it back, and the success terminal settles
   to the untouched counts line. The served bodies use the status
   endpoint's unchanged key set — the client's own decision points
   (``total > 0``, ``total − done > 0``) are what is under test.
3. ``test_upload_label_shares_the_count_down_builder`` — **(b) the
   upload arm (A2)**: the upload branch of the same two-job tree shares
   the builder, so a running upload status that carries counts renders
   ``Importing <file> N remaining`` and hides the part at the upload's
   final file, settling to the upload's own unchanged idle surface. The
   REAL background upload run is asserted first, against the live
   endpoint: since phase 90 it is unpack + register only, so it answers
   ``null``/``0``/``0`` for the whole run — which is exactly why the
   shared *shape* is pinned through a served status (the house
   ``page.route`` pattern) instead of a real upload, and why no real
   upload label can ever regress to ``(n/m)``. The upload branch never
   receives a status object (its endpoint has no phase / chunk / image
   fields — phase 129 / 152), so the chunk part coexists in the SYNC
   arms only (test 1 and 2); that is asserted here too.

Test → phase-task mapping (Playwright Mapping Rule, task 03 arms a/b/c):
1 → (a) + the terminal-line pin + the untouched-payload pin,
2 → (c) + the A1 hidden-at-final-file edge (+ (a)'s arithmetic on exact
inputs), 3 → (b).
"""
from __future__ import annotations

import io
import json
import os
import re
import subprocess
import sys
import tarfile
import threading
import time
from collections.abc import Iterator
from datetime import UTC, datetime
from pathlib import Path
from typing import Any

import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text

from app.config import Settings
from app.db import SessionLocal
from app.rag.chunker import chunk_document
from e2e.auth_helpers import login
from e2e.conftest import (
    ADMIN_PASSWORD,
    MOCK_PORT,
    SESSION_SECRET,
    USE_REAL_LLM,
    _wait_http,
)
from e2e.sync_helpers import counts_line

REPO = Path(__file__).resolve().parents[2]

#: This module's own port (the conftest session app owns 8123 in a
#: combined run — the phase-79 convention). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_REMAINING", "8211"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
SLOW_PORT = int(os.environ.get("E2E_SLOW_LLM_PORT_REMAINING", "8914"))
SLOW_URL = f"http://127.0.0.1:{SLOW_PORT}"

SOURCES_URL = "/sources.html"

#: Per-LLM-request delay on the proxy (the phase-64/129 house value).
#: The run's PRELUDE (the two model probes) is ≈ 3 s and therefore
#: outlives the page's first 2 s tick (the bare-label contract the click
#: leaves up), and a one-chunk file costs 3 calls ≈ 4.5 s while a
#: multi-chunk file ≈ 10 s — every file gets several rendered labels of
#: its own (the module docstring's determinism note).
SLOW_DELAY_S = "1.5"

#: The module's wall-time budget (app boot + one full 5-file sync + the
#: two served-status arms). A breach is a speed canary, not flake
#: tolerance (the phase-64 convention).
WALL_BUDGET_S = 300.0

#: The sync's subject: a git repo whose DIRECTORY NAME is the source
#: name (the ``repo_name`` basename rule, phase 28). Deliberately NOT
#: named after the part under test — a source path containing the word
#: "remaining" would make every "does this label carry the count part?"
#: check match the path itself.
SOURCE = "countdown"
N_FILES = 5

#: The walk order is the sorted relative path, so the names below pin
#: the ORDER the owner watches: the first file (multi-chunk → the chunk
#: part climbs while the count reads ``4 remaining``), three single-chunk
#: files (3 → 2 → 1 remaining), the LAST file (multi-chunk → its own
#: chunk part, NO count part — the A1 hidden edge).
FIRST_REL = "notes/a-multi-first.md"
LAST_REL = "notes/z-multi-last.md"
REL_ORDER = [FIRST_REL, "notes/b-second.md", "notes/c-third.md", "notes/d-fourth.md", LAST_REL]

#: The count-bearing values the label may ever show (A1): ``1 … N−1``.
MAX_COUNT = N_FILES - 1

#: The archive the upload arm uploads (the phase-98 fixture convention —
#: the same kind of content as the sync fixture, packaged).
UPLOAD_NAME = "uploadkb"
UPLOAD_FILES: dict[str, str] = {
    f"docs/{i:02d}.md": f"# Upload doc {i:02d}\n\nDeterministic upload body {i:02d}.\n"
    for i in range(3)
}

#: The chunker settings the app under test is pinned to (the leak-guard
#: convention): the expected ``chunks_total`` is computed with the SAME
#: function and arguments the importer uses, so a local ``.env`` cannot
#: move the number the suite asserts.
CHUNK_TARGET = int(Settings.model_fields["chunk_target_chars"].default)
CHUNK_OVERLAP = int(Settings.model_fields["chunk_overlap_chars"].default)

#: The import-phase label grammar since phase 193 — the main part (the
#: prefix + the ``source/relative/path``), the OPTIONAL count part, the
#: OPTIONAL phase-129 chunk part. ``<file>`` is kept loose on purpose
#: where a suite-wide grammar is the point; the per-file assertions
#: below re-anchor it on this suite's own paths.
IMPORT_LABEL_RE = re.compile(
    r"^Syncing… \S+(?: [0-9]+ remaining)?(?: · chunk \d+/\d+)?$"
)
#: The pre-phase shape phase 193 removed from this part — no ``Syncing…``
#: label may ever carry a ``(n/m)`` tail again.
OLD_FRACTION_RE = re.compile(r"\(\d+/\d+\)")
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")

SETTLE_TIMEOUT_MS = 180_000


def _now_iso() -> str:
    """A "just finished" stamp for the served terminals — the phase-154
    same-day rule renders it as the plain ``Synced HH:MM`` (computed per
    call, so a module that runs across midnight stays honest)."""
    return datetime.now(UTC).isoformat()


# ---------------------------------------------------------------------------
# The fixture KB
# ---------------------------------------------------------------------------


def _tiny_md(title: str) -> str:
    """A one-chunk document (its import span = 3 slowed LLM calls)."""
    return f"# {title}\n\nDeterministic one-chunk body for the {title}.\n"


def _multi_md(tag: str, sections: int = 4) -> str:
    """A multi-chunk document: an H1 preamble (one chunk) + *sections*
    ``##`` sections of ~1500 chars each (one chunk apiece at the
    code-default 2000-char target, and one embed batch apiece at the
    700-token request budget) — no randomness, so the chunk count is
    stable run to run (the phase-129 fixture idiom)."""
    parts = [f"# {tag} multi-chunk fixture document\n"]
    parts.append(
        f"Preamble paragraph of the {tag} remaining-count fixture. This "
        "paragraph sits under the H1 and chunks on its own.\n"
    )
    for k in range(1, sections + 1):
        parts.append(f"\n## Section {k}\n")
        base = (
            f"Section {k} paragraph of the {tag} remaining-count fixture "
            "carries deterministic filler for the chunker: alpha bravo "
            "charlie delta echo foxtrot golf hotel india juliet kilo lima "
            "mike november oscar papa quebec romeo sierra tango uniform "
            "victor whiskey xray yankee zulu number "
        )
        parts.append((base * 4)[:1500] + "\n\n")
    return "".join(parts)


def _fixture_files() -> dict[str, str]:
    return {
        FIRST_REL: _multi_md("first"),
        REL_ORDER[1]: _tiny_md("second"),
        REL_ORDER[2]: _tiny_md("third"),
        REL_ORDER[3]: _tiny_md("fourth"),
        LAST_REL: _multi_md("last"),
    }


def _expected_content_chunks(rel: str) -> int:
    """The file's TRUE content-chunk count — the SAME function and
    arguments the importer's chunk replacement uses (the phase-129
    convention: the endpoint's ``chunks_total`` is the chunk count, not
    a batch count)."""
    content = _fixture_files()[rel]
    return len(chunk_document(content, rel, max(400, CHUNK_TARGET), CHUNK_OVERLAP))


def _git(cwd: Path, *args: str) -> None:
    """Run git in *cwd*; a non-zero exit fails the fixture loudly."""
    proc = subprocess.run(["git", *args], cwd=cwd, capture_output=True, text=True)
    if proc.returncode != 0:
        raise AssertionError(f"git {' '.join(args)} failed: {proc.stderr.strip()}")


@pytest.fixture(scope="module")
def remaining_repo(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The real git repo the sync clones over ``file://`` (5 markdown
    docs — see ``REL_ORDER``). ``tmp_path_factory`` because the
    module-scoped app needs it for the module's lifetime."""
    root = tmp_path_factory.mktemp("bor_remaining")
    repo = root / SOURCE
    for rel, content in _fixture_files().items():
        target = repo / rel
        target.parent.mkdir(parents=True, exist_ok=True)
        target.write_text(content, encoding="utf-8")
    _git(repo, "init", "-q")
    _git(repo, "add", "-A")
    _git(
        repo,
        "-c", "user.email=e@x", "-c", "user.name=t", "-c", "commit.gpgsign=false",
        "commit", "-qm", "remaining-count fixture",
    )
    assert (repo / ".git").is_dir()
    assert len(list(repo.rglob("*.md"))) == N_FILES
    return repo


@pytest.fixture(scope="module")
def upload_archive(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The upload arm's archive (the ``test_sync_upload_progress.py``
    convention: the same shape of content as the sync fixture, packed as
    a deterministic ``.tar.gz``)."""
    root = tmp_path_factory.mktemp("bor_remaining_archive")
    path = root / f"{UPLOAD_NAME}.tar.gz"
    with tarfile.open(path, "w:gz") as tf:
        for rel, content in UPLOAD_FILES.items():
            data = content.encode("utf-8")
            info = tarfile.TarInfo(rel)
            info.size = len(data)
            info.mtime = 0
            tf.addfile(info, io.BytesIO(data))
    return path


def _app_env(
    base_url: str, repo: Path, upload_dir: Path, tmp_path_factory: pytest.TempPathFactory
) -> dict[str, str]:
    """The per-module app env (the conftest pattern): the slow proxy is
    the LLM, ``BOR_GIT_SOURCES`` is this suite's own ``file://`` repo (the
    env fallback IS the effective source — the autouse truncate keeps the
    ``git_sources`` table empty), and the chunker knobs are pinned to the
    code defaults so the expected ``chunks_total`` stays predictable
    (process env outranks an operator's gitignored ``.env``)."""
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["BOR_ENVIRONMENT"] = "e2e"
    env["BOR_STATIC_DIR"] = str(REPO / "frontend")
    env["BOR_LLM_BASE_URL"] = (
        "https://aipi.reeseapps.com/v1" if USE_REAL_LLM else base_url
    )
    env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
    env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
    env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
    env["BOR_LLM_RETRY_DELAY"] = "0"
    env.setdefault(
        "BOR_DATABASE_URL",
        "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
    )
    env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
    env["BOR_SESSION_SECRET"] = SESSION_SECRET
    env["BOR_GIT_SOURCES"] = f"file://{repo}"
    env["BOR_SOURCES_DIR"] = str(tmp_path_factory.mktemp("bor_checkouts"))
    env["BOR_UPLOAD_DIR"] = str(upload_dir)
    env["BOR_CHUNK_TARGET_CHARS"] = str(CHUNK_TARGET)
    env["BOR_CHUNK_OVERLAP_CHARS"] = str(CHUNK_OVERLAP)
    env["BOR_DOCS_REPO"] = ""
    env["BOR_SUGGESTIONS"] = json.dumps(["Ask something neutral"])
    return env


@pytest.fixture(scope="module")
def slow_llm(mock_llm: int) -> Iterator[int]:
    """The delay-injecting reverse proxy in front of the mock LLM
    (``tests/e2e/slow_llm.py``) — the suite's timing fixture: it delays
    EVERY LLM call, embeds included, so each file's import span outlives
    the UI's 2 s poll (see ``SLOW_DELAY_S``)."""
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["SLOW_LLM_DELAY_S"] = SLOW_DELAY_S
    env["E2E_MOCK_PORT"] = str(MOCK_PORT)
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "tests.e2e.slow_llm:app",
         "--host", "127.0.0.1", "--port", str(SLOW_PORT), "--log-level", "warning"],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{SLOW_URL}/v1/models")
        yield SLOW_PORT
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def app_server(
    mock_llm: int,
    slow_llm: int,
    remaining_repo: Path,
    tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[str]:
    """The app under test — the slow one, on its own port."""
    env = _app_env(
        f"{SLOW_URL}/v1",
        remaining_repo,
        tmp_path_factory.mktemp("bor_remaining_uploads") / "uploads",
        tmp_path_factory,
    )
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "app.main:app",
         "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{APP_URL}/api/health")
        yield APP_URL
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
    return app_server


def _truncate_all() -> None:
    """Fresh registry + KB per test (the E2E isolation pattern): the
    suites share one Postgres, and a leftover document would move the
    sync's ``added`` counts the terminal line pins."""
    with SessionLocal() as db:
        db.execute(
            text(
                "TRUNCATE chunks, documents, query_log, steering_notes, "
                "kb_overview, git_sources, folder_summaries"
            )
        )
        db.commit()


def _wait_sync_settled(base_url: str, timeout_s: float = 240.0) -> None:
    """No run may leak across a test boundary (the run state lives in the
    app's memory and a running run keeps importing into the NEXT test's
    truncated KB). Own admin session; usually a no-op."""
    with httpx.Client(base_url=base_url, timeout=5.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        deadline = time.monotonic() + timeout_s
        while time.monotonic() < deadline:
            r = client.get("/api/sync/status")
            assert r.status_code == 200, r.text
            body = r.json()
            if body["state"] == "failed":
                print(f"sync remaining-count: FAILED terminal at a boundary: {body.get('error')}")
            if body["state"] != "running":
                return
            time.sleep(0.2)
    raise AssertionError(f"a sync was still running at a test boundary: {base_url}")


@pytest.fixture(autouse=True)
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
    _wait_sync_settled(app_url)
    _truncate_all()
    yield
    _truncate_all()


@pytest.fixture(scope="module", autouse=True)
def _wall_time_guard() -> Iterator[None]:
    start = time.monotonic()
    yield
    elapsed = time.monotonic() - start
    assert elapsed < WALL_BUDGET_S, (
        f"the module took {elapsed:.1f}s — over the {WALL_BUDGET_S:.0f}s wall budget"
    )


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------

#: The in-browser recorder: every value the ONE label writer lands, with
#: the surfaces phase 64's A4 pins together (the box text, the button
#: title, the button aria-label) plus the count span's own visibility —
#: read in one JS task, so a frame can never straddle two writes.
_INSTALL_RECORDER = """
() => {
  if (window.__borFrames) return window.__borFrames.length;
  window.__borFrames = [];
  const box = document.querySelector('#sync-progress');
  const textEl = document.querySelector('#sync-progress-text');
  const btn = document.querySelector('#sync-btn');
  const countEl = document.querySelector('.sync-label-count');
  const rec = () => {
    const frame = {
      text: textEl.textContent,
      title: btn.getAttribute('title') || '',
      aria: btn.getAttribute('aria-label') || '',
      count: countEl ? countEl.textContent : '',
      countHidden: countEl ? countEl.hidden : null,
      hidden: box.hidden,
    };
    const last = window.__borFrames[window.__borFrames.length - 1];
    if (last
        && last.text === frame.text
        && last.title === frame.title
        && last.aria === frame.aria
        && last.countHidden === frame.countHidden) {
      return;
    }
    window.__borFrames.push(frame);
  };
  const mo = new MutationObserver(rec);
  mo.observe(textEl, { childList: true, characterData: true, subtree: true });
  mo.observe(btn, { attributes: true, attributeFilter: ['title', 'aria-label'] });
  if (countEl) mo.observe(countEl, { attributes: true, attributeFilter: ['hidden'] });
  rec();
  return window.__borFrames.length;
}
"""


def _install_recorder(page: Page) -> None:
    page.evaluate(_INSTALL_RECORDER)


def _frames(page: Page) -> list[dict[str, Any]]:
    frames: list[dict[str, Any]] = page.evaluate("() => window.__borFrames")
    assert frames, "the in-browser recorder captured no label frames"
    return frames


def _texts(frames: list[dict[str, Any]]) -> list[str]:
    """The recorded label sequence, empty frames dropped (the box's
    idle-hide is not a label)."""
    return [f["text"] for f in frames if f["text"]]


class _TickRecorder:
    """The deterministic layer: a daemon thread tight-polling (~100 ms)
    the SAME status endpoint the UI's 2 s poll reads, with its OWN admin
    session (the Playwright sync API is not thread-safe, so the thread
    never touches the page). It starts nothing — it only watches."""

    def __init__(self, base_url: str, path: str) -> None:
        self._url = f"{base_url}{path}"
        self._login_url = f"{base_url}/api/login"
        self._ticks: list[dict[str, Any]] = []
        self._terminal: dict[str, Any] | None = None
        self._stop = threading.Event()
        self._thread: threading.Thread | None = None

    def start(self) -> None:
        def run() -> None:
            with httpx.Client(timeout=5.0) as client:
                client.post(self._login_url, json={"password": ADMIN_PASSWORD})
                while not self._stop.is_set():
                    try:
                        r = client.get(self._url)
                        if r.status_code == 200:
                            body = r.json()
                            self._ticks.append(body)
                            if body["state"] in ("success", "failed", "cancelled"):
                                self._terminal = body
                                return
                    except Exception:  # noqa: BLE001 — blip: retry next tick
                        pass
                    self._stop.wait(0.1)

        self._thread = threading.Thread(target=run, daemon=True)
        self._thread.start()

    def stop(self, timeout_s: float = 240.0) -> dict[str, Any]:
        deadline = time.monotonic() + timeout_s
        while time.monotonic() < deadline:
            if self._terminal is not None:
                break
            time.sleep(0.05)
        self._stop.set()
        if self._thread is not None:
            self._thread.join(timeout=5)
        assert self._terminal is not None, (
            f"the recorder saw no terminal state (last: {self._ticks[-3:]})"
        )
        return self._terminal

    @property
    def running_ticks(self) -> list[dict[str, Any]]:
        return [t for t in self._ticks if t["state"] == "running"]


def _serve_statuses(page: Page, stages: dict[str, Any]) -> None:
    """Route the two job-status endpoints to the CURRENT stage body (the
    house ``page.route`` pattern). ``stages`` is mutated by the test
    between the UI waits, so each 2 s poll of the REAL page renders
    exactly the payload the test put in front of it — the client's own
    decision points, pinned without a timing race.

    ``stages`` keys: ``sync`` / ``upload`` → the body dicts. A ``None``
    value falls through to the real endpoint."""

    def handle(route: Any) -> None:
        path = route.request.url
        if "/api/sync/status" in path:
            body = stages.get("sync")
        elif "/api/git-sources/upload/status" in path:
            body = stages.get("upload")
        else:  # pragma: no cover — only the two patterns are routed
            route.fallback()
            return
        if body is None:
            route.fallback()
            return
        route.fulfill(status=200, content_type="application/json", body=json.dumps(body))

    page.route("**/api/sync/status", handle)
    page.route("**/api/git-sources/upload/status", handle)


def _sync_status(**overrides: Any) -> dict[str, Any]:
    """One ``GET /api/sync/status`` body with the endpoint's UNCHANGED
    key set (the phase-64/98/129/152/187 fields the client reads) — the
    served stubs are faithful to the payload shape; only the values are
    the test's."""
    body: dict[str, Any] = {
        "state": "idle",
        "started_at": None,
        "finished_at": None,
        "current_file": None,
        "files_done": 0,
        "files_total": 0,
        "phase": None,
        "current_summary": None,
        "summaries_done": 0,
        "summaries_total": 0,
        "current_chunk": 0,
        "chunks_total": 0,
        "images_done": 0,
        "images_total": 0,
        "detail": {},
        "error": None,
        "rss_mb": None,
        "rss_peak_mb": None,
    }
    body.update(overrides)
    return body


def _upload_status(**overrides: Any) -> dict[str, Any]:
    body: dict[str, Any] = {
        "state": "idle",
        "started_at": None,
        "finished_at": None,
        "current_file": None,
        "files_done": 0,
        "files_total": 0,
        "detail": {},
        "error": None,
    }
    body.update(overrides)
    return body


def _admin_sources_page(page: Page, app_url: str, *, settled: bool = True) -> None:
    """Real form login landing on the RAG page (the button's home).

    ``settled=True`` also waits for the boot re-attach to finish in a
    NON-running state (the phase-64 pattern — no later re-render can
    clobber the state under test). The served-status arms pass
    ``settled=False``: their boot fetch IS the state under test."""
    login(page, app_url)
    expect(page).to_have_url(app_url + SOURCES_URL, timeout=30_000)
    expect(page.locator("#sync-btn")).to_be_visible(timeout=30_000)
    if settled:
        expect(page.locator("#sync-label")).to_have_text(
            re.compile(r"^(Sync sources|Synced \d{1,2}:\d{2})$"), timeout=30_000
        )


# ---------------------------------------------------------------------------
# 1. (a) The count-down across a REAL sync of a REAL git source
# ---------------------------------------------------------------------------


def test_running_label_counts_down_across_a_real_sync(
    page: Page, app_url: str, db_ready: None, remaining_repo: Path
) -> None:
    """The owner-visible contract: the label counts DOWN and goes quiet
    at the final file — over a real ``file://`` git sync with the real
    (untouched) status endpoint."""
    page.set_default_timeout(SETTLE_TIMEOUT_MS)
    _admin_sources_page(page, app_url)
    _install_recorder(page)

    recorder = _TickRecorder(app_url, "/api/sync/status")
    recorder.start()
    btn = page.locator("#sync-btn")
    btn.click()

    # The click's own state (the phase-64 contract the pre-walk keeps):
    # the button becomes the spinner and the box reveals the BARE label.
    expect(btn).to_be_disabled()
    expect(btn).to_have_attribute("aria-busy", "true")
    expect(page.locator("#sync-progress")).to_have_text("Syncing…")

    # The first file's window: the count part AND the chunk part (the
    # first file is the multi-chunk one, so its embed batches climb
    # while the count reads N − 1 remaining — the coexistence pin).
    expect(page.locator("#sync-progress-text")).to_have_text(
        re.compile(
            rf"^Syncing… {re.escape(SOURCE)}/{re.escape(FIRST_REL)}"
            rf" {MAX_COUNT} remaining · chunk \d+/\d+$"
        ),
        timeout=SETTLE_TIMEOUT_MS,
    )

    # The final file's window: the main part names it and NO count part
    # rides along (A1's hidden edge; the chunk part may — the last file
    # is multi-chunk too).
    expect(page.locator("#sync-progress-text")).to_have_text(
        re.compile(
            rf"^Syncing… {re.escape(SOURCE)}/{re.escape(LAST_REL)}"
            r"(?!.* remaining)(?: · chunk \d+/\d+)?$"
        ),
        timeout=SETTLE_TIMEOUT_MS,
    )

    terminal = recorder.stop()
    assert terminal["state"] == "success", terminal

    frames = _frames(page)
    texts = _texts(frames)

    # --- the rendered sequence's shape ---------------------------------
    import_frames = [f for f in frames if f["text"].startswith("Syncing… ")]
    assert import_frames, f"no running import label was ever rendered: {texts[:8]}"
    for frame in import_frames:
        assert IMPORT_LABEL_RE.match(frame["text"]), frame
        # The pre-phase position shape is gone from this part for good.
        assert not OLD_FRACTION_RE.search(frame["text"]), frame
        # A4: the untruncated label rides the title AND the announcer.
        assert frame["title"] == frame["text"], frame
        assert frame["aria"] == frame["text"], frame
        # The count span mirrors the part: visible iff the text carries it.
        assert frame["countHidden"] is not (" remaining" in frame["text"]), frame
        if " remaining" in frame["text"]:
            shown = re.search(r" (\d+) remaining", frame["text"])
            assert shown is not None, frame
            assert 1 <= int(shown.group(1)) <= MAX_COUNT, (
                f"a count-bearing label showed {shown.group(1)} "
                f"(allowed 1…{MAX_COUNT}): {frame}"
            )

    # --- the count-down itself -----------------------------------------
    observed: list[int] = []
    for label in texts:
        m = re.search(r" (\d+) remaining(?: · |$)", label)
        if m:
            value = int(m.group(1))
            if value not in observed:
                observed.append(value)
    assert len(observed) >= 3, (
        f"the label never visibly counted down (distinct N values: {observed})"
    )
    assert observed == sorted(observed, reverse=True), (
        f"the count part did not count DOWN: {observed}"
    )
    assert observed[0] == MAX_COUNT, f"the walk did not start at N − 1: {observed}"
    assert observed[-1] == 1, f"the count part never reached the last file: {observed}"
    assert 0 not in observed, "a '0 remaining' label rendered (A1's hidden edge)"

    # --- the chunk part coexists (phase 129, unbroken) ------------------
    coexist = [
        t for t in texts if re.search(r" \d+ remaining · chunk \d+/\d+$", t)
    ]
    assert coexist, f"the count and chunk parts never shared a label: {texts[:12]}"
    chunk_totals = {
        int(m.group(1))
        for t in texts
        for m in [re.search(r" · chunk \d+/(\d+)$", t)]
        if m
    }
    assert _expected_content_chunks(FIRST_REL) in chunk_totals, (
        f"the chunk part never reported the first file's real chunk count "
        f"({_expected_content_chunks(FIRST_REL)}): {sorted(chunk_totals)}"
    )

    # --- the A1 edge, sequence-wide ------------------------------------
    first_last = next(
        (i for i, t in enumerate(texts) if t.startswith(f"Syncing… {SOURCE}/{LAST_REL}")),
        None,
    )
    assert first_last is not None, f"the final file was never named: {texts[:12]}"
    after = texts[first_last:]
    assert not any(" remaining" in t for t in after), (
        f"the count part came back after the final file: {after[:8]}"
    )

    # --- the untouched phases and the untouched terminal line -----------
    assert any(t == "Writing KB overview…" for t in texts), texts[:20]
    summaries = [t for t in texts if t.startswith("Summarizing folders…")]
    assert summaries and all(OLD_FRACTION_RE.search(t) for t in summaries), (
        f"the summaries phase's folder position changed shape: {summaries}"
    )
    box = page.locator("#sync-progress-text")
    expect(box).to_have_text(counts_line(f"{N_FILES} added"))
    expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL)
    expect(btn).to_be_enabled()
    assert terminal["detail"]["added"] == N_FILES, terminal

    # --- the cross-layer invariant: the label is the payload's subtraction
    # The endpoint is untouched: it still answers with the 1-based
    # position, and each rendered N equals total − done of the file it
    # named.
    position: dict[str, tuple[int, int]] = {}
    for tick in recorder.running_ticks:
        if tick["current_file"]:
            position[tick["current_file"]] = (tick["files_done"], tick["files_total"])
    assert {f"{SOURCE}/{rel}" for rel in REL_ORDER} <= set(position), (
        f"the status never reported every file: {sorted(position)}"
    )
    for frame in import_frames:
        named = frame["text"].split(" · ")[0]
        path = named[len("Syncing… ") :].split(" ")[0]
        done, total = position[path]
        assert total == N_FILES, (path, done, total)
        expected = total - done
        shown = re.search(r" (\d+) remaining", frame["text"])
        if expected > 0:
            assert shown is not None, (
                f"{path}: the payload said {expected} left, the label said none: {frame}"
            )
            assert int(shown.group(1)) == expected, (path, done, total, frame)
        else:
            assert shown is None, f"{path}: done==total but a part rendered: {frame}"


# ---------------------------------------------------------------------------
# 2. (c) The pre-walk edge + the A1 hidden edge, on the exact payloads the
#    guards read
# ---------------------------------------------------------------------------


def test_prewalk_and_final_file_labels_have_no_count_part(
    page: Page, app_url: str, db_ready: None
) -> None:
    """The two edges of A1, pinned on the payloads the guards actually
    read (the served bodies use the status endpoint's unchanged key set —
    what is under test is the client's ONE builder): ``total === 0``
    renders no count part, and neither does ``done === total``."""
    page.set_default_timeout(SETTLE_TIMEOUT_MS)
    stages: dict[str, Any] = {
        "sync": _sync_status(state="running"),  # the pre-walk: no file, 0/0
        "upload": None,
    }
    _serve_statuses(page, stages)
    # settled=False: the boot fetch is the pre-walk stub — the re-attach
    # itself is part of what this arm pins.
    _admin_sources_page(page, app_url, settled=False)
    _install_recorder(page)

    box = page.locator("#sync-progress-text")
    btn = page.locator("#sync-btn")

    # (c) The pre-walk — phase null, current_file null, files_total 0:
    # the bare prefix, no count part, the count span hidden, the title
    # identical (A4).
    expect(box).to_have_text("Syncing…", timeout=SETTLE_TIMEOUT_MS)
    expect(page.locator(".sync-label-count")).to_be_hidden()
    expect(btn).to_have_attribute("title", "Syncing…")
    expect(btn).to_be_disabled()

    # The walk knows the total: the first file counts down from N − 1.
    stages["sync"] = _sync_status(
        state="running",
        phase="import",
        current_file=f"{SOURCE}/{FIRST_REL}",
        files_done=1,
        files_total=N_FILES,
    )
    expect(box).to_have_text(
        f"Syncing… {SOURCE}/{FIRST_REL} {MAX_COUNT} remaining",
        timeout=SETTLE_TIMEOUT_MS,
    )

    # The last count-bearing position, with the phase-129 chunk part
    # beside it (the two parts are independent and coexist).
    stages["sync"] = _sync_status(
        state="running",
        phase="import",
        current_file=f"{SOURCE}/{REL_ORDER[3]}",
        files_done=N_FILES - 1,
        files_total=N_FILES,
        current_chunk=2,
        chunks_total=5,
    )
    expect(box).to_have_text(
        f"Syncing… {SOURCE}/{REL_ORDER[3]} 1 remaining · chunk 2/5",
        timeout=SETTLE_TIMEOUT_MS,
    )

    # (a/A1) The final file: done == total → the part is ABSENT while the
    # main part still names the file (and the chunk part may stay).
    stages["sync"] = _sync_status(
        state="running",
        phase="import",
        current_file=f"{SOURCE}/{LAST_REL}",
        files_done=N_FILES,
        files_total=N_FILES,
        current_chunk=1,
        chunks_total=4,
    )
    expect(box).to_have_text(
        f"Syncing… {SOURCE}/{LAST_REL} · chunk 1/4",
        timeout=SETTLE_TIMEOUT_MS,
    )
    expect(page.locator(".sync-label-count")).to_be_hidden()
    expect(btn).to_have_attribute("title", f"Syncing… {SOURCE}/{LAST_REL} · chunk 1/4")

    # The success terminal: the untouched phase-64 counts line. The
    # stamp is NOW (the phase-154 rule renders a same-day completion as
    # the plain "Synced HH:MM").
    stages["sync"] = _sync_status(
        state="success",
        started_at=_now_iso(),
        finished_at=_now_iso(),
        files_done=N_FILES,
        files_total=N_FILES,
        detail={"added": N_FILES, "updated": 0, "unchanged": 0, "pruned": 0},
    )
    expect(box).to_have_text(counts_line(f"{N_FILES} added"), timeout=SETTLE_TIMEOUT_MS)
    expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
    expect(btn).to_be_enabled()

    # Nothing rendered in this run ever spoke the pre-phase shape or a
    # "0 remaining".
    texts = _texts(_frames(page))
    assert not any(OLD_FRACTION_RE.search(t) for t in texts if t.startswith("Syncing…")), texts
    assert not any("0 remaining" in t for t in texts), texts
    assert "Syncing…" in texts, texts


# ---------------------------------------------------------------------------
# 3. (b) A2 — the upload job shares the builder and the shape
# ---------------------------------------------------------------------------


def test_upload_label_shares_the_count_down_builder(
    page: Page, app_url: str, db_ready: None, upload_archive: Path
) -> None:
    """A2: the upload's running label is the SAME builder's output, so it
    counts down in the SAME shape."""
    page.set_default_timeout(SETTLE_TIMEOUT_MS)

    # The real upload run, against the live endpoint first: since phase
    # 90 it is unpack + register only, so its status carries
    # null/0/0 for the WHOLE run — the reason the shared shape is pinned
    # through a served status below (a real upload can never show a
    # count at all, and can therefore never regress to "(n/m)").
    with httpx.Client(base_url=app_url, timeout=60.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        with upload_archive.open("rb") as fh:
            r = client.post(
                "/api/git-sources/upload",
                files={"file": (f"{UPLOAD_NAME}.tar.gz", fh, "application/gzip")},
            )
        assert r.status_code == 202, r.text
        body: dict[str, Any] = {}
        for _ in range(200):
            body = client.get("/api/git-sources/upload/status").json()
            if body["state"] in ("success", "failed"):
                break
            time.sleep(0.05)
        assert body["state"] == "success", body
        assert body["current_file"] is None, body
        assert body["files_done"] == 0 and body["files_total"] == 0, body

    # The shared shape, on the page's own poll: a running upload status
    # that DOES carry counts (the key set the endpoint already answers
    # with) renders the count-down part through the one builder.
    stages: dict[str, Any] = {
        "sync": _sync_status(state="idle"),
        "upload": _upload_status(
            state="running",
            current_file=f"{UPLOAD_NAME}/docs/01.md",
            files_done=1,
            files_total=3,
        ),
    }
    _serve_statuses(page, stages)
    # settled=False: the boot re-attach of the in-flight UPLOAD run is
    # the entry point this arm pins.
    _admin_sources_page(page, app_url, settled=False)
    _install_recorder(page)

    box = page.locator("#sync-progress-text")
    btn = page.locator("#sync-btn")
    expect(box).to_have_text(
        f"Importing {UPLOAD_NAME}/docs/01.md 2 remaining", timeout=SETTLE_TIMEOUT_MS
    )
    expect(btn).to_have_attribute("title", f"Importing {UPLOAD_NAME}/docs/01.md 2 remaining")
    expect(btn).to_be_disabled()

    # The upload's final file: the same A1 edge through the same builder.
    stages["upload"] = _upload_status(
        state="running",
        current_file=f"{UPLOAD_NAME}/docs/02.md",
        files_done=3,
        files_total=3,
    )
    expect(box).to_have_text(f"Importing {UPLOAD_NAME}/docs/02.md", timeout=SETTLE_TIMEOUT_MS)
    expect(page.locator(".sync-label-count")).to_be_hidden()

    # The upload settle (branch 6): the button returns to "Sync sources"
    # and the box hides — the upload's own unchanged surface.
    stages["upload"] = _upload_status(
        state="success", detail={"message": "uploaded"}, finished_at=_now_iso()
    )
    expect(page.locator("#sync-label")).to_have_text("Sync sources", timeout=SETTLE_TIMEOUT_MS)
    expect(page.locator("#sync-progress")).to_be_hidden(timeout=SETTLE_TIMEOUT_MS)

    texts = _texts(_frames(page))
    importing = [t for t in texts if t.startswith("Importing")]
    assert importing, texts
    for label in importing:
        assert re.fullmatch(
            rf"Importing {re.escape(UPLOAD_NAME)}/docs/\d{{2}}\.md"
            r"(?: [12] remaining)?",
            label,
        ), label
        assert not OLD_FRACTION_RE.search(label), label
        # The upload branch never receives a status object (its endpoint
        # has no phase / chunk / image fields — phase 129 / 152), so the
        # chunk part belongs to the sync arms only.
        assert " · chunk " not in label, label