**Phase 193 — final verification pass: all gates green.**
- **Verified (all 3 tasks already complete in `complete/193_sync_remaining_count/`):** `fmtSyncLabel`'s count part is `total > 0 && total - done > 0 ? \`${total - done} remaining\` : ""` (plain int, hidden at `N==0`, `total===0` → no part); docstrings updated; chunk/image/summaries parts + `counts_line` terminal + `app/` byte-untouched (`git diff --stat`: no `app/` files).
- **Completion criteria:** `N remaining` shape + marker pins ✅ · upload shares the builder (E2E upload arm) ✅ · chunk/image/summaries pins green ✅ · terminal `counts_line` green unedited + A4 title/`#sync-progress` pin ✅ · suite green, coverage 99% ✅ · lint/types clean ✅ · commit + `todo/→complete/` left to the harness (no git index/history commands run).
- `uv run pytest -q --cov=app --cov-report=term-missing` → **exit 0**, junit: **4792 tests, 0 failures/errors/skipped**, `TOTAL 7181 16 → 99%` (>90%)
- `uv run pytest tests/e2e/test_sync_remaining_count.py -v --no-cov` → **3 passed** (isolated, 53 s)
- Isolated `--no-cov` sweep of the 14 sync-lineage suites (label_mobile, upload_progress, chunk_progress, image_progress, progress_box, summary_visibility, archive_upload_sources, status_freshness, upload_no_scan, peak_memory_visible, session_reset, fast_path_deferred_content, git_freeze, smoke) → **14/14 exit 0**
- `uv run ruff check .` → All checks passed · `uv run pyright` → **0 errors**
- **Docs:** README.md reviewed in full (all 2408 lines) + AGENTS.md. Fixed 2 inconsistencies: the sync running-label clause claimed the upload label shows the count — now states it shares the ONE builder/shape but reports no counts since phase 90 (renders bare); phase-190's "the other ~72 rows" → "the other 83 rows of the panel's 86-row catalogue" (mechanically: 86 CATALOG rows, 84 overridable). `N remaining` already in the sync paragraph; task-01's `(n/m)` examples gone. AGENTS.md verified consistent (no clause names the running count part → unchanged per the phase's "do not add if absent"). Mechanically re-verified: all 86 `BOR_*` settings documented + defaults match, `.env.example` 88 vars covered, 0 broken README anchors, all cited file paths exist, 17 palette colors, migrations head `0030`, pages/shell routes, ports 8123/8901, E2E port allocations (new suite's 8211/8914 unique).
- **Notable:** task 02's documented deviation stands — `files_done` is the importer's 1-based *current* file, so the phase doc's `[1-4]` range is unreachable; count-bearing frames are `1 … N−1` (pinned). Pre-existing, untouched: `E2E_APP_PORT_TOOLBUDGET`/`SAVEDOC` both 8129 and `SYNCUP`/`SYCNCEL` both 8133 — harmless under the one-suite-at-a-time rule.
- **Next pending phase:** none — `193_sync_remaining_count` is the last entry in `.agents/phases/todo/`.
3601 lines
163 KiB
Python
3601 lines
163 KiB
Python
"""Unit: the admin "Sync sources" button contract (phase 32, task 02).
|
||
|
||
The browser behavior is E2E-covered (tests/e2e/test_sync_button.py,
|
||
task 03); here we pin the source-level wiring — the anonymous-safe
|
||
ship-hidden button markup, the Sources-page admin reveal on the SAME
|
||
cached whoami (no extra fetch), the sync state machine that now lives
|
||
in sources.js (owner rework 2026-08-28: the button is Sources-page
|
||
only, so the page owns the machine it drives — 2 s poll, 202 start
|
||
/ 409 adoption / 403 hide, terminal labels, the "bor:sync-status"
|
||
event with the status object as detail, the single-poll-loop guard, no
|
||
client-side hard timeout), the page's #sync-progress box (the sole
|
||
aria-live region — phase 150 retired the announcer span) +
|
||
#sync-error-banner, the lazily created error modal (phase 41), and the
|
||
§7.4 never-stale CSS (spin + reduced-motion opt-out,
|
||
disabled state, 44px floor, contrast pair) — so a silent regression is
|
||
caught without a browser.
|
||
|
||
Phase 64 (task 02): unit coverage of the ``app/api/sync.py`` status
|
||
contract — the idle response dict is pinned in full (every pre-existing
|
||
key unchanged, plus ``current_file``/``files_done``/``files_total`` as
|
||
null/0/0); mid-run the status reports the file the (mock) import is
|
||
processing, through the runner's own hook closure (no file yet during
|
||
the clone/pull phase — A4); the terminal states clear
|
||
``current_file`` while keeping the run's final counts.
|
||
|
||
Phase 98 (task 01): the status's phase machine joins the pins — the
|
||
four new keys (``phase`` / ``current_summary`` / ``summaries_done`` /
|
||
``summaries_total``) ride along as null/0/0/0 idle, the prelude keeps
|
||
``phase: null``, the import reports ``phase: "import"``, and the
|
||
terminal states clear ``phase`` + ``current_summary`` while keeping
|
||
the run's final summary counts.
|
||
|
||
Phase 98 (task 02): the running-state title rule is adjusted — the
|
||
FULL untruncated LABEL (the file label, "Writing KB overview…", or
|
||
"Summarizing folders… <folder> (n/m)") rides the button title for
|
||
EVERY running label; the phase-64 "title only when there is a current
|
||
file" (remove-when-null) rule is gone.
|
||
|
||
Phase 128 (task 01): the label becomes PARTS — #sync-label ships a
|
||
main span (the idle text pre-JS) + a hidden count span (the pinned
|
||
"(n/m)"), the builders return { main, count }, and the single writer
|
||
setSyncLabel(main, count, chunk — the chunk part is phase 129) lands
|
||
the parts + the untruncated full label (main + " " + count +
|
||
" " + chunk) on the title + #sync-progress; the running-state /
|
||
settle pins move onto that shape (their labels stay byte-identical
|
||
as full strings). Phase 129 (task 04): #sync-label ships a THIRD
|
||
hidden span (the pinned "· chunk n/m" — the in-flight file's
|
||
embed-batch position), rendered only while a file's embed batch is in
|
||
flight (the import phase after the first batch — hidden in every
|
||
other state); the markup pin below pins the exact three-part ship
|
||
shape.
|
||
|
||
Phase 149 (task 01): the running button becomes the SPINNER — the
|
||
writer gains the `running` flag (the running entry passes `true`,
|
||
the settle the default `false`), which reveals the detail box
|
||
(#sync-progress — the full untruncated label; phase 150: the box
|
||
became the SOLE announcer — the previous live-region span was
|
||
retired) and owns the button's accessible name (the full live label
|
||
while running, the
|
||
ship value "Sync sources" restored on every settle); the
|
||
`is-running` class travels with the spin (added in the running
|
||
entry, removed in the settle). The pins below move onto that shape
|
||
(the aria-label write leaves the running entry for the writer).
|
||
|
||
Phase 129 (task 03): the status contract gains the in-flight file's
|
||
EMBED-chunk position — ``current_chunk`` (the 1-based first chunk of
|
||
the in-flight token-budgeted batch) + ``chunks_total`` (the file's
|
||
total content chunks): the idle dict pins them 0/0, the running
|
||
dict's pins gain them, and the new mid-run pair test reads the
|
||
positions between ticks (1/38 → 13/38) through the runner's own
|
||
``_chunk_hook`` closure; the convention split is pinned — the chunk
|
||
position clears at every point the import phase ends (the
|
||
overview/summaries transitions + both terminals) while the FILE
|
||
counts keep their final values.
|
||
|
||
Phase 134 (task 01): the FIFTH state — ``cancelled`` — joins the
|
||
contract. The admin-only ``POST /api/sync/cancel`` (the 403 is pinned
|
||
in the integration suite — the router's ``require_admin`` dependency,
|
||
no DB/HTTP layer here) is 202 ``cancel requested`` while a run is in
|
||
flight (it sets the module ``_cancel_event`` AND ``_task.cancel()``)
|
||
and 409 ``no sync is running`` idle / after a terminal run. A
|
||
user-requested cancel converts the ``CancelledError`` into the
|
||
``cancelled`` terminal (``error`` None, ``detail`` empty, the final
|
||
file counts kept, the task ends CANCELLED); an UNREQUESTED cancel
|
||
(bare ``task.cancel()`` — the app-shutdown path) re-raises with the
|
||
state untouched (never swallowed — the completion criterion's pin);
|
||
``start_sync`` clears a stale event so the next run completes to
|
||
``success``. The cancel runs here are driven through the REAL
|
||
``start_sync`` on a worker loop — ``sync_api._task`` is the endpoint's
|
||
target — and the cancel is issued FROM that loop thread (a same-thread
|
||
``Task.cancel()`` — a cross-thread cancel would schedule through the
|
||
loop's selector wake, which a parked loop never receives).
|
||
|
||
Phase 134 (task 02): the FRONTEND cancel contract is pinned at the
|
||
source level (the browser behavior is E2E-covered in
|
||
tests/e2e/test_sync_cancel.py, task 03): the ship-hidden
|
||
``#sync-cancel-btn`` markup right after ``#sync-btn`` (the visible
|
||
"Cancel" label is the accessible name — WCAG 2.4), the
|
||
``.sync-cancel-btn`` secondary-pill CSS (the 44px floor, the AA
|
||
ink-soft pair, the guarded hover, the :disabled + ``[hidden]`` rules,
|
||
the full-width mobile rule next to the main button's), the show/hide
|
||
helpers (show = unhide + enable + aria-label, hide = hide + disable),
|
||
the two choke points (``enterSyncRunningState`` shows for kind
|
||
"sync" / hides for kind "upload" — the upload job is NOT cancellable
|
||
in this phase — and ``settleSyncButton`` hides on EVERY settle), the
|
||
``requestCancel`` never-stale lifecycle (disable FIRST, then
|
||
POST /api/sync/cancel; the 403 backstop, the silent 409, the 202
|
||
wait, the phase-55 retry copy + re-enable on any other failure), the
|
||
poll tree's new 5th branch (after failed, before the upload branches
|
||
— the 7-branch tree is now 8), the neutral ``applySyncCancelled``
|
||
(no modal, no banner, no error styling), and the
|
||
``initSyncButton`` re-attach of a ``cancelled`` run.
|
||
|
||
Phase 152 (task 01): the status contract gains the in-flight file's
|
||
VISION-image position — ``images_done`` (the 1-based index of the
|
||
image about to be described) + ``images_total`` (the file's total
|
||
images for this run — the three PDF vision spans share ONE continuous
|
||
counter per file, the importer's ``on_image_progress`` hook that
|
||
task 02/03 wire). The idle dict pins them 0/0; the per-file reset is
|
||
the pairing rule, extended — a ``progress`` call names the file AND
|
||
clears the image position, pinned through the runner's own ``_hook``
|
||
closure (the two-file fake below: the position fired through the
|
||
runner's own ``_image_hook`` closure while file A is in flight is
|
||
0/0 when file B is named — a file with no images must not inherit
|
||
the previous PDF's position); the clear rule mirrors the chunk
|
||
position — 0/0 at the overview/summaries transitions and in EVERY
|
||
terminal (success, failed, and cancelled — each pinned with a live
|
||
position in flight; task 04 moves every live position below onto the
|
||
runner's OWN closures, the phase-64/129 idiom), while the FILE
|
||
counts keep their final values (the convention split: a
|
||
sub-position of a file that's done clears with the file).
|
||
|
||
Phase 152 (task 03): the runner's THIRD paired closure —
|
||
``_image_hook`` (beside ``_hook`` and ``_chunk_hook``) — lands the
|
||
importer's ``on_image_progress`` pairs on the status, and the import
|
||
call now passes ``on_image_progress=_image_hook`` (the wiring pin:
|
||
the fakes receive the kwarg and record the live closure). The
|
||
frontend gains the "· image n/m" label PART: the running entry
|
||
destructures ``image = ""`` and the writer call becomes
|
||
``setSyncLabel(main, count, chunk, true, image)`` — the image part
|
||
rides ONLY the composed full string (box + title + running
|
||
aria-label — no new button span; the running button is the
|
||
spinner), the chunk part keeps its own span, and the bare builders
|
||
+ non-running calls pass nothing (both parts default "").
|
||
|
||
Phase 152 (task 04): the position pins move onto the runner's OWN
|
||
closures (the phase-64/129 idiom — the fakes fire the image pairs
|
||
through the ``on_image_progress`` kwarg, the ``_image_hook`` under
|
||
test, instead of standing in on the status object): a mid-run pair
|
||
(3/15, then 15/15) climbs on the status BETWEEN ticks while
|
||
``current_file`` and the chunk position keep their own values (the
|
||
THREE positions are independent sub-positions of the file), the
|
||
per-file reset, the overview/summaries transition clears, and the
|
||
FAILED + CANCELLED terminals all fire their live positions through
|
||
the closure too. The frontend source pins the ``fmtSyncPhaseLabel``
|
||
image part (present iff ``images_total > 0`` — the exact guard +
|
||
template), the ``overview`` / ``summaries`` branches' ``image: ""``
|
||
(unchanged labels), and ``setSyncLabel``'s composed ``full`` string
|
||
byte-for-byte (a status carrying both positions reads "… (2540/2540)
|
||
· chunk 125/126 · image 3/15" on the box + title + running
|
||
aria-label — the runtime string is behaviorally pinned in
|
||
tests/unit/test_frontend_sync_upload.py's node check).
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import asyncio
|
||
import json
|
||
import logging
|
||
import re
|
||
import shutil
|
||
import subprocess
|
||
import threading
|
||
import time
|
||
from collections.abc import Callable, Iterator, Sequence
|
||
from datetime import datetime
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import pytest
|
||
from fastapi import HTTPException
|
||
|
||
from app.api import sync as sync_api
|
||
from app.config import Settings
|
||
from app.models import GitSource
|
||
from app.rag.importer import ImportSummary
|
||
from app.rag.llm import EmbeddingError
|
||
|
||
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||
ASSETS = FRONTEND / "assets"
|
||
|
||
HEADER_JS = ASSETS / "header.js"
|
||
SOURCES_JS = ASSETS / "sources.js"
|
||
STYLES_CSS = ASSETS / "styles.css"
|
||
# Phase 76 (task 02): sources.html is folded into the ONE-document shell
|
||
# — the sync markup now lives in the RAG view section of index.html.
|
||
SHELL_HTML = FRONTEND / "index.html"
|
||
# Phase 150: the retired sync-announcer span's id/class are assembled
|
||
# (never spelled out) so the phase gate's project-wide grep for the
|
||
# retired name stays clean while the absence pins below stay in force.
|
||
RETIRED_ID = "-".join(("sync", "result")) # built at runtime: 3.12+ folds
|
||
# adjacent/"+" constant strings at compile time, which would leave the
|
||
# retired name in the .pyc — the phase gate greps even the bytecode.
|
||
|
||
HAVE_NODE = shutil.which("node") is not None
|
||
|
||
#: Phase 187 (task 02): what :func:`app.core.memory.fmt_mb` may render for
|
||
#: an MB field — ``?`` when the platform cannot know, else a positive int.
|
||
MB = r"(?:\?|[1-9][0-9]*)"
|
||
|
||
|
||
def _text(path: Path) -> str:
|
||
assert path.is_file(), f"missing frontend file: {path}"
|
||
return path.read_text(encoding="utf-8")
|
||
|
||
|
||
def _rag_view(html: str) -> str:
|
||
"""The RAG view section of the shell (view-scoped page-sub scope —
|
||
the shell carries one .page-sub per view, so a whole-file match
|
||
would hit the earlier views first)."""
|
||
i = html.find('<section class="view" id="view-rag"')
|
||
assert i != -1, "the RAG view section must be in the shell"
|
||
j = html.find('<section class="view" id="view-git-sources"', i)
|
||
assert j != -1, "the Sources view section must follow the RAG view"
|
||
return html[i:j]
|
||
|
||
|
||
def _body(js: str, fn_name: str) -> str:
|
||
"""The source of the first `function <fn_name>` in js (brace
|
||
balanced — since phase 76 task 02 the functions live inside
|
||
mount(root), so the closing brace is indented, not column 0)."""
|
||
fn = js.find(f"function {fn_name}")
|
||
assert fn != -1, f"{fn_name} must be defined"
|
||
open_idx = js.find("{", fn)
|
||
depth = 0
|
||
for i in range(open_idx, len(js)):
|
||
c = js[i]
|
||
if c == "{":
|
||
depth += 1
|
||
elif c == "}":
|
||
depth -= 1
|
||
if depth == 0:
|
||
return js[fn : i + 1]
|
||
raise AssertionError(f"unbalanced braces in {fn_name}")
|
||
|
||
|
||
# ---------- the shell's RAG view: anonymous-safe ship-hidden markup ----------
|
||
|
||
|
||
def test_sync_button_ships_hidden_and_labeled() -> None:
|
||
"""#sync-btn SHIPS with the hidden attribute (anonymous-safe —
|
||
sources.js reveals it for the admin at view boot), is a real
|
||
<button type="button">, and carries aria-label="Sync sources" so
|
||
the accessible name stays stable across the label states."""
|
||
tag = re.search(r"<button[^>]*id=\"sync-btn\"[^>]*>", _text(SHELL_HTML))
|
||
assert tag, "the shell must carry the #sync-btn button (RAG view)"
|
||
attrs = tag.group(0)
|
||
assert 'class="sync-btn"' in attrs
|
||
assert 'type="button"' in attrs
|
||
assert re.search(r"\bhidden\b", attrs), "#sync-btn must ship hidden"
|
||
assert 'aria-label="Sync sources"' in attrs
|
||
|
||
|
||
def test_sync_button_has_icon_and_label_parts() -> None:
|
||
"""The button body is a refresh-cycle svg (aria-hidden — decorative,
|
||
the spin is the visible running state) + the .sync-label span with
|
||
the idle text, so the label can be swapped by sources.js.
|
||
|
||
Phase 128 (task 01): #sync-label is PARTS — the MAIN span
|
||
(.sync-label-main) carries the idle text (the pre-JS label renders
|
||
from the static markup; the part CSS may ellipsize it), the
|
||
COUNT span (.sync-label-count — the pinned phase-193
|
||
"N remaining" position, never clipped) ships hidden + empty
|
||
(there is no count at idle),
|
||
and — phase 129 (task 04) — the CHUNK span (.sync-label-chunk —
|
||
the pinned "· chunk n/m" embed-batch position) ships hidden +
|
||
empty too (no chunk position at idle — the part renders only
|
||
while a file's embed batch is in flight). The class/id stay (the
|
||
module query + the E2E label assertions)."""
|
||
text = _text(SHELL_HTML)
|
||
btn = text[text.find('id="sync-btn"') : text.find("</button>", text.find('id="sync-btn"'))]
|
||
assert re.search(r'<svg[^>]*class="sync-icon"[^>]*aria-hidden="true"', btn)
|
||
# Phase 128 (task 01) + Phase 129 (task 04): the parts markup —
|
||
# the exact three-part shape (the main span holds the idle text,
|
||
# the count + chunk spans are hidden + empty).
|
||
label = re.search(
|
||
r'<span\b[^>]*id="sync-label"[^>]*>'
|
||
r'<span class="sync-label-main">Sync sources</span>'
|
||
r'<span class="sync-label-count" hidden></span>'
|
||
r'<span class="sync-label-chunk" hidden></span>'
|
||
r"</span>",
|
||
btn,
|
||
)
|
||
assert label, (
|
||
"the #sync-label parts markup (main + hidden count + hidden chunk) is missing"
|
||
)
|
||
assert 'class="sync-label"' in label.group(0)
|
||
|
||
|
||
def test_sync_progress_is_the_aria_live_announcer() -> None:
|
||
"""#sync-progress (the sync detail box) sits right after the button
|
||
row and is a polite live region (role="status" +
|
||
aria-live="polite") — phase 150: it is the SOLE aria-live
|
||
announcer for the sync area (the live label while a run is in
|
||
flight, the settle counts line after it), and the retired
|
||
announcer span is gone (one live region, no double announce)."""
|
||
text = _text(SHELL_HTML)
|
||
tag = re.search(r'<div[^>]*id="sync-progress"[^>]*>', text)
|
||
assert tag, "the shell must carry the #sync-progress announcer box (RAG view)"
|
||
attrs = tag.group(0)
|
||
assert 'role="status"' in attrs
|
||
assert 'aria-live="polite"' in attrs
|
||
assert 'aria-hidden' not in attrs, "an aria-live region must not be aria-hidden"
|
||
assert -1 < text.find("</button>", text.find('id="sync-btn"')) < text.find('id="sync-progress"')
|
||
assert f'id="{RETIRED_ID}"' not in text, (
|
||
"the retired announcer span is gone from the shell (phase 150)"
|
||
)
|
||
|
||
|
||
def test_sync_error_banner_is_a_hidden_alert() -> None:
|
||
"""The failure banner uses the chat error-banner markup style
|
||
(kb-banner + is-error) and role="alert", shipping hidden —
|
||
sources.js un-hides it with the error text on a failed run."""
|
||
text = _text(SHELL_HTML)
|
||
tag = re.search(r'<div[^>]*id="sync-error-banner"[^>]*>', text)
|
||
assert tag, "the shell must carry the #sync-error-banner (RAG view)"
|
||
attrs = tag.group(0)
|
||
assert "kb-banner" in attrs and "is-error" in attrs
|
||
assert 'role="alert"' in attrs
|
||
assert re.search(r"\bhidden\b", attrs)
|
||
assert 'id="sync-error-text"' in text
|
||
# The banner lives in the page content, not the 64px header bar.
|
||
assert text.find('id="sync-error-banner"') > text.find('<main id="main"')
|
||
|
||
|
||
def test_page_sub_copy_mentions_the_button() -> None:
|
||
"""The page-sub copy names the button as the one-click way to pull
|
||
the latest and re-import (the import CLI docs live elsewhere).
|
||
Phase 61: the copy describes the current source model (git repos +
|
||
local directories + uploaded archives), not the old ~/Homelab +
|
||
~/Deployments clone. Phase 76 (task 02): scoped to the RAG view —
|
||
the shell carries one .page-sub per view."""
|
||
sub = re.search(r'<p class="page-sub">(.*?)</p>', _rag_view(_text(SHELL_HTML)), re.DOTALL)
|
||
assert sub, "the RAG view must keep the .page-sub copy"
|
||
copy = re.sub(r"\s+", " ", sub.group(1)) # the markup wraps lines
|
||
assert "Press <strong>Sync sources</strong>" in copy
|
||
assert "pull the latest and re-import" in copy
|
||
assert "git repositories," in copy
|
||
assert "local directories, and uploaded archives" in copy
|
||
|
||
|
||
def test_sources_page_stays_cdn_free() -> None:
|
||
"""No-CDN rule (PLAN §7.3, A11): the new button markup adds no
|
||
external references — same-origin assets only (the integration
|
||
test_index_html_served_locally re-checks this on the served page)."""
|
||
text = _text(SHELL_HTML)
|
||
assert 'src="https://' not in text
|
||
assert 'href="https://' not in text
|
||
|
||
|
||
# ---------- sources.js: the admin reveal (view boot) ----------
|
||
|
||
|
||
def test_sources_js_reveals_sync_btn_on_the_admin_branch() -> None:
|
||
"""The view boot (mount's tail, phase 76 task 02) reveals #sync-btn
|
||
for the admin on the SAME cached whoami fetchIsAdmin() reads (no
|
||
extra fetch — header.js keeps the single /api/whoami call site; the
|
||
header itself is booted exactly once, by the chat module at shell
|
||
boot); anonymous users never leave the ship-hidden default."""
|
||
js = _text(SOURCES_JS)
|
||
boot = js[js.find("view boot (phase 76 task 02)"):]
|
||
assert "const admin = await fetchIsAdmin()" in boot
|
||
assert "syncBtn.hidden = !admin" in boot, "#sync-btn must join the admin reveal"
|
||
# The reveal must not introduce a second whoami call site.
|
||
header = _text(HEADER_JS)
|
||
assert header.count('fetch("/api/whoami")') == 1
|
||
assert 'fetch("/api/whoami")' not in js
|
||
|
||
|
||
# ---------- sources.js: the sync state machine (moved here from
|
||
# ---------- header.js in the owner rework 2026-08-28) ----------
|
||
|
||
|
||
def test_sources_js_owns_the_sync_button_elements() -> None:
|
||
"""The button refs are module-level and null-safe: #sync-btn, the
|
||
.sync-label inside the button, the .sync-icon inside the button — a
|
||
page without the markup is a complete no-op."""
|
||
js = _text(SOURCES_JS)
|
||
assert 'querySelector("#sync-btn")' in js
|
||
assert 'syncBtn.querySelector(".sync-label")' in js
|
||
assert 'syncBtn.querySelector(".sync-icon")' in js
|
||
|
||
|
||
def test_sources_js_calls_the_sync_api() -> None:
|
||
"""The click posts to POST /api/sync and the poll loop GETs
|
||
/api/sync/status — both through the same-origin API (A10)."""
|
||
js = _text(SOURCES_JS)
|
||
assert 'fetch("/api/sync", { method: "POST" })' in js
|
||
assert 'fetch("/api/sync/status")' in js
|
||
|
||
|
||
def test_sources_js_polls_every_2000ms() -> None:
|
||
"""The feedback loop is a 2000 ms poll of the status endpoint,
|
||
re-scheduled one tick at a time (setTimeout, not setInterval — an
|
||
in-flight fetch can never overlap the next tick)."""
|
||
js = _text(SOURCES_JS)
|
||
assert "SYNC_POLL_MS = 2000" in js
|
||
assert "setTimeout(tick, SYNC_POLL_MS)" in js
|
||
assert "setInterval" not in js
|
||
|
||
|
||
def test_sources_js_adopts_409_and_starts_on_202() -> None:
|
||
"""202 (started) and 409 (a run started elsewhere — e.g. a second
|
||
tab) both enter the running state and start polling: the UI never
|
||
starts a second run, it adopts the in-flight one.
|
||
|
||
Phase 64 (task 04): the entry is the sync-kind live label with no
|
||
file yet (the run is just starting — bare "Syncing…", A4)."""
|
||
js = _text(SOURCES_JS)
|
||
assert "r.status === 202 || r.status === 409" in js
|
||
idx = js.find("r.status === 202 || r.status === 409")
|
||
branch = js[idx : idx + 600]
|
||
assert "enterSyncRunningState(\"sync\", null, 0, 0)" in branch
|
||
assert "startSyncPolling()" in branch
|
||
|
||
|
||
def test_sources_js_hides_the_button_on_403() -> None:
|
||
"""A 403 anywhere (POST, the status poll, the load re-attach) is
|
||
treated as not-admin: the button hides — defense in depth behind
|
||
the whoami reveal (the primary gate)."""
|
||
js = _text(SOURCES_JS)
|
||
assert len(re.findall(r"r\.status === 403", js)) >= 3, (
|
||
"POST, the status poll, and the load re-attach must all handle 403"
|
||
)
|
||
assert js.count("syncBtn.hidden = true") >= 3, (
|
||
"every 403 branch must hide the button"
|
||
)
|
||
|
||
|
||
def test_sources_js_running_state_is_never_stale() -> None:
|
||
"""Entering the running state disables the button, sets aria-busy,
|
||
spins the icon, and swaps the label to the live file label
|
||
(fmtSyncLabel — the §7.4 feedback while the poll waits) — and a
|
||
fresh run starts clean: the previous failure's title / aria-label /
|
||
.is-error come off NOW, not when the run settles.
|
||
|
||
Phase 64 (task 04): the aria-live announcer carries the FULL
|
||
untruncated label (phase 150: that is the #sync-progress box —
|
||
the sole live region). Phase 98 (task 02): the title rule is
|
||
adjusted — the FULL untruncated LABEL (the file label, the overview
|
||
label, or the summaries label — whatever the running form is) rides
|
||
the button title for EVERY running label; the "title only when
|
||
there is a current file" (remove-when-null) rule is gone.
|
||
Phase 128 (task 01): the entry hands the label PARTS (the
|
||
phase-aware split — sync + status → fmtSyncPhaseLabel, the
|
||
fall-through → the byte-identical fmtSyncLabel) to the SINGLE
|
||
writer, setSyncLabel(main, count, chunk, running, image — phase
|
||
129: the chunk part destructures with a "" default, phase 152:
|
||
the image part does too), which lands the main part,
|
||
the pinned count part (hidden when absent), the pinned chunk part
|
||
(phase 129 — hidden when absent), and the FULL untruncated label
|
||
on the button title + #sync-progress (A4 — phase 152: with the
|
||
image part appended after the chunk part when a PDF's vision span
|
||
is in flight; the image part gets no span of its own) — the
|
||
direct span / title / announcer writes are GONE from the entry
|
||
(they live in setSyncLabel + setSyncProgress). Phase 149 (task 01): the writer call passes
|
||
`running = true` (the detail box reveals with the full label),
|
||
and the button's accessible name rides in the writer too (the
|
||
full live label — the icon-only running button keeps a name);
|
||
the entry no longer sets aria-label at all. The `is-running`
|
||
class is added here, with the spin."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "enterSyncRunningState")
|
||
assert "syncBtn.disabled = true" in body
|
||
assert 'syncBtn.setAttribute("aria-busy", "true")' in body
|
||
# the parts split (phase 98) is intact…
|
||
assert "fmtSyncPhaseLabel(status)" in body
|
||
assert "fmtSyncLabel(kind, currentFile, done, total)" in body
|
||
# …and the single writer lands the parts (A4 in it — the title is
|
||
# set for EVERY running label: the file-only rule is adjusted
|
||
# away) with the phase-149 `running` flag (the box reveals) +
|
||
# the phase-152 image part (rides only the composed full string —
|
||
# no span).
|
||
assert "setSyncLabel(main, count, chunk, true, image)" in body
|
||
assert body.count("setSyncLabel(") == 1, "the entry writes ONLY through the parts writer"
|
||
assert "syncBtn.title =" not in body, ("the title write lives in setSyncLabel (A4)")
|
||
assert "syncLabel.textContent" not in body, ("no direct span write — the parts")
|
||
assert "syncProgress" not in body, (
|
||
"the box write lives in setSyncLabel/setSyncProgress (phase 150)"
|
||
)
|
||
assert 'syncBtn.setAttribute("aria-label"' not in body, (
|
||
"the aria-label write lives in setSyncLabel (phase 149: the full"
|
||
" live label while running)"
|
||
)
|
||
assert "syncBtn.classList.remove(\"is-error\")" in body
|
||
assert "syncIcon.classList.add(\"is-spinning\")" in body
|
||
assert "syncBtn.classList.add(\"is-running\")" in body, (
|
||
"the icon-only running button class travels with the spin"
|
||
)
|
||
|
||
|
||
def test_fmt_sync_phase_label_image_part_is_pinned_source_level() -> None:
|
||
"""Phase 152 (task 04): the frontend SOURCE pins the "· image
|
||
n/m" label part (the browser behavior is pinned in
|
||
tests/unit/test_frontend_sync_upload.py's node check — the REAL
|
||
extracted functions run against stub refs; this pin keeps the
|
||
template itself from drifting without a browser):
|
||
|
||
* the import fall-through is the ONLY reader of the status's
|
||
image fields, guarded on ``images_total > 0`` (the part is
|
||
present IFF ``images_total > 0`` — 0/0 hides it, the exact
|
||
guard + template + empty false-branch);
|
||
* the ``overview`` / ``summaries`` branches return ``image: ""``
|
||
(the position belongs to the import phase only — those labels
|
||
stay unchanged by the phase);
|
||
* the running entry destructures the part with a `""` default and
|
||
hands it to the single writer as the FIFTH argument (after the
|
||
`running` flag);
|
||
* ``setSyncLabel``'s composed ``full`` string appends the image
|
||
part LAST (after the chunk part) — so a status carrying both
|
||
positions reads exactly "Syncing… <file> 1 remaining · chunk
|
||
125/126 · image 3/15" on the box, the title, and the running
|
||
aria-label (byte-for-byte; the docstring's worked example
|
||
carries the shape).
|
||
"""
|
||
js = _text(SOURCES_JS)
|
||
builder = _body(js, "fmtSyncPhaseLabel")
|
||
# The import fall-through: present IFF images_total > 0 — the
|
||
# exact guard + template (the phase-129 chunk part's sibling,
|
||
# same shape).
|
||
assert (
|
||
"status.images_total > 0\n"
|
||
" ? `· image ${status.images_done}/${status.images_total}`\n"
|
||
" : \"\";"
|
||
) in builder
|
||
assert "return { main, count, chunk, image };" in builder
|
||
# The overview branch: the unchanged label + the part absent.
|
||
assert (
|
||
'return { main: "Writing KB overview…", count: "", chunk: "", image: "" };'
|
||
) in builder
|
||
# The summaries branch: the unchanged label + the part absent
|
||
# (the phase's own (n/m) is its position).
|
||
assert "image: \"\", // the image position belongs to the import phase only" in builder
|
||
assert "`Summarizing folders…${folder}`" in builder
|
||
# The running entry: the part destructures with a `""` default
|
||
# and rides the writer call as the fifth argument.
|
||
entry = _body(js, "enterSyncRunningState")
|
||
assert 'const { main, count, chunk = "", image = "" }' in entry
|
||
assert "setSyncLabel(main, count, chunk, true, image)" in entry
|
||
# The writer: the composed full string — the image part LAST,
|
||
# after the chunk part (the "… · chunk 125/126 · image 3/15"
|
||
# shape the docstring's worked example carries).
|
||
writer = _body(js, "setSyncLabel")
|
||
assert (
|
||
"const full =\n"
|
||
" mainText +\n"
|
||
" (countText ? ` ${countText}` : \"\") +\n"
|
||
" (chunkText ? ` ${chunkText}` : \"\") +\n"
|
||
" (imageText ? ` ${imageText}` : \"\");"
|
||
) in writer
|
||
assert "· chunk 125/126 · image 3/15" in js, (
|
||
"the worked example (both positions at once) is documented"
|
||
)
|
||
|
||
|
||
def test_sources_js_terminal_states() -> None:
|
||
"""Terminal rendering: success → enabled + 'Synced HH:MM' (local
|
||
time of finished_at); failed → enabled + retry-ready 'Sync sources'
|
||
label + the sanitized error in the button's title + aria-label +
|
||
the .is-error class. The counts formatting (fmtSyncResult) lives
|
||
here ('added' always announced, zero terms omitted — a no-op
|
||
re-sync reads '0 added · 1 unchanged')."""
|
||
js = _text(SOURCES_JS)
|
||
|
||
success = _body(js, "applySyncSuccess")
|
||
assert '"Synced"' in success and "fmtSyncTime(status.finished_at)" in success
|
||
assert 'settleSyncButton(time ? `Synced ${time}` : "Synced")' in success, (
|
||
"the settle string — phase 154: the qualified stamp rides the SAME"
|
||
" shape (the qualifier lives inside the formatter)"
|
||
)
|
||
|
||
failure = _body(js, "applySyncFailure")
|
||
assert 'settleSyncButton("Sync sources")' in failure # retry-ready
|
||
assert "syncBtn.title = error" in failure
|
||
assert 'syncBtn.setAttribute("aria-label", error)' in failure
|
||
assert "syncBtn.classList.add(\"is-error\")" in failure
|
||
assert "sanitizeSyncError(status.error)" in failure
|
||
|
||
result = _body(js, "fmtSyncResult")
|
||
assert "added" in result and "unchanged" in result
|
||
assert " · " in result
|
||
assert "> 0" in result, "zero terms must be omitted"
|
||
|
||
time = _body(js, "fmtSyncTime")
|
||
assert "getHours()" in time and "getMinutes()" in time, "local HH:MM of finished_at"
|
||
assert 'if (Number.isNaN(d.getTime())) return "";' in time, (
|
||
"the parse guard is UNCHANGED — unparseable ISO → "" (the bare"
|
||
" 'Synced' settle)"
|
||
)
|
||
# Phase 154 (task 01): the DATE-QUALIFIER (A1 rule — the LOCAL
|
||
# calendar dates, the page's clock; the table dates' family).
|
||
assert "const now = new Date();" in time, "'now' is the page's clock"
|
||
assert (
|
||
"d.getFullYear() === now.getFullYear() &&" in time
|
||
and "d.getMonth() === now.getMonth()" in time
|
||
and "d.getDate() === now.getDate()" in time
|
||
), "same local calendar day as now → HH:MM (byte-identical to pre-phase)"
|
||
assert "d.getHours() % 12 || 12" in time, (
|
||
"12-hour WITHOUT leading zero on the hour (the 12 o'clock edge)"
|
||
)
|
||
assert '"AM"' in time and '"PM"' in time, "the AM/PM family"
|
||
assert "pad(d.getMinutes())" in time, "MM zero-padded in every form"
|
||
assert "`${d.getMonth() + 1}/${d.getDate()}`" in time, (
|
||
"M/D, no leading zeros (same year, different day)"
|
||
)
|
||
assert "`${d.getMonth() + 1}/${d.getDate()}/${d.getFullYear()}`" in time, (
|
||
"M/D/YYYY across years"
|
||
)
|
||
|
||
|
||
# Phase 154 (task 01) — the date-qualifier's boundary set, behavioral
|
||
# under node with a FIXED clock. The test body never calls `new
|
||
# Date()` for its own timing: the fake clock is installed in the
|
||
# driver (global Date patched BEFORE the call — `new Date()` → the
|
||
# fixed now, `new Date(iso)` → the real parse), and every ISO is
|
||
# designator-less (local) so the comparison is timezone-independent.
|
||
# The fixed now is Oct 3, 2026, 18:30 LOCAL.
|
||
|
||
_STAMP_NOW_JS = "new RealDate(2026, 9, 3, 18, 30, 0)"
|
||
|
||
|
||
def _stamp_driver(fn_src: str, expr: str) -> str:
|
||
"""Run `expr` under node with the REAL sources.js function + the
|
||
fake clock (the idiom: the Date patch precedes the call)."""
|
||
driver = (
|
||
f"{fn_src}\n"
|
||
"const RealDate = Date;\n"
|
||
f"const NOW = {_STAMP_NOW_JS};\n"
|
||
"globalThis.Date = function (...args) {\n"
|
||
" return args.length === 0 ? NOW : new RealDate(...args);\n"
|
||
"};\n"
|
||
f"{expr}\n"
|
||
)
|
||
proc = subprocess.run(
|
||
["node", "-e", driver], capture_output=True, text=True, timeout=60
|
||
)
|
||
assert proc.returncode == 0, f"node failed: {proc.stderr}"
|
||
return proc.stdout
|
||
|
||
|
||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||
def test_sources_js_fmt_sync_time_qualifier_boundaries_fixed_clock() -> None:
|
||
"""The REAL sources.js ``fmtSyncTime`` under node (fake clock, the
|
||
fixed now Oct 3 2026 18:30 local) — the A1 boundary set:
|
||
same-day → ``HH:MM`` (byte-identical to the pre-phase output —
|
||
the audit's own ``16:06`` case, plus the midnight / end-of-day
|
||
edges); yesterday (same year) → ``M/D, h:MM AM/PM`` (no leading
|
||
zero on the hour — ``3:14 PM``; the ``12:05 AM`` / ``12:07 PM``
|
||
o'clock edges); last-year same M/D → ``M/D/YYYY, h:MM AM/PM``;
|
||
garbage / missing ISO → ``""`` (the guard unchanged).
|
||
The ``null`` edge (``new Date(null)`` IS the epoch — parseable,
|
||
the guard unchanged) is pinned TZ-independently: ``null`` and the
|
||
epoch ISO render identically (same instant), in the dated + year
|
||
form (1969 ≠ the fixed 2026). The wall clock never appears in an
|
||
assertion."""
|
||
fn = _body(_text(SOURCES_JS), "fmtSyncTime")
|
||
# (iso, expected) — the expected strings are the pins.
|
||
cases = [
|
||
("2026-10-03T16:06:00", "16:06"), # same day — the audit case
|
||
("2026-10-03T00:05:00", "00:05"), # same day, midnight edge
|
||
("2026-10-03T23:59:00", "23:59"), # same day, end-of-day edge
|
||
("2026-10-02T15:14:00", "10/2, 3:14 PM"), # yesterday, no zero hour
|
||
("2026-10-02T00:05:00", "10/2, 12:05 AM"), # yesterday, 12:05 AM edge
|
||
("2026-10-02T12:07:00", "10/2, 12:07 PM"), # yesterday, noon edge
|
||
("2026-10-02T23:58:00", "10/2, 11:58 PM"), # yesterday, PM minutes
|
||
("2025-10-03T21:12:00", "10/3/2025, 9:12 PM"), # last year, same M/D
|
||
("2025-01-03T09:12:00", "1/3/2025, 9:12 AM"), # last year, month 1
|
||
("garbage", ""), # unparseable → the pre-phase fallback
|
||
("", ""), # empty → the pre-phase fallback
|
||
("undefined", ""), # missing ISO (the app-restart case) → ""
|
||
]
|
||
iso_literals = []
|
||
for iso, _ in cases:
|
||
if iso == "undefined":
|
||
iso_literals.append("undefined")
|
||
else:
|
||
iso_literals.append(f'"{iso}"')
|
||
driver_expr = (
|
||
"const CASES = [" + ", ".join(iso_literals) + "];\n"
|
||
"const STAMPS = CASES.map((c) => fmtSyncTime(c));\n"
|
||
"const EPOCH = [fmtSyncTime(null), fmtSyncTime(new RealDate(0).toISOString())];\n"
|
||
"console.log(JSON.stringify({ STAMPS, EPOCH }));"
|
||
)
|
||
out = json.loads(_stamp_driver(fn, driver_expr))
|
||
assert out["STAMPS"] == [expected for _, expected in cases], out
|
||
assert out["EPOCH"][0] == out["EPOCH"][1], (
|
||
"null and the epoch ISO render identically (the same instant —"
|
||
" the guard is unchanged: new Date(null) IS the epoch)"
|
||
)
|
||
assert re.fullmatch(
|
||
r"\d{1,2}/\d{1,2}/\d{4}, \d{1,2}:\d{2} (AM|PM)", out["EPOCH"][0]
|
||
), "the epoch takes the M/D/YYYY, h:MM AM/PM form (1969 ≠ the fixed year)"
|
||
|
||
|
||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||
def test_sources_js_synced_settle_string_qualifies_the_stamp() -> None:
|
||
"""The ``applySyncSuccess`` settle string (the pinned shape
|
||
``time ? `Synced ${time}` : "Synced"`` — the qualifier lives
|
||
INSIDE the formatter, the shape is unchanged) under node with the
|
||
fixed clock: a same-day ``finished_at`` settles as
|
||
``Synced 16:06`` (byte-identical to pre-phase), a non-same-day
|
||
one settles as ``Synced 10/2, 3:14 PM`` (the date-qualified
|
||
badge — the phase's ask), and an unparseable/missing ISO settles
|
||
as the bare ``Synced`` (the existing fallback). The ``null`` edge
|
||
pins identically to the boundary test (the epoch — the pre-phase
|
||
rendered ``Synced 00:00`` there)."""
|
||
fn = _body(_text(SOURCES_JS), "fmtSyncTime")
|
||
settle = (
|
||
"const settle = (iso) => {\n"
|
||
" const t = fmtSyncTime(iso);\n"
|
||
" return t ? `Synced ${t}` : \"Synced\";\n"
|
||
"};\n"
|
||
)
|
||
iso_literals = [
|
||
'"2026-10-03T16:06:00"', # same day
|
||
'"2026-10-02T15:14:00"', # yesterday
|
||
'"2025-10-03T21:12:00"', # last year
|
||
'"garbage"', # unparseable
|
||
"undefined", # missing ISO
|
||
]
|
||
driver_expr = (
|
||
settle
|
||
+ "const CASES = [" + ", ".join(iso_literals) + "];\n"
|
||
+ "const SETTLES = CASES.map(settle);\n"
|
||
+ "const EPOCH = [settle(null), settle(new RealDate(0).toISOString())];\n"
|
||
+ "console.log(JSON.stringify({ SETTLES, EPOCH }));"
|
||
)
|
||
out = json.loads(_stamp_driver(fn, driver_expr))
|
||
assert out["SETTLES"] == [
|
||
"Synced 16:06", # same day — byte-identical to pre-phase
|
||
"Synced 10/2, 3:14 PM", # yesterday — date-qualified
|
||
"Synced 10/3/2025, 9:12 PM", # last year — date + year
|
||
"Synced", # garbage — the bare fallback
|
||
"Synced", # missing ISO — the bare fallback
|
||
], out
|
||
assert out["EPOCH"][0] == out["EPOCH"][1], (
|
||
"settle(null) and the epoch ISO settle identically (the guard"
|
||
" unchanged)"
|
||
)
|
||
assert out["EPOCH"][0].startswith("Synced "), (
|
||
"the epoch settles with a stamp (new Date(null) is parseable"
|
||
" — the pre-phase rendered 'Synced 00:00' there)"
|
||
)
|
||
|
||
|
||
# Phase 154 (task 03) — the qualifier's BYTE-IDENTITY: the pre-phase
|
||
# fmtSyncTime, pinned VERBATIM from git history (the phase-154
|
||
# pre-state — the time-only formatter). Only the function NAME is
|
||
# renamed (so the real + the reference run side by side under node);
|
||
# the body is the old code byte-for-byte — the old strings (the
|
||
# `HH:MM` return + the `""` guard) live in this constant, so the
|
||
# byte-identity below is assertable, not assumed. The reference is
|
||
# clock-free (it reads no `now` — only the ISO's local time), but
|
||
# the same fixed-clock driver (the task-01 boundary set's clock) is
|
||
# used, so the real function's same-day comparison runs on the fixed
|
||
# now exactly as in the boundary test.
|
||
PRE_PHASE_FMT_SYNC_TIME_JS = """
|
||
function prePhaseSyncTime(iso) {
|
||
const d = new Date(iso);
|
||
if (Number.isNaN(d.getTime())) return "";
|
||
const pad = (n) => String(n).padStart(2, "0");
|
||
return `${pad(d.getHours())}:${pad(d.getMinutes())}`;
|
||
}
|
||
"""
|
||
|
||
|
||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||
def test_sources_js_fmt_sync_time_same_day_is_byte_identical_to_pre_phase() -> None:
|
||
"""Phase 154 (task 03): the REAL sources.js ``fmtSyncTime`` and
|
||
the VERBATIM pre-phase formatter (pinned above) run side by side
|
||
under node with the fixed clock (Oct 3 2026 18:30 local — the
|
||
task-01 boundary set's clock):
|
||
|
||
* the pinned reference's own output is asserted against the
|
||
verbatim old strings FIRST (a self-check on the constant — a
|
||
typo in the pinned function fails here, not silently);
|
||
* EVERY same-day ISO renders BYTE-IDENTICALLY to the pre-phase
|
||
output (the old ``HH:MM`` — the audit's ``16:06`` case, the
|
||
midnight / end-of-day edges), and its settle string is
|
||
byte-identical to the pre-phase settle (``"Synced " + HH:MM``
|
||
— a same-day badge never moved);
|
||
* EVERY unparseable/missing ISO (``garbage`` / ``""`` /
|
||
``undefined``) renders byte-identically to the pre-phase
|
||
``""`` (the guard unchanged);
|
||
* the NON-same-day ISOs (yesterday + last year + the
|
||
``null``/epoch edge — the epoch's local time is a different
|
||
day AND a different year, so the qualified form applies; its
|
||
value is TZ-dependent, so it is pinned by SHAPE: the
|
||
``M/D/YYYY, h:MM AM/PM`` family vs the old ``HH:MM``) are the
|
||
pinned DIVERGENT set — the qualified form is byte-different
|
||
from the pre-phase time-only string (the phase's change lives
|
||
exactly there).
|
||
|
||
The wall clock never appears in an assertion."""
|
||
# Source-level self-check: the pinned constant carries the old
|
||
# strings verbatim (the `HH:MM` return + the parse guard).
|
||
assert (
|
||
"return `${pad(d.getHours())}:${pad(d.getMinutes())}`;"
|
||
in PRE_PHASE_FMT_SYNC_TIME_JS
|
||
), "the old HH:MM return is pinned verbatim"
|
||
assert (
|
||
'if (Number.isNaN(d.getTime())) return "";' in PRE_PHASE_FMT_SYNC_TIME_JS
|
||
), "the old parse guard is pinned verbatim"
|
||
|
||
fn = _body(_text(SOURCES_JS), "fmtSyncTime")
|
||
# (js literal, category) — category: same / fallback / diverge.
|
||
cases = [
|
||
('"2026-10-03T16:06:00"', "same"), # the audit case
|
||
('"2026-10-03T00:05:00"', "same"), # midnight edge
|
||
('"2026-10-03T23:59:00"', "same"), # end-of-day edge
|
||
('"garbage"', "fallback"),
|
||
('""', "fallback"), # empty ISO
|
||
("undefined", "fallback"), # missing ISO (the app-restart case)
|
||
("null", "diverge"), # the epoch edge — shape-pinned (TZ-dependent)
|
||
('"2026-10-02T15:14:00"', "diverge"), # yesterday
|
||
('"2026-10-02T00:05:00"', "diverge"),
|
||
('"2026-10-02T12:07:00"', "diverge"),
|
||
('"2026-10-02T23:58:00"', "diverge"),
|
||
('"2025-10-03T21:12:00"', "diverge"), # last year
|
||
('"2025-01-03T09:12:00"', "diverge"),
|
||
]
|
||
# The verbatim old strings (the pre-phase output — pinned here so
|
||
# the identity is assertable); the null/epoch value is
|
||
# TZ-dependent, so its old string is pinned by shape, not value.
|
||
old = {
|
||
'"2026-10-03T16:06:00"': "16:06",
|
||
'"2026-10-03T00:05:00"': "00:05",
|
||
'"2026-10-03T23:59:00"': "23:59",
|
||
'"garbage"': "",
|
||
'""': "",
|
||
"undefined": "",
|
||
}
|
||
driver_expr = (
|
||
"const CASES = [" + ", ".join(lit for lit, _ in cases) + "];\n"
|
||
"const PRE = CASES.map((c) => prePhaseSyncTime(c));\n"
|
||
"const REAL = CASES.map((c) => fmtSyncTime(c));\n"
|
||
"const settle = (f) => (iso) => {\n"
|
||
" const t = f(iso);\n"
|
||
" return t ? `Synced ${t}` : \"Synced\";\n"
|
||
"};\n"
|
||
"const PRE_SETTLE = CASES.map((c) => settle(prePhaseSyncTime)(c));\n"
|
||
"const REAL_SETTLE = CASES.map((c) => settle(fmtSyncTime)(c));\n"
|
||
"console.log(JSON.stringify({ PRE, REAL, PRE_SETTLE, REAL_SETTLE }));"
|
||
)
|
||
out = json.loads(
|
||
_stamp_driver(fn + "\n" + PRE_PHASE_FMT_SYNC_TIME_JS, driver_expr)
|
||
)
|
||
# 1. the pinned reference reproduces the verbatim old strings
|
||
# (self-check on the constant — assertable, not assumed).
|
||
for i, (lit, _cat) in enumerate(cases):
|
||
if lit in old:
|
||
assert out["PRE"][i] == old[lit], (
|
||
f"the pre-phase reference for {lit} must be the verbatim old"
|
||
f" string {old[lit]!r} — got {out['PRE'][i]!r} (a typo in the"
|
||
" pinned constant?"
|
||
)
|
||
null_i = [lit for lit, _ in cases].index("null")
|
||
assert re.fullmatch(r"\d{2}:\d{2}", out["PRE"][null_i]), (
|
||
"the null/epoch edge takes the old HH:MM shape on the reference"
|
||
" side (the pre-phase is time-only — the value is TZ-dependent,"
|
||
" the shape is not)"
|
||
)
|
||
assert out["PRE_SETTLE"][null_i] == "Synced " + out["PRE"][null_i] # old settle
|
||
assert re.fullmatch(
|
||
r"\d{1,2}/\d{1,2}/\d{4}, \d{1,2}:\d{2} (AM|PM)", out["REAL"][null_i]
|
||
), (
|
||
"the null/epoch edge takes the M/D/YYYY, h:MM AM/PM shape on the"
|
||
" real side (the epoch is a different day + year — the qualified"
|
||
" form; the value is TZ-dependent, the shape is not)"
|
||
)
|
||
assert out["REAL_SETTLE"][null_i].startswith("Synced ")
|
||
# 2. the audit case's settle is byte-identical to pre-phase.
|
||
audit_i = [lit for lit, _ in cases].index('"2026-10-03T16:06:00"')
|
||
assert out["REAL_SETTLE"][audit_i] == "Synced 16:06", (
|
||
"the same-day settle is byte-identical to the pre-phase string"
|
||
)
|
||
# 3. same-day + fallbacks: byte-identical to the reference, settle
|
||
# included; the divergent set: byte-different (the qualified form).
|
||
for i, (lit, cat) in enumerate(cases):
|
||
if cat == "diverge":
|
||
assert out["REAL"][i] != out["PRE"][i], (
|
||
f"{lit} must DIFFER from the pre-phase form (the qualified"
|
||
f" branch) — got {out['REAL'][i]!r} == {out['PRE'][i]!r}"
|
||
)
|
||
assert out["REAL_SETTLE"][i] != out["PRE_SETTLE"][i]
|
||
else:
|
||
assert out["REAL"][i] == out["PRE"][i], (
|
||
f"{lit} must be byte-identical to the pre-phase output"
|
||
f" (category {cat}) — got {out['REAL'][i]!r} vs"
|
||
f" {out['PRE'][i]!r}"
|
||
)
|
||
assert out["REAL_SETTLE"][i] == out["PRE_SETTLE"][i]
|
||
|
||
|
||
def test_sources_js_failed_error_is_sanitized() -> None:
|
||
"""The button's title/aria-label error is sanitized for the
|
||
attributes: the server already masks credentials (sync.py
|
||
_sanitize_error); the page collapses whitespace to a single line
|
||
and caps the length, and a missing error still names a failure."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "sanitizeSyncError")
|
||
assert "replace(/\\s+/g, \" \")" in body, "single line for the attributes"
|
||
assert "200" in body, "long errors (chatty git stderr) are capped"
|
||
assert '"The sync failed."' in body
|
||
|
||
|
||
def test_sources_js_settles_the_button_on_terminal() -> None:
|
||
"""settleSyncButton re-enables the control, drops aria-busy, the
|
||
title, and the failed affordances, and un-spins the icon — the
|
||
button can never sit disabled after a run reaches a terminal state
|
||
(failed included: retry-ready). Phase 128 (task 01): the settle /
|
||
idle label ("Sync sources", "Synced HH:MM") goes through the parts
|
||
writer — setSyncLabel(label), count hidden (absent), the full
|
||
string riding the title (A4, every label state).
|
||
Phase 150 (task 02): the `is-running` class is removed WITH the
|
||
spin (every terminal + both idles pass through the settle), and
|
||
the writer call's default `running = false` leaves the detail
|
||
box alone (the settle paths own the box's result line through
|
||
setSyncProgress) + restores the idle accessible name."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "settleSyncButton")
|
||
assert "syncBtn.disabled = false" in body
|
||
assert 'syncBtn.removeAttribute("aria-busy")' in body
|
||
assert 'syncBtn.removeAttribute("title")' in body
|
||
assert 'syncBtn.setAttribute("aria-label", "Sync sources")' in body
|
||
assert "syncIcon.classList.remove(\"is-spinning\")" in body
|
||
assert "syncBtn.classList.remove(\"is-running\")" in body, (
|
||
"the running class travels with the spin — the settle removes both"
|
||
)
|
||
assert "setSyncLabel(label)" in body, ("the settle label goes through the parts writer")
|
||
assert "setSyncLabel(label, true)" not in body, (
|
||
"the settle is NEVER a running state — the box hides (phase 149)"
|
||
)
|
||
assert "syncLabel.textContent" not in body, ("no direct span write — the parts")
|
||
|
||
|
||
def test_sources_js_never_starts_a_second_poll_loop() -> None:
|
||
"""startSyncPolling is guarded by the module-level timer: a 409
|
||
adoption, a reload re-attach, or a stray call can never run two
|
||
poll loops at once (phase completion criterion)."""
|
||
js = _text(SOURCES_JS)
|
||
fn = js.find("function startSyncPolling")
|
||
head = js[fn : js.find("const tick", fn)]
|
||
assert re.search(r"if\s*\(\s*syncPollTimer\s*!==\s*null\s*\)\s*return", head), (
|
||
"the single-loop guard must be the first statement"
|
||
)
|
||
assert "clearTimeout(syncPollTimer)" in _body(js, "stopSyncPolling")
|
||
|
||
|
||
def test_sources_js_has_no_client_side_hard_timeout() -> None:
|
||
"""Phase locked decision: a sync can legitimately run for minutes
|
||
(and outlive the page), so there is NO client-side hard timeout —
|
||
the 2 s poll is the feedback loop and the server state is
|
||
authoritative (the 120 s LLM-turn guard must not leak into the
|
||
sync path)."""
|
||
js = _text(SOURCES_JS)
|
||
assert "TURN_TIMEOUT" not in js
|
||
sync_start = js.find("Sync sources button (Sources page only)")
|
||
assert sync_start != -1, "the sync section marker comment"
|
||
assert "120" not in js[sync_start:]
|
||
|
||
|
||
def test_sources_js_reattaches_on_load_admin_only() -> None:
|
||
"""initSyncButton (run at page boot, after the click binding)
|
||
awaits the SAME cached whoami — ADMIN ONLY (non-admins never poll,
|
||
the status endpoint is admin-only): a running run re-enters the
|
||
running state (reload mid-sync), a terminal run renders its last
|
||
result, idle settles retry-ready; the click binding wires
|
||
startSync to the button."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "initSyncButton")
|
||
assert "await fetchIsAdmin()" in body, "admin-only boot (no extra fetch)"
|
||
assert 'fetch("/api/sync/status")' in body
|
||
assert 'status.state === "running"' in body
|
||
assert 'status.state === "success"' in body
|
||
assert 'status.state === "failed"' in body
|
||
assert (
|
||
'syncBtn.addEventListener("click", startSync);\n initSyncButton();'
|
||
in js
|
||
), "the click binding and the boot re-attach ship together, guarded on syncBtn"
|
||
|
||
|
||
def test_sources_js_emits_bor_sync_status_on_state_changes() -> None:
|
||
"""Every state change dispatches window 'bor:sync-status' with the
|
||
status object as detail — the channel other page scripts (or
|
||
future ones) can subscribe to. The click path emits the synthetic
|
||
running frame IMMEDIATELY (no 2 s poll lag)."""
|
||
js = _text(SOURCES_JS)
|
||
assert (
|
||
'window.dispatchEvent(new CustomEvent("bor:sync-status", { detail: status }))'
|
||
in js
|
||
)
|
||
for fn in ("applySyncSuccess", "applySyncFailure", "applySyncIdle"):
|
||
assert "emitSyncStatus" in _body(js, fn), f"{fn} must emit its frame"
|
||
# the running frame: synthetic on click, the real object on boot
|
||
assert 'emitSyncStatus({ state: "running" })' in js
|
||
assert "emitSyncStatus(status)" in _body(js, "initSyncButton")
|
||
|
||
|
||
# ---------- sources.js: the sync failure modal (phase 41, TODO.md L4) ----------
|
||
|
||
|
||
def test_sources_js_owns_a_lazily_created_sync_modal() -> None:
|
||
"""The modal is page-owned and created lazily ONCE (module-level
|
||
`let syncModal = null`): a .sync-modal-backdrop holding a
|
||
.sync-modal panel with role="alertdialog" + aria-modal +
|
||
labelled/described ids + the close button, appended to
|
||
document.body — no page-markup changes needed (the section marker
|
||
records that the former header.js modal was recreated here)."""
|
||
js = _text(SOURCES_JS)
|
||
assert "let syncModal = null" in js, "module-level once-only modal ref"
|
||
assert 'role="alertdialog"' in js
|
||
assert 'aria-modal="true"' in js
|
||
assert 'aria-labelledby="sync-modal-title"' in js
|
||
assert 'aria-describedby="sync-modal-error"' in js
|
||
assert "sync failure modal" in js, "the modal section marker"
|
||
create = _body(js, "createSyncModal")
|
||
assert "sync-modal-backdrop" in create
|
||
assert 'document.body.appendChild(backdrop)' in create
|
||
assert "return backdrop" in create, "the module ref must hold the created element"
|
||
assert "Sync failed" in create, "the dialog title"
|
||
assert 'class="sync-modal-close"' in create
|
||
assert 'aria-label="Close error dialog"' in create
|
||
|
||
|
||
def test_sync_modal_error_is_rendered_via_text_content() -> None:
|
||
"""The error text is ALWAYS set via textContent (XSS-safe — no
|
||
innerHTML with user data in the open path), and it is set BEFORE
|
||
the already-open check, so a second failure while open updates the
|
||
text IN PLACE (no stacking, no focus jump)."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "showSyncModal")
|
||
assert 'querySelector("#sync-modal-error").textContent' in body
|
||
assert "innerHTML" not in body, "the open path never touches innerHTML"
|
||
text = body.find("textContent")
|
||
open_check = body.find('contains("is-open")')
|
||
assert text != -1 and open_check != -1 and text < open_check, (
|
||
"the in-place update happens while the modal is already open"
|
||
)
|
||
|
||
|
||
def test_sync_modal_focus_goes_in_and_out_to_sync_btn() -> None:
|
||
"""On open: document.activeElement is remembered and focus moves
|
||
to the close button — with a fallback to #sync-btn when the active
|
||
element is <body> (the run's disabled button dropped focus there;
|
||
the close must still land on the control that started the run);
|
||
on close: focus returns to the remembered element (guarded by
|
||
document.contains — a detached target is a no-op)."""
|
||
js = _text(SOURCES_JS)
|
||
open_body = _body(js, "showSyncModal")
|
||
assert "const active = document.activeElement" in open_body
|
||
assert "active !== document.body" in open_body, (
|
||
"the disabled-button window leaves focus on <body> — the fallback"
|
||
)
|
||
assert "? active : syncBtn" in open_body, ("the fallback remembers #sync-btn")
|
||
assert 'querySelector(".sync-modal-close").focus()' in open_body
|
||
close_body = _body(js, "closeSyncModal")
|
||
assert "document.contains(target)" in close_body
|
||
assert "target.focus()" in close_body
|
||
assert 'contains("is-open")' in close_body, "closing a closed modal is a no-op"
|
||
|
||
|
||
def test_sync_modal_closes_via_button_esc_and_backdrop() -> None:
|
||
"""All three dismissal paths call the SAME close function: the
|
||
close button, Esc (ONE document keydown binding, acting only while
|
||
the modal is open), and a click on the backdrop element itself —
|
||
the event.target check keeps clicks bubbling from the panel from
|
||
closing it."""
|
||
js = _text(SOURCES_JS)
|
||
create = _body(js, "createSyncModal")
|
||
assert 'addEventListener("click", closeSyncModal)' in create
|
||
assert 'e.key === "Escape"' in create
|
||
assert 'addEventListener("keydown"' in create
|
||
assert 'backdrop.classList.contains("is-open")' in create
|
||
assert "e.target === backdrop" in create
|
||
|
||
|
||
def test_sources_js_apply_sync_failure_opens_the_modal_after_the_event() -> None:
|
||
"""applySyncFailure opens the modal with the SANITIZED error, and
|
||
the call comes AFTER the existing button title/aria/.is-error
|
||
lines + emitSyncStatus (the bor:sync-status event — those lines
|
||
stay byte-identical; the modal is additive)."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "applySyncFailure")
|
||
call = body.find("showSyncModal(")
|
||
emit = body.find("emitSyncStatus(status)")
|
||
assert call != -1, "the modal must open from the failure path"
|
||
assert emit != -1 and call > emit, "the event still emits first (byte-identical)"
|
||
assert "showSyncModal(error)" in body, "the sanitized error goes to the modal"
|
||
# The pre-existing affordances stay (the Sources banner contract).
|
||
assert "syncBtn.title = error" in body
|
||
assert 'syncBtn.setAttribute("aria-label", error)' in body
|
||
assert 'syncBtn.classList.add("is-error")' in body
|
||
assert "emitSyncStatus(status)" in body
|
||
|
||
|
||
# ---------- sources.js: the page-specific result line + banner ----------
|
||
|
||
|
||
def test_sources_js_renders_result_line_and_banner_directly() -> None:
|
||
"""sources.js owns the page-specific rendering on the Sources page:
|
||
#sync-progress (the sole aria-live region — phase 150 retired the
|
||
announcer span) + #sync-error-banner (role=alert), driven directly
|
||
by the state appliers: success → the counts (fmtSyncResult, the
|
||
box's result line) + a live catalog refresh; failed → the banner
|
||
with the sanitized error (the box hides, empty); every state
|
||
change still emits 'bor:sync-status' (the status object as detail)
|
||
for other subscribers."""
|
||
js = _text(SOURCES_JS)
|
||
success = _body(js, "applySyncSuccess")
|
||
# Phase 187 (task 03): the counts line keeps its builder and gains
|
||
# the TRAILING peak clause — a null pair appends the empty string, so
|
||
# every pre-phase line stays byte-identical.
|
||
assert (
|
||
"setSyncProgress(fmtSyncResult(status.detail) + fmtSyncPeakClause(status))"
|
||
in success
|
||
), "the counts live in status.detail — a bare status renders all zeros"
|
||
assert "loadTree()" in success, (
|
||
"the catalog re-fetches live on a successful sync (phase 97: the tree)"
|
||
)
|
||
failure = _body(js, "applySyncFailure")
|
||
assert "showSyncError(error)" in failure
|
||
assert "emitSyncStatus" in _body(js, "applySyncIdle")
|
||
assert (
|
||
'window.dispatchEvent(new CustomEvent("bor:sync-status", { detail: status }))'
|
||
in _body(js, "emitSyncStatus")
|
||
)
|
||
|
||
|
||
def test_header_js_no_longer_owns_the_sync_machine() -> None:
|
||
"""The state machine is GONE from header.js (sources.js owns it —
|
||
the button is Sources-page only, so the page drives it): no button
|
||
refs, no POST, no status poll, no button-state helpers, no modal.
|
||
sources.js keeps every one of them."""
|
||
header = _text(HEADER_JS)
|
||
for gone in (
|
||
"SYNC_POLL_MS",
|
||
"syncPollTimer",
|
||
"startSyncPolling",
|
||
"stopSyncPolling",
|
||
"enterSyncRunningState",
|
||
"settleSyncButton",
|
||
"applySyncSuccess",
|
||
"applySyncFailure",
|
||
"applySyncIdle",
|
||
"initSyncButton",
|
||
"syncModal",
|
||
"syncBtn",
|
||
'fetch("/api/sync", { method: "POST" })',
|
||
'fetch("/api/sync/status")',
|
||
'querySelector("#sync-btn")',
|
||
):
|
||
assert gone not in header, f"{gone!r} must be gone from header.js (sources.js owns it)"
|
||
js = _text(SOURCES_JS)
|
||
for kept in (
|
||
"SYNC_POLL_MS = 2000",
|
||
"syncPollTimer",
|
||
"startSyncPolling",
|
||
"enterSyncRunningState",
|
||
"initSyncButton",
|
||
'fetch("/api/sync", { method: "POST" })',
|
||
'fetch("/api/sync/status")',
|
||
'querySelector("#sync-btn")',
|
||
):
|
||
assert kept in js, f"{kept!r} must be in sources.js (the owner)"
|
||
|
||
|
||
# ---------- styles.css: the §7.4 states ----------
|
||
|
||
|
||
def test_sync_button_css_ghost_pill_and_disabled_state() -> None:
|
||
""".sync-btn is a pill in the shared button family (contrast pair
|
||
ink-soft on surface ≥ 4.5:1), with a ≥44px touch floor and a
|
||
:disabled state (never stale — the busy look is visible)."""
|
||
css = _text(STYLES_CSS)
|
||
block = re.search(r"\.sync-btn\s*\{([^}]*)\}", css)
|
||
assert block, "styles.css must define .sync-btn"
|
||
body = block.group(1)
|
||
assert "min-height: 44px" in body
|
||
assert "color: var(--ink-soft)" in body
|
||
assert "border-radius: 999px" in body
|
||
disabled = re.search(r"\.sync-btn:disabled\s*\{([^}]*)\}", css)
|
||
assert disabled, ".sync-btn:disabled must be styled"
|
||
assert "cursor: wait" in disabled.group(1)
|
||
|
||
|
||
def test_sync_icon_spins_and_respects_reduced_motion() -> None:
|
||
"""The running state spins the refresh icon on the shared spin
|
||
keyframes (1s linear infinite), and prefers-reduced-motion stills
|
||
it — the existing opt-out pattern."""
|
||
css = _text(STYLES_CSS)
|
||
spin = re.search(r"\.sync-btn \.sync-icon\.is-spinning\s*\{([^}]*)\}", css)
|
||
assert spin, "the .is-spinning state must be styled"
|
||
assert "animation: spin 1s linear infinite" in spin.group(1)
|
||
spinner = r"\.sync-btn \.sync-icon\.is-spinning\s*\{([^}]*)\}"
|
||
reduced = re.search(r"@media \(prefers-reduced-motion: reduce\)\s*\{\s*" + spinner, css)
|
||
assert reduced, "the spin must opt out under prefers-reduced-motion"
|
||
assert "animation: none" in reduced.group(1)
|
||
assert "@keyframes spin" in css, "the spin keyframes are shared (pre-existing)"
|
||
|
||
|
||
def test_sync_progress_box_is_the_styled_result_surface() -> None:
|
||
"""Phase 150: the retired announcer's CSS rule is GONE from styles.css
|
||
(the retirement is pinned, not implied), and #sync-progress reads
|
||
as a bordered panel in BOTH modes — the running label and the
|
||
settled result line (ASSUMPTION A3: one surface, no settled
|
||
variant). .sync-progress carries the house panel language (--surface
|
||
fill, 1px --line border, the house radius, the 4rem min-height) and
|
||
sets no display of its own, so the global [hidden] rule keeps it
|
||
display:none while its text is empty; .sync-progress-text is the
|
||
small mono soft-ink language that WRAPS (overflow-wrap: anywhere —
|
||
the point of the box) (the pinned AA pair on --surface)."""
|
||
css = _text(STYLES_CSS)
|
||
assert f".{RETIRED_ID}" not in css, (
|
||
"the retired announcer rule (incl. the mobile override) is gone (phase 150)"
|
||
)
|
||
box = re.search(r"\.sync-progress\s*\{([^}]*)\}", css)
|
||
assert box, "styles.css must define .sync-progress"
|
||
b = box.group(1)
|
||
assert "background: var(--surface)" in b
|
||
assert "border: 1px solid var(--line)" in b
|
||
assert "border-radius: var(--radius)" in b
|
||
assert "min-height: 4rem" in b
|
||
assert "display:" not in b, (
|
||
"the box sets no display of its own — the global [hidden] rule owns it"
|
||
)
|
||
hidden_rule = re.search(r"^\[hidden\]\s*\{([^}]*)\}", css, re.M)
|
||
assert hidden_rule and "display: none !important" in hidden_rule.group(1), (
|
||
"the global [hidden] rule must keep the empty box display:none"
|
||
)
|
||
text_rule = re.search(r"\.sync-progress-text\s*\{([^}]*)\}", css)
|
||
assert text_rule, "styles.css must define .sync-progress-text"
|
||
t = text_rule.group(1)
|
||
assert "var(--mono)" in t
|
||
assert "0.8rem" in t
|
||
assert "var(--ink-soft)" in t, ("the pinned AA pair on --surface (8.2:1 dark)")
|
||
assert "overflow-wrap: anywhere" in t, (
|
||
"the point of the box — a long source/relative/path wraps, never clipped"
|
||
)
|
||
|
||
|
||
# ---------- styles.css: the sync failure modal (phase 41, TODO.md L4) ----------
|
||
|
||
|
||
def test_sync_modal_css_error_palette_and_stacking() -> None:
|
||
""".sync-modal-backdrop: fixed, full-viewport, the --bg-82% dim
|
||
(phase 92: color-mix of the identity variable), z-index above the
|
||
sticky header; .sync-modal: the centered ≈28rem panel on
|
||
the phase-08 error palette (panel on --err-bg, 1px --err-line
|
||
border, --err-ink error text, --ink title — all computed ≥4.5:1);
|
||
open/close via .is-open (visibility/opacity)."""
|
||
css = _text(STYLES_CSS)
|
||
backdrop = re.search(r"\.sync-modal-backdrop\s*\{([^}]*)\}", css)
|
||
assert backdrop, "styles.css must define .sync-modal-backdrop"
|
||
b = backdrop.group(1)
|
||
assert "position: fixed" in b
|
||
assert "inset: 0" in b
|
||
assert "z-index: 1000" in b, "above the sticky header (20) + skip-link (100)"
|
||
assert "color-mix(in srgb, var(--bg) 82%, transparent)" in b, (
|
||
"the dim over the page (phase 92: --bg at 82%)"
|
||
)
|
||
open_state = re.search(r"\.sync-modal-backdrop\.is-open\s*\{([^}]*)\}", css)
|
||
assert open_state, ".is-open must be the open state"
|
||
assert "visibility: visible" in open_state.group(1)
|
||
assert "opacity: 1" in open_state.group(1)
|
||
panel = re.search(r"\.sync-modal\s*\{([^}]*)\}", css)
|
||
assert panel, "styles.css must define .sync-modal"
|
||
p = panel.group(1)
|
||
assert "max-width: 28rem" in p
|
||
assert "var(--err-bg)" in p
|
||
assert "var(--err-line)" in p
|
||
title = re.search(r"#sync-modal-title\s*\{([^}]*)\}", css)
|
||
assert title, "the modal title must be styled"
|
||
assert "var(--ink)" in title.group(1)
|
||
error = re.search(r"#sync-modal-error\s*\{([^}]*)\}", css)
|
||
assert error, "the modal error line must be styled"
|
||
assert "var(--err-ink)" in error.group(1)
|
||
|
||
|
||
def test_sync_modal_close_button_touch_floor() -> None:
|
||
"""The close button keeps the 44px touch floor at every width (the
|
||
global 3px :focus-visible outline applies — no per-button
|
||
override)."""
|
||
css = _text(STYLES_CSS)
|
||
block = re.search(r"\.sync-modal-close\s*\{([^}]*)\}", css)
|
||
assert block, "styles.css must define .sync-modal-close"
|
||
assert "min-width: 44px" in block.group(1)
|
||
assert "min-height: 44px" in block.group(1)
|
||
|
||
|
||
def test_sync_modal_respects_reduced_motion() -> None:
|
||
"""The open/close fade is stilled under prefers-reduced-motion —
|
||
the phase-25 / doc-modal opt-out pattern."""
|
||
css = _text(STYLES_CSS)
|
||
reduced = re.search(
|
||
r"@media \(prefers-reduced-motion: reduce\)\s*\{\s*\.sync-modal-backdrop\s*\{([^}]*)\}",
|
||
css,
|
||
)
|
||
assert reduced, "the backdrop fade must opt out under prefers-reduced-motion"
|
||
assert "transition: none" in reduced.group(1)
|
||
|
||
|
||
# ---------- phase 64 (task 02): per-file progress on the sync status ----------
|
||
#
|
||
# The GET /api/sync/status contract in app/api/sync.py (task 04 renders
|
||
# the live file label on the button — its pins live in
|
||
# tests/unit/test_frontend_sync_upload.py). The runner's seams are
|
||
# monkeypatched on ``app.api.sync`` (the house mock-import pattern from
|
||
# tests/integration/test_sync_api.py); the background task runs on a
|
||
# worker thread's own loop so the test can read the status mid-run.
|
||
# No DB, no HTTP: every seam is faked.
|
||
|
||
|
||
@pytest.fixture()
|
||
def fresh_sync_status() -> Iterator[None]:
|
||
"""The module-level status object + task are process-global: reset
|
||
them before AND after every progress test (the integration suite's
|
||
``_fresh_sync_state`` pattern).
|
||
|
||
Phase 134 (task 01): the module ``_cancel_event`` joins the reset —
|
||
a stale (set) event from a previous test's cancelled run would
|
||
otherwise bleed into the next run's boundary checks.
|
||
|
||
Phase 187 (task 03): the memory-sampling guard (``_rss_sampled_at``)
|
||
joins the reset — it is module-global state beside ``_status``, and a
|
||
test that pins "the per-file hook sampled" has to know whether the
|
||
guard says "never read" or "still inside the 5 s window"."""
|
||
sync_api._status = sync_api.SyncStatus()
|
||
sync_api._task = None
|
||
sync_api._cancel_event = asyncio.Event()
|
||
sync_api._rss_sampled_at = float("-inf")
|
||
yield
|
||
sync_api._status = sync_api.SyncStatus()
|
||
sync_api._task = None
|
||
sync_api._cancel_event = asyncio.Event()
|
||
sync_api._rss_sampled_at = float("-inf")
|
||
|
||
|
||
class _GatedClone:
|
||
"""A ``clone_or_pull`` that parks between start and finish on a
|
||
threading gate, so the test can read the status during the
|
||
clone/pull phase (A4: no file yet). Phase 135: accepts the row's
|
||
branch pin (the third argument — ``None`` for this file's rows)."""
|
||
|
||
def __init__(self, started: threading.Event, release: threading.Event) -> None:
|
||
self.started = started
|
||
self.release = release
|
||
self.calls: list[tuple[str, Path]] = []
|
||
|
||
def __call__(self, url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||
dest = Path(dest)
|
||
self.calls.append((url, dest))
|
||
dest.mkdir(parents=True, exist_ok=True)
|
||
(dest / "notes.md").write_text("# repo\ncontent for the KB\n", encoding="utf-8")
|
||
self.started.set()
|
||
self.release.wait() # blocking is fine: the clone is a sync call
|
||
return dest
|
||
|
||
|
||
class _GatedImport:
|
||
"""The mock import: fires the runner's OWN progress hook once (the
|
||
progress-shaped call goes through the real hook closure — the
|
||
closure under test), parks on a threading gate so the test can read
|
||
the status mid-run, then returns the canned summary (or raises
|
||
``fail``).
|
||
|
||
Phase 129 (task 03): optionally ALSO fires the in-flight file's
|
||
chunk-position pairs (``chunk_pairs``) through the runner's OWN
|
||
``_chunk_hook`` closure: pair one lands before the main park
|
||
(readable mid-run), and every later pair lands after the main
|
||
park + its own park (``chunk_release``), so each position is
|
||
readable on the status between ticks.
|
||
|
||
Phase 152 (task 03): the runner now ALWAYS passes the
|
||
image-position closure (task 03's wiring) — recorded for the
|
||
wiring pins.
|
||
|
||
Phase 152 (task 04): optionally ALSO fires the in-flight file's
|
||
image-position pairs (``image_pairs``) through the runner's OWN
|
||
``_image_hook`` closure (pair one before the main park —
|
||
readable mid-run; every later pair after the main park + its own
|
||
park ``image_release`` — the phase-129 chunk idiom, mirrored for
|
||
the vision-image position). With no pairs the fake never fires
|
||
it (the status image fields stay 0/0, like the chunk fields of a
|
||
no-batch import)."""
|
||
|
||
def __init__(
|
||
self,
|
||
summary: ImportSummary,
|
||
started: threading.Event,
|
||
release: threading.Event,
|
||
fail: BaseException | None = None,
|
||
chunk_pairs: Sequence[tuple[int, int]] = (),
|
||
chunk_release: threading.Event | None = None,
|
||
image_pairs: Sequence[tuple[int, int]] = (), # phase 152 (task 04)
|
||
image_release: threading.Event | None = None, # phase 152 (task 04)
|
||
) -> None:
|
||
self.summary = summary
|
||
self.started = started
|
||
self.release = release
|
||
self.fail = fail
|
||
self.hook_calls: list[tuple[str, str, int, int]] = []
|
||
self.prune_flags: list[bool] = []
|
||
self.chunk_pairs = list(chunk_pairs)
|
||
self.chunk_release = chunk_release
|
||
self.chunk_hook_calls: list[tuple[int, int]] = []
|
||
# Phase 152 (task 03): the image-position hook the runner
|
||
# passes (recorded for the wiring pins).
|
||
self.image_hook: Callable[[int, int], None] | None = None
|
||
self.image_pairs = list(image_pairs)
|
||
self.image_release = image_release
|
||
self.image_hook_calls: list[tuple[int, int]] = []
|
||
|
||
async def __call__(
|
||
self,
|
||
sources: list[Path],
|
||
llm: object,
|
||
*,
|
||
prune: bool = False,
|
||
limit: int | None = None,
|
||
session: object = None,
|
||
progress: Callable[[str, str, int, int], None] | None = None,
|
||
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
|
||
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
|
||
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
|
||
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
|
||
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
|
||
) -> ImportSummary:
|
||
self.prune_flags.append(prune)
|
||
if progress is not None:
|
||
progress("repo", "notes/deep.md", 1, 3)
|
||
self.hook_calls.append(("repo", "notes/deep.md", 1, 3))
|
||
# Phase 152 (task 03): the runner now ALWAYS passes the
|
||
# image-position closure (task 03's wiring) — recorded for the
|
||
# wiring pins.
|
||
self.image_hook = on_image_progress
|
||
# Phase 152 (task 04): image pair one lands BEFORE the main
|
||
# park (readable mid-run — the phase-129 chunk-pair-one
|
||
# idiom, mirrored for the vision-image position).
|
||
if on_image_progress is not None and self.image_pairs:
|
||
on_image_progress(*self.image_pairs[0])
|
||
self.image_hook_calls.append(self.image_pairs[0])
|
||
if on_chunk_progress is not None and self.chunk_pairs:
|
||
# Phase 129 (task 03): the file's chunk-position pairs,
|
||
# fired through the runner's OWN hook closure (the closure
|
||
# under test) — pair one before the main park, each later
|
||
# pair after the main park + its own park (each park lets
|
||
# the test read the status mid-run). The runner passes the
|
||
# hook only when the wiring is intact — a regression to
|
||
# None skips this branch and the pair tests time out.
|
||
on_chunk_progress(*self.chunk_pairs[0])
|
||
self.chunk_hook_calls.append(self.chunk_pairs[0])
|
||
self.started.set()
|
||
await asyncio.to_thread(self.release.wait)
|
||
for pair in self.chunk_pairs[1:]:
|
||
on_chunk_progress(*pair)
|
||
self.chunk_hook_calls.append(pair)
|
||
if self.chunk_release is not None:
|
||
await asyncio.to_thread(self.chunk_release.wait)
|
||
else:
|
||
self.started.set()
|
||
await asyncio.to_thread(self.release.wait) # park without freezing the worker loop
|
||
# Phase 152 (task 04): every later image pair lands after the
|
||
# main park + its own park (``image_release``), so each
|
||
# position is readable on the status between ticks. A wiring
|
||
# regression to None skips this branch and the pair tests
|
||
# time out.
|
||
if on_image_progress is not None:
|
||
for pair in self.image_pairs[1:]:
|
||
on_image_progress(*pair)
|
||
self.image_hook_calls.append(pair)
|
||
if self.image_release is not None:
|
||
await asyncio.to_thread(self.image_release.wait)
|
||
if self.fail is not None:
|
||
raise self.fail
|
||
return self.summary
|
||
|
||
|
||
def _patch_sync_seams(
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
fake_import: object, # any import-shaped fake (the phase-134 fakes too)
|
||
fake_clone: object, # any clone-shaped fake (the phase-134 fakes too)
|
||
) -> None:
|
||
"""The runner's seams, monkeypatched on ``app.api.sync`` (the house
|
||
mock-import pattern): fresh settings (no ``.env`` leak), a no-op
|
||
model probe, a sentinel LLM client, one git row, the gated clone +
|
||
import, a no-op overview, a no-op folder-summary step (phase 94 —
|
||
the sentinel LLM client has no ``chat``), and the DB-free
|
||
sources-version step (dummy session + pinned counters)."""
|
||
monkeypatch.setattr(
|
||
sync_api,
|
||
"get_settings",
|
||
lambda: Settings(_env_file=None, sources_dir=str(tmp_path / "bor")), # pyright: ignore[reportCallIssue]
|
||
)
|
||
|
||
async def fake_probe(llm: object) -> None:
|
||
pass
|
||
|
||
monkeypatch.setattr(sync_api, "check_models", fake_probe)
|
||
monkeypatch.setattr(sync_api, "LLMClient", lambda: object())
|
||
monkeypatch.setattr(
|
||
sync_api,
|
||
"effective_sources",
|
||
lambda session: (
|
||
# kind explicit: the Python-side default applies at INSERT
|
||
# flush, not on an in-memory instance
|
||
[GitSource(url="https://git.example.com/repo.git", kind="git")],
|
||
"db",
|
||
),
|
||
)
|
||
monkeypatch.setattr(sync_api, "clone_or_pull", fake_clone)
|
||
monkeypatch.setattr(sync_api, "import_sources", fake_import)
|
||
|
||
async def fake_overview(llm: object, session: object = None) -> bool:
|
||
return False
|
||
|
||
monkeypatch.setattr(sync_api, "regenerate_overview", fake_overview)
|
||
|
||
async def fake_folder_summaries(
|
||
db: object,
|
||
llm: object,
|
||
*,
|
||
skip: bool = False,
|
||
only_missing: bool = False,
|
||
changed_docs: Sequence[tuple[str, str]] | None = None, # phase 127
|
||
on_progress: Callable[[int, int, str, str], None] | None = None,
|
||
) -> dict[str, int]:
|
||
# Phase 98 (task 01): the runner wires the progress hook into
|
||
# the generation branches — the fake accepts it (the state-
|
||
# machine tests' canned run never fires it: zero summary
|
||
# counters, like a no-gap skip). Phase 127 (task 03): the
|
||
# changed-KB branch passes the change set as changed_docs= —
|
||
# the fake accepts it (these canned runs record nothing).
|
||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||
|
||
monkeypatch.setattr(sync_api, "generate_folder_summaries", fake_folder_summaries)
|
||
# Phase 96 (task 03): the unchanged-walk gap probe is DB-free in
|
||
# these state-machine tests — no gap, so the folder step stays
|
||
# skipped exactly as before the gate change.
|
||
monkeypatch.setattr(sync_api, "missing_folder_summaries", lambda session: [])
|
||
|
||
class _DummySession:
|
||
def close(self) -> None:
|
||
pass
|
||
|
||
def commit(self) -> None:
|
||
pass
|
||
|
||
monkeypatch.setattr(sync_api, "SessionLocal", _DummySession)
|
||
monkeypatch.setattr(sync_api, "bump_sources_version", lambda session: 1)
|
||
monkeypatch.setattr(sync_api, "current_sources_version", lambda session: 1)
|
||
|
||
|
||
def _start_run() -> tuple[threading.Thread, list[BaseException]]:
|
||
"""Run the module-level runner on a worker thread's own event loop
|
||
(the house background-task pattern), capturing any unexpected
|
||
exception — the runner is supposed to die in state, never raise."""
|
||
errors: list[BaseException] = []
|
||
|
||
def _run() -> None:
|
||
try:
|
||
asyncio.run(sync_api._run_sync())
|
||
except BaseException as e: # noqa: BLE001 — surfaced to the test
|
||
errors.append(e)
|
||
|
||
thread = threading.Thread(target=_run, daemon=True)
|
||
thread.start()
|
||
return thread, errors
|
||
|
||
|
||
def test_idle_status_pins_full_shape_including_progress_keys(
|
||
fresh_sync_status: None,
|
||
) -> None:
|
||
"""Idle: the three phase-64 progress keys ride along as null/0/0,
|
||
the two phase-129 chunk keys as 0/0, the two phase-152 image keys
|
||
as 0/0, the four phase-98 phase keys as null/0/0/0, the two phase-187
|
||
memory keys as null/null (NO reading is not 0 — and the Sources page
|
||
renders no peak clause for it), and EVERY pre-existing key is
|
||
unchanged — the full response dict is pinned, so the current UI and
|
||
every existing consumer keep working."""
|
||
assert sync_api.sync_status() == {
|
||
"state": "idle",
|
||
"started_at": None,
|
||
"finished_at": None,
|
||
"detail": {},
|
||
"error": None,
|
||
"current_file": None,
|
||
"files_done": 0,
|
||
"files_total": 0,
|
||
# Phase 129 (task 03): the chunk-position keys — 0/0 idle
|
||
# (the dataclass defaults, no special-casing).
|
||
"current_chunk": 0,
|
||
"chunks_total": 0,
|
||
# Phase 152 (task 01): the image-position keys — 0/0 idle
|
||
# (the dataclass defaults, no special-casing).
|
||
"images_done": 0,
|
||
"images_total": 0,
|
||
# Phase 98 (task 01): the phase-machine keys — null/0/0/0 idle.
|
||
"phase": None,
|
||
"current_summary": None,
|
||
"summaries_done": 0,
|
||
"summaries_total": 0,
|
||
# Phase 187 (task 03): the memory keys — null/null idle (never
|
||
# sampled: never 0, never an invented number).
|
||
"rss_mb": None,
|
||
"rss_peak_mb": None,
|
||
}
|
||
|
||
|
||
def test_mid_run_status_reports_current_file(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Mid-run: the status carries the file the (mock) import is
|
||
processing — assigned through the runner's own hook closure; during
|
||
the clone/pull phase (A4) no file is reported yet. The success
|
||
terminal clears ``current_file`` but keeps the final counts."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
# Clone/pull prelude: running — but no file yet (A4: bare
|
||
# "Syncing…") and no phase yet either (D1: the prelude stays
|
||
# null — the bare label's pin), and no chunk position yet
|
||
# (phase 129: 0/0 before the first embed batch).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] is None
|
||
assert s["current_file"] is None
|
||
assert s["current_summary"] is None
|
||
assert s["files_done"] == 0 and s["files_total"] == 0
|
||
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
# Phase 152 (task 01): the image position is 0/0 in the
|
||
# prelude too (no vision span yet).
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the run never reached the import"
|
||
# Import phase: the hook's file is live on the status, and the
|
||
# phase machine names the phase (the summary span has not
|
||
# started — its counters are still 0/0); no embed batch has
|
||
# fired the chunk hook (this canned import passes no pairs),
|
||
# so the chunk position stays 0/0; the image position stays
|
||
# 0/0 too (this canned import fires no image hook — task 03
|
||
# wires the runner's image closure, the importer's span is
|
||
# what would feed it).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1
|
||
assert s["files_total"] == 3
|
||
assert s["current_summary"] is None
|
||
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
fake_import.release.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
# Wiring: prune=True is preserved, and the hook fired through the
|
||
# runner's own closure (the recorded call is what the closure
|
||
# assigned to the status above).
|
||
assert fake_import.prune_flags == [True]
|
||
assert fake_import.hook_calls == [("repo", "notes/deep.md", 1, 3)]
|
||
# Success terminal: current_file null, final file counts retained;
|
||
# the phase machine clears phase + current_summary — the fake
|
||
# folder step never fired the summary hook, so the summary counts
|
||
# stay 0/0 (the keep-final-counts convention keeps what fired);
|
||
# the chunk position clears with the file (phase 129 — it is a
|
||
# sub-position of a done file, never a keep-final counter), and
|
||
# so does the image position (phase 152 — same convention).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_summary"] is None
|
||
assert s["current_file"] is None
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
assert s["error"] is None
|
||
assert s["started_at"] is not None and s["finished_at"] is not None
|
||
|
||
|
||
def test_failed_terminal_clears_current_file_keeps_counts(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Terminal (failed): a run that dies inside the import clears
|
||
``current_file`` but keeps the hook's final counts — the last
|
||
position is useful context next to the (sanitized) error.
|
||
|
||
Phase 129 (task 03): the convention split extends to the chunk
|
||
position — the (about-to-fail) import reports a live pair
|
||
(1/38), the failed terminal clears it to 0/0 (a sub-position of a
|
||
file that's done/abandoned), while the FILE counts keep their
|
||
final values next to the error."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
fake_import = _GatedImport(
|
||
ImportSummary(),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
fail=EmbeddingError(
|
||
"embeddings request to https://u:p@aipi.example.com/v1 "
|
||
"failed: connection refused"
|
||
),
|
||
chunk_pairs=[(1, 38)], # phase 129: the live pair before the raise
|
||
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair before the raise
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the chunk pair never landed"
|
||
# While the (about-to-fail) import is parked: the file is live,
|
||
# and so is its chunk position (the pair the fake fired through
|
||
# the runner's own closure).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
assert s["current_chunk"] == 1 and s["chunks_total"] == 38
|
||
# Phase 152 (task 04): a live image position in flight when
|
||
# the run dies — fired through the runner's OWN _image_hook
|
||
# closure (image_pairs[0] — the importer's span would do the
|
||
# same); the failed terminal must clear it (the
|
||
# sub-position of an abandoned file, like the chunk
|
||
# position).
|
||
assert s["images_done"] == 3 and s["images_total"] == 15
|
||
fake_import.release.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == []
|
||
# Wiring: the pairs fired through the runner's OWN closure.
|
||
assert fake_import.chunk_hook_calls == [(1, 38)]
|
||
assert fake_import.image_hook_calls == [(3, 15)]
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "failed"
|
||
assert s["current_file"] is None # cleared in the terminal state
|
||
assert s["phase"] is None # phase 98: cleared in the terminal state
|
||
assert s["current_summary"] is None
|
||
assert s["files_done"] == 1 and s["files_total"] == 3 # final counts kept
|
||
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
|
||
# Phase 129 (task 03): the chunk position clears in the FAILED
|
||
# terminal too — the convention split, pinned (the FILE counts
|
||
# above keep their final values, the sub-position does not).
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
# Phase 152 (task 04): the image position clears in the FAILED
|
||
# terminal too — the convention split, pinned (the live 3/15 fired
|
||
# through the closure mid-run is gone, the FILE counts keep their
|
||
# final values).
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
assert s["detail"] == {}
|
||
error = s["error"] or ""
|
||
assert "*****@aipi.example.com" in error # credentials masked
|
||
assert "u:p" not in error
|
||
assert "connection refused" in error # the reason survives
|
||
|
||
|
||
# ---------- phase 129 (task 03): the in-flight file's embed-chunk ----------
|
||
# ---------- position (current_chunk / chunks_total) ----------------------
|
||
#
|
||
# Same seams as the phase-64 block above: the fake import fires the
|
||
# runner's OWN ``_chunk_hook`` closure (the task under test) with the
|
||
# file's chunk-position pairs, parking between pairs so the test reads
|
||
# the status BETWEEN ticks (the A6 batch-granularity contract: 14 =
|
||
# the first chunk of the in-flight token-budgeted batch).
|
||
|
||
|
||
def _poll_until(
|
||
predicate: Callable[[], bool],
|
||
timeout: float = 5.0,
|
||
what: str = "condition",
|
||
) -> None:
|
||
"""Poll (main thread) until *predicate* — the worker-thread run's
|
||
state is only observable between its parks."""
|
||
deadline = time.monotonic() + timeout
|
||
while not predicate():
|
||
if time.monotonic() >= deadline:
|
||
raise AssertionError(f"timed out waiting for {what}")
|
||
time.sleep(0.01)
|
||
|
||
|
||
def test_mid_run_status_reports_chunk_position_between_ticks(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Phase 129 (task 03): the status carries the in-flight file's
|
||
embed-chunk position — the hook's (first, total) pair climbs
|
||
through the file's batches (1/38 → 13/38, read between ticks),
|
||
paired with the file the ``progress`` hook named; the prelude
|
||
reports 0/0 (no embed batch yet), and the success terminal clears
|
||
the position to 0/0 while the FILE counts keep their final values
|
||
(the convention split — the chunk position is a sub-position of a
|
||
done file, never a keep-final counter)."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
import_release = threading.Event()
|
||
chunk_release = threading.Event()
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
import_release,
|
||
chunk_pairs=[(1, 38), (13, 38)], # two batches of a 38-chunk file
|
||
chunk_release=chunk_release,
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
# Prelude: the chunk position is 0/0 (no file, no batch yet —
|
||
# alongside the bare-label pins of the phase-64 test above).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["current_file"] is None
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the first chunk pair never landed"
|
||
# Import phase, between ticks: pair one is live on the status,
|
||
# paired with the file the progress hook named (the two hooks
|
||
# are paired — the position belongs to that file).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
assert s["current_chunk"] == 1 and s["chunks_total"] == 38
|
||
# Release pair one: the import fires pair two (13/38 — the
|
||
# second batch of the 38-chunk file) and parks again.
|
||
import_release.set()
|
||
_poll_until(
|
||
lambda: sync_api.sync_status()["current_chunk"] == 13,
|
||
what="the second chunk pair on the status",
|
||
)
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
# The position CLIMBED (batch granularity) — total unchanged
|
||
# (the file's chunk count, not the batch size).
|
||
assert s["current_chunk"] == 13 and s["chunks_total"] == 38
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
chunk_release.set()
|
||
finally:
|
||
clone.release.set()
|
||
import_release.set()
|
||
chunk_release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
# Wiring: the pairs fired through the runner's OWN closure (the
|
||
# status reads above came through it; a wiring regression to
|
||
# None would have skipped the firing branch and timed out).
|
||
assert fake_import.chunk_hook_calls == [(1, 38), (13, 38)]
|
||
# Success terminal: the chunk position clears (0/0) ...
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
# ... while the FILE counts keep their final values (the
|
||
# convention split, pinned alongside the phase-64 terminal pin).
|
||
assert s["current_file"] is None
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
|
||
|
||
def test_chunk_position_clears_at_phase_transitions_and_terminal(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Phase 129 (task 03): the chunk position is CLEARED (0/0) at
|
||
every point the import phase ends. This run's summary changes the
|
||
KB, so BOTH post-import transitions run and are pinned: the
|
||
import→``"overview"`` transition and the import→``"summaries"``
|
||
transition (the changed-KB branch) each clear it to 0/0, and the
|
||
success terminal keeps it 0/0 — through the whole span the FILE
|
||
counts keep their final values (the convention split: the file
|
||
counts survive for the result line's context, the chunk position
|
||
is a sub-position of a file that's done)."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
import_release = threading.Event()
|
||
overview_started = threading.Event()
|
||
overview_release = threading.Event()
|
||
folders_started = threading.Event()
|
||
folders_release = threading.Event()
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
import_release,
|
||
chunk_pairs=[(7, 38)], # a live pair to clear at each transition
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
# Park the post-import steps (re-monkeypatched over the seams
|
||
# helper's no-ops): the test reads the status DURING the
|
||
# "overview" phase and DURING the "summaries" phase — the two
|
||
# import-phase-ending transitions the task pins.
|
||
async def gated_overview(llm: object, session: object = None) -> bool:
|
||
overview_started.set()
|
||
await asyncio.to_thread(overview_release.wait)
|
||
return True
|
||
|
||
async def gated_folders(
|
||
db: object,
|
||
llm: object,
|
||
*,
|
||
skip: bool = False,
|
||
only_missing: bool = False,
|
||
changed_docs: Sequence[tuple[str, str]] | None = None, # phase 127
|
||
on_progress: Callable[[int, int, str, str], None] | None = None,
|
||
) -> dict[str, int]:
|
||
folders_started.set()
|
||
await asyncio.to_thread(folders_release.wait)
|
||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||
|
||
monkeypatch.setattr(sync_api, "regenerate_overview", gated_overview)
|
||
monkeypatch.setattr(sync_api, "generate_folder_summaries", gated_folders)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the chunk pair never landed"
|
||
# Mid-import: the live pair (7/38) is on the status.
|
||
s = sync_api.sync_status()
|
||
assert s["phase"] == "import"
|
||
assert s["current_chunk"] == 7 and s["chunks_total"] == 38
|
||
# Release the import: it returns (added+updated > 0) → the
|
||
# phase machine moves to "overview" and the position clears.
|
||
import_release.set()
|
||
assert overview_started.wait(5.0), "the run never reached the overview"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "overview"
|
||
# CLEARED at the import→overview transition (0/0) ... the file
|
||
# counts KEEP their final values (the convention split).
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
overview_release.set()
|
||
# ... and the changed-KB branch moves to "summaries" (the
|
||
# second import-phase-ending transition — cleared there too,
|
||
# idempotently) before parking.
|
||
assert folders_started.wait(5.0), "the run never reached the summaries"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "summaries"
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
folders_release.set()
|
||
finally:
|
||
clone.release.set()
|
||
import_release.set()
|
||
overview_release.set()
|
||
folders_release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
assert fake_import.chunk_hook_calls == [(7, 38)]
|
||
# Success terminal: still 0/0 (the terminal clears too — same
|
||
# spots as current_file), file counts kept.
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_file"] is None
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
|
||
|
||
# ---------- phase 152 (task 01 + task 04): the in-flight file's -----
|
||
# ---------- vision-image position (images_done / images_total) -------
|
||
#
|
||
# Same seams as the phase-129 block above. Task 04 pins every live
|
||
# position through the runner's OWN closures (the phase-64/129 idiom):
|
||
# the fakes fire the image pairs through the ``on_image_progress``
|
||
# kwarg — the ``_image_hook`` under test (task 03's wiring: the
|
||
# importer's span's exact path) — and drive the per-file ``_hook``
|
||
# reset (through the ``progress`` kwarg the fake import receives) and
|
||
# the transition/terminal clears.
|
||
|
||
|
||
def test_mid_run_status_reports_image_position_between_ticks(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Phase 152 (task 04): the status carries the in-flight file's
|
||
VISION-image position — the pairs fire through the runner's OWN
|
||
``_image_hook`` closure (task 03's wiring — the importer's
|
||
``on_image_progress`` span's exact path), and the position CLIMBS
|
||
between ticks (3/15 → 15/15 — the PDF's vision span moving, the
|
||
owner's stuck label, now moving) while ``current_file`` and the
|
||
chunk position (14/38, a concurrent embed batch) keep their own
|
||
values — the THREE positions are independent sub-positions of the
|
||
file; the prelude reports 0/0, and the success terminal clears
|
||
BOTH sub-positions (image + chunk) to 0/0 while the FILE counts
|
||
keep their final values (the convention split)."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
image_release = threading.Event()
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
chunk_pairs=[(14, 38)], # a concurrent embed batch (independent)
|
||
image_pairs=[(3, 15), (15, 15)], # two describe calls of a 15-image span
|
||
image_release=image_release,
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
# Prelude: every position 0/0 (no file, no batch, no image
|
||
# yet — alongside the bare-label pins of the phase-64 test).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["current_file"] is None
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the run never reached the import"
|
||
# Import phase: the file is live (the progress hook's call),
|
||
# the chunk position 14/38 (the embed batch), and the image
|
||
# position 3/15 (the vision span) — the THREE positions are
|
||
# independent sub-positions of the same file.
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
assert s["current_chunk"] == 14 and s["chunks_total"] == 38
|
||
assert s["images_done"] == 3 and s["images_total"] == 15
|
||
# Release the main park: the import fires image pair two
|
||
# (15/15 — the span's last describe call) and parks on
|
||
# image_release.
|
||
fake_import.release.set()
|
||
_poll_until(
|
||
lambda: sync_api.sync_status()["images_done"] == 15,
|
||
what="the second image pair on the status",
|
||
)
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
# The image position CLIMBED (3 → 15 — monotonic, no
|
||
# backwards jump, never above the total) ... the FILE and
|
||
# chunk positions keep their own values (independent).
|
||
assert s["images_done"] == 15 and s["images_total"] == 15
|
||
assert s["current_file"] == "repo/notes/deep.md"
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
assert s["current_chunk"] == 14 and s["chunks_total"] == 38
|
||
image_release.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release.set()
|
||
image_release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
# Wiring: every pair fired through the runner's OWN closure (the
|
||
# status reads above came through it; a wiring regression to
|
||
# None would have skipped the firing branch and timed out).
|
||
assert fake_import.chunk_hook_calls == [(14, 38)]
|
||
assert fake_import.image_hook_calls == [(3, 15), (15, 15)]
|
||
# Success terminal: BOTH sub-positions clear (0/0) ...
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
# ... while the FILE counts keep their final values (the
|
||
# convention split, pinned alongside the phase-64 terminal pin).
|
||
assert s["current_file"] is None
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
|
||
|
||
class _TwoFileImport:
|
||
"""The phase-152 (task 01 + task 04) fake: fires the runner's
|
||
OWN progress hook for file A, then file A's image position through
|
||
the runner's OWN ``on_image_progress`` closure (task 03's wiring —
|
||
what the importer's span would do), parks (the test reads the
|
||
live position on the status), fires the progress hook again for
|
||
file B (the runner's ``_hook`` under test must reset the image
|
||
position to 0/0 — the pairing rule, extended), parks again, and
|
||
completes."""
|
||
|
||
def __init__(self, summary: ImportSummary) -> None:
|
||
self.summary = summary
|
||
self.parked_a = threading.Event() # parked after file A's progress
|
||
self.release_a = threading.Event() # test releases → file B's progress
|
||
self.parked_b = threading.Event() # parked after file B's progress
|
||
self.release_b = threading.Event()
|
||
self.hook_calls: list[tuple[str, str, int, int]] = []
|
||
self.image_hook_calls: list[tuple[int, int]] = [] # phase 152 (task 04)
|
||
|
||
async def __call__(
|
||
self,
|
||
sources: list[Path],
|
||
llm: object,
|
||
*,
|
||
prune: bool = False,
|
||
limit: int | None = None,
|
||
session: object = None,
|
||
progress: Callable[[str, str, int, int], None] | None = None,
|
||
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
|
||
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
|
||
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
|
||
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
|
||
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
|
||
) -> ImportSummary:
|
||
if progress is not None:
|
||
progress("repo", "docs/a.pdf", 1, 2)
|
||
self.hook_calls.append(("repo", "docs/a.pdf", 1, 2))
|
||
# Phase 152 (task 04): file A's vision span is in flight —
|
||
# its image position (3/15) fires through the runner's OWN
|
||
# on_image_progress closure (task 03's wiring), then the fake
|
||
# parks.
|
||
if on_image_progress is not None:
|
||
on_image_progress(3, 15)
|
||
self.image_hook_calls.append((3, 15))
|
||
self.parked_a.set()
|
||
await asyncio.to_thread(self.release_a.wait)
|
||
if progress is not None:
|
||
# The SECOND progress call names file B — the closure under
|
||
# test must reset the image position the closure fired
|
||
# while file A was in flight (a file with no images must
|
||
# not inherit the previous PDF's position).
|
||
progress("repo", "docs/b.md", 2, 2)
|
||
self.hook_calls.append(("repo", "docs/b.md", 2, 2))
|
||
self.parked_b.set()
|
||
await asyncio.to_thread(self.release_b.wait)
|
||
return self.summary
|
||
|
||
|
||
def test_image_position_resets_per_file(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Phase 152 (task 01 + task 04): the pairing rule, extended — a
|
||
``progress`` call names the file AND clears the image position;
|
||
every image position until the next ``progress`` call belongs to
|
||
it. File A's live position (3/15, fired through the runner's OWN
|
||
``_image_hook`` closure while parked on file A — task 04: the
|
||
phase-64/129 idiom, the importer's image hook's exact path) is
|
||
0/0 the moment the runner's own ``_hook`` names file B: a file
|
||
with no images must not inherit the previous PDF's position
|
||
(unlike the chunk position, not every file embeds chunks). The
|
||
success terminal keeps the position at 0/0 and the FILE counts
|
||
keep their final values (the convention split)."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
fake_import = _TwoFileImport(ImportSummary(files=2, added=1, updated=1))
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
clone.release.set()
|
||
assert fake_import.parked_a.wait(5.0), "the run never parked on file A"
|
||
# File A is named by the runner's own _hook (a wiring
|
||
# regression to None would leave current_file null here).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
assert s["current_file"] == "repo/docs/a.pdf"
|
||
assert s["files_done"] == 1 and s["files_total"] == 2
|
||
# File A's vision span is in flight: its image position is
|
||
# live on the status — fired through the runner's OWN
|
||
# _image_hook closure (the fake fired it before parking;
|
||
# a wiring regression to None would leave it 0/0 here).
|
||
assert s["images_done"] == 3 and s["images_total"] == 15
|
||
# Release file A: the runner's _hook names file B — the image
|
||
# position must reset to 0/0 in the same call (the pairing
|
||
# rule, extended; the closure under test, driven through the
|
||
# fake's progress kwarg).
|
||
fake_import.release_a.set()
|
||
assert fake_import.parked_b.wait(5.0), "the run never parked on file B"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "import"
|
||
assert s["current_file"] == "repo/docs/b.md"
|
||
assert s["files_done"] == 2 and s["files_total"] == 2
|
||
# RESET per file: file B (no images) does not inherit file
|
||
# A's 3/15.
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
fake_import.release_b.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release_a.set()
|
||
fake_import.release_b.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
# Wiring: BOTH progress calls went through the runner's own
|
||
# closure (the reset above was its doing), and the image pair
|
||
# went through the runner's own _image_hook closure.
|
||
assert fake_import.hook_calls == [
|
||
("repo", "docs/a.pdf", 1, 2),
|
||
("repo", "docs/b.md", 2, 2),
|
||
]
|
||
assert fake_import.image_hook_calls == [(3, 15)]
|
||
# Success terminal: the image position stays 0/0 (cleared in the
|
||
# terminal too — every terminal clears it) and the FILE counts
|
||
# keep their final values (the convention split).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_file"] is None
|
||
assert s["files_done"] == 2 and s["files_total"] == 2
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
|
||
|
||
def test_image_position_clears_at_phase_transitions_and_terminal(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""Phase 152 (task 01 + task 04): the image position is CLEARED
|
||
(0/0) at every point the import phase ends. This run's summary
|
||
changes the KB, so BOTH post-import transitions run and are
|
||
pinned: the import→``"overview"`` transition and the
|
||
import→``"summaries"`` transition (the changed-KB branch) each
|
||
clear the live position (3/15, fired mid-import through the
|
||
runner's OWN ``_image_hook`` closure — task 04: the
|
||
phase-64/129 idiom, the importer's span's exact path) to 0/0,
|
||
and the success terminal keeps it 0/0 — through the whole span
|
||
the FILE counts keep their final values (the convention split:
|
||
the file counts survive for the result line's context, the image
|
||
position is a sub-position of a file that's done)."""
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
import_release = threading.Event()
|
||
overview_started = threading.Event()
|
||
overview_release = threading.Event()
|
||
folders_started = threading.Event()
|
||
folders_release = threading.Event()
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
import_release,
|
||
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair to clear
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
# Park the post-import steps (re-monkeypatched over the seams
|
||
# helper's no-ops): the test reads the status DURING the
|
||
# "overview" phase and DURING the "summaries" phase — the two
|
||
# import-phase-ending transitions the task pins (the same shape
|
||
# as the phase-129 chunk-position transition test above).
|
||
async def gated_overview(llm: object, session: object = None) -> bool:
|
||
overview_started.set()
|
||
await asyncio.to_thread(overview_release.wait)
|
||
return True
|
||
|
||
async def gated_folders(
|
||
db: object,
|
||
llm: object,
|
||
*,
|
||
skip: bool = False,
|
||
only_missing: bool = False,
|
||
changed_docs: Sequence[tuple[str, str]] | None = None, # phase 127
|
||
on_progress: Callable[[int, int, str, str], None] | None = None,
|
||
) -> dict[str, int]:
|
||
folders_started.set()
|
||
await asyncio.to_thread(folders_release.wait)
|
||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||
|
||
monkeypatch.setattr(sync_api, "regenerate_overview", gated_overview)
|
||
monkeypatch.setattr(sync_api, "generate_folder_summaries", gated_folders)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the run never reached the import"
|
||
# Mid-import: the file is live; its image position went live
|
||
# too — fired through the runner's OWN _image_hook closure
|
||
# (image_pairs[0] — the importer's span would do the same).
|
||
s = sync_api.sync_status()
|
||
assert s["phase"] == "import"
|
||
assert s["images_done"] == 3 and s["images_total"] == 15
|
||
# Release the import: it returns (added+updated > 0) → the
|
||
# phase machine moves to "overview" and the position clears.
|
||
import_release.set()
|
||
assert overview_started.wait(5.0), "the run never reached the overview"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "overview"
|
||
# CLEARED at the import→overview transition (0/0) ... the file
|
||
# counts KEEP their final values (the convention split).
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
overview_release.set()
|
||
# ... and the changed-KB branch moves to "summaries" (the
|
||
# second import-phase-ending transition — cleared there too,
|
||
# idempotently) before parking.
|
||
assert folders_started.wait(5.0), "the run never reached the summaries"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert s["phase"] == "summaries"
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
folders_release.set()
|
||
finally:
|
||
clone.release.set()
|
||
import_release.set()
|
||
overview_release.set()
|
||
folders_release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
assert fake_import.image_hook_calls == [(3, 15)]
|
||
# Success terminal: still 0/0 (the terminal clears too — same
|
||
# spots as current_file), file counts kept.
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert s["phase"] is None
|
||
assert s["current_file"] is None
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
|
||
|
||
# ---------- phase 134 (task 01): the cooperative cancel (5th state) ----------
|
||
#
|
||
# The cancel runs are driven through the REAL ``start_sync`` on a
|
||
# worker thread's own event loop — ``sync_api._task`` is the running
|
||
# task (the cancel endpoint's target), unlike the direct-``_run_sync``
|
||
# tests above. Two driver shapes:
|
||
#
|
||
# * ``_run_via_start_sync`` — the ACTION runs ON the loop thread once
|
||
# the run is in flight (a same-thread ``Task.cancel()`` — a
|
||
# cross-thread cancel would schedule through the loop's selector
|
||
# wake, which a parked loop never receives); the loop is drained
|
||
# afterwards so the runner's terminal step (the
|
||
# ``except CancelledError`` conversion) lands before the test reads
|
||
# the status. Because the action owns the loop thread, waiting for
|
||
# run progress inside it goes through ``_pump_until`` — phase 179
|
||
# (task 02) moved the runner's git ops into ``asyncio.to_thread``, so
|
||
# the run no longer reaches the import gate inside the one loop
|
||
# iteration that created it and a blocking ``Event.wait()`` here
|
||
# deadlocks the loop the run needs to advance.
|
||
# * ``_start_run_via_api`` — the OLD house pattern (blocking threading
|
||
# gates, the loop turning on its own thread) with the main thread
|
||
# orchestrating the event set / releases — the actions run OUTSIDE the
|
||
# loop, so plain ``Event.wait()`` is fine there.
|
||
|
||
|
||
def _pump_until(
|
||
loop: asyncio.AbstractEventLoop,
|
||
event: threading.Event,
|
||
*,
|
||
why: str,
|
||
timeout: float = 5.0,
|
||
) -> None:
|
||
"""Wait for *event* while KEEPING the run's loop turning (phase 179,
|
||
task 02).
|
||
|
||
The loop-thread actions of :func:`_run_via_start_sync` need run
|
||
progress, and that progress is now produced by coroutines plus
|
||
``asyncio.to_thread`` completions this very thread has to service —
|
||
pre-fix the prelude (including the blocking git calls) ran to the
|
||
import gate before the action was ever reached, so a plain
|
||
``Event.wait()`` could not deadlock. Pump the loop instead: same
|
||
wait, same thread, no deadlock."""
|
||
deadline = time.monotonic() + timeout
|
||
while not event.is_set():
|
||
if time.monotonic() >= deadline:
|
||
raise AssertionError(why)
|
||
loop.run_until_complete(asyncio.sleep(0.01))
|
||
|
||
|
||
def _fast_clone(url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||
"""A non-gated ``clone_or_pull`` — the loop thread must STAY FREE
|
||
for the loop-thread action, so nothing in the prelude blocks here
|
||
(the import's gate is an ``asyncio.to_thread`` park — loop-safe,
|
||
and since phase 179 so are the runner's git calls).
|
||
Phase 135: accepts the row's branch pin (the third argument —
|
||
``None`` for this file's rows)."""
|
||
dest = Path(dest)
|
||
dest.mkdir(parents=True, exist_ok=True)
|
||
(dest / "notes.md").write_text("# repo\ncontent for the KB\n", encoding="utf-8")
|
||
return dest
|
||
|
||
|
||
def _run_via_start_sync(
|
||
action: Callable[[asyncio.AbstractEventLoop], Any],
|
||
) -> tuple[threading.Thread, list[BaseException], dict[str, Any]]:
|
||
"""Start a run through the real ``start_sync`` on a worker thread's
|
||
own event loop; once the run is in flight, run *action* ON that
|
||
loop thread, then drain the loop so the runner's terminal step
|
||
(the ``except CancelledError`` conversion) lands before the test
|
||
reads the status. Returns (thread, the exceptions the run
|
||
surfaced — the CancelledError of a converted cancel, the action's
|
||
result under ``"result"``)."""
|
||
errors: list[BaseException] = []
|
||
out: dict[str, Any] = {}
|
||
started = threading.Event()
|
||
|
||
def _run() -> None:
|
||
loop = asyncio.new_event_loop()
|
||
try:
|
||
asyncio.set_event_loop(loop)
|
||
loop.run_until_complete(sync_api.start_sync())
|
||
started.set()
|
||
out["result"] = action(loop)
|
||
for _ in range(20):
|
||
if sync_api._task is not None and sync_api._task.done():
|
||
break
|
||
loop.run_until_complete(asyncio.sleep(0))
|
||
assert sync_api._task is not None and sync_api._task.done(), (
|
||
"the runner task never finished"
|
||
)
|
||
except BaseException as e: # noqa: BLE001 — surfaced to the test
|
||
errors.append(e)
|
||
finally:
|
||
asyncio.set_event_loop(None)
|
||
loop.close()
|
||
|
||
thread = threading.Thread(target=_run, daemon=True)
|
||
thread.start()
|
||
assert started.wait(5.0), "start_sync never ran"
|
||
return thread, errors, out
|
||
|
||
|
||
def _start_run_via_api() -> tuple[threading.Thread, list[BaseException]]:
|
||
"""The house background-task pattern over the real ``start_sync`` —
|
||
the worker loop thread may park inside a SYNC span (the blocking
|
||
clone / ``file_commit_dates`` park); the main thread orchestrates
|
||
the event set / gate releases. The runner's surfaced exception
|
||
(the CancelledError of a converted cancel) is captured."""
|
||
errors: list[BaseException] = []
|
||
|
||
def _run() -> None:
|
||
loop = asyncio.new_event_loop()
|
||
try:
|
||
asyncio.set_event_loop(loop)
|
||
loop.run_until_complete(sync_api.start_sync())
|
||
if sync_api._task is not None:
|
||
loop.run_until_complete(sync_api._task)
|
||
except BaseException as e: # noqa: BLE001 — surfaced to the test
|
||
errors.append(e)
|
||
finally:
|
||
asyncio.set_event_loop(None)
|
||
loop.close()
|
||
|
||
thread = threading.Thread(target=_run, daemon=True)
|
||
thread.start()
|
||
return thread, errors
|
||
|
||
|
||
def test_cancel_is_409_idle_and_after_a_terminal_run(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The 409 contract (the exact detail ``no sync is running``):
|
||
idle (never started — ``_task`` None) AND after a terminal run
|
||
(``_task`` done) — the harmless race the UI's poll settles (no
|
||
error copy, no state change)."""
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=1, added=1), threading.Event(), threading.Event()
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, _fast_clone)
|
||
|
||
# Idle (never started): 409, the state untouched.
|
||
with pytest.raises(HTTPException) as idle:
|
||
asyncio.run(sync_api.cancel_sync())
|
||
assert idle.value.status_code == 409
|
||
assert idle.value.detail == "no sync is running"
|
||
assert sync_api.sync_status()["state"] == "idle"
|
||
|
||
# A full (uncancelled) run to the success terminal ...
|
||
def act(loop: asyncio.AbstractEventLoop) -> None:
|
||
_pump_until(loop, fake_import.started, why="the run never reached the import")
|
||
fake_import.release.set()
|
||
task = sync_api._task
|
||
assert task is not None
|
||
loop.run_until_complete(task)
|
||
|
||
thread, errors, out = _run_via_start_sync(act)
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
assert sync_api.sync_status()["state"] == "success"
|
||
|
||
# ... and after a terminal run the cancel is 409 again.
|
||
with pytest.raises(HTTPException) as after:
|
||
asyncio.run(sync_api.cancel_sync())
|
||
assert after.value.status_code == 409
|
||
assert after.value.detail == "no sync is running"
|
||
# The state is untouched by the rejected cancel.
|
||
assert sync_api.sync_status()["state"] == "success"
|
||
|
||
|
||
def test_cancel_endpoint_stops_the_in_flight_run(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
caplog: pytest.LogCaptureFixture,
|
||
) -> None:
|
||
"""The 202 path, end to end at the function level: a started run
|
||
(parked inside the import) + ``cancel_sync`` → 202 ``cancel
|
||
requested``; the event is set AND the task is cancelled, the run
|
||
converts the ``CancelledError`` into the ``cancelled`` terminal —
|
||
``error`` None, ``detail`` empty, ``finished_at`` set, the FINAL
|
||
file counts kept (the hook fired 1/3 before the park), the file /
|
||
phase / summary / chunk positions cleared — the PLAN §9 line is
|
||
logged, and the task ends CANCELLED (the re-raise; the status
|
||
object stays authoritative)."""
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair before the cancel
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, _fast_clone)
|
||
|
||
def act(loop: asyncio.AbstractEventLoop) -> dict[str, str]:
|
||
_pump_until(loop, fake_import.started, why="the run never reached the import")
|
||
# Phase 152 (task 04): the live image position in flight when
|
||
# the cancel converts was fired through the runner's OWN
|
||
# _image_hook closure (image_pairs[0] — the fake fired it
|
||
# before parking, like the importer's span would); the
|
||
# cancelled terminal must clear it.
|
||
return loop.run_until_complete(sync_api.cancel_sync())
|
||
|
||
try:
|
||
with caplog.at_level(logging.INFO, logger="app.api.sync"):
|
||
thread, errors, out = _run_via_start_sync(act)
|
||
thread.join(10.0)
|
||
finally:
|
||
fake_import.release.set() # free the to_thread park thread (non-daemon)
|
||
assert not thread.is_alive()
|
||
assert out.get("result") == {"detail": "cancel requested"}
|
||
# The endpoint did BOTH: the event (the cooperative marker) ...
|
||
assert sync_api._cancel_event.is_set()
|
||
# ... and the task cancel (the task ended CANCELLED — the re-raise
|
||
# happened; a swallowed cancel would have ended the task plain).
|
||
# The CancelledError itself is stored in the task (nothing awaits
|
||
# it directly — start_sync's create_task), so no thread-level
|
||
# exception surfaces — and nothing ELSE may have either.
|
||
assert sync_api._task is not None
|
||
assert sync_api._task.cancelled()
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "cancelled"
|
||
assert s["error"] is None # intent, not failure
|
||
assert s["detail"] == {}
|
||
assert s["started_at"] is not None and s["finished_at"] is not None
|
||
assert s["current_file"] is None
|
||
assert s["phase"] is None
|
||
assert s["current_summary"] is None
|
||
assert s["files_done"] == 1 and s["files_total"] == 3 # final counts kept
|
||
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
|
||
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
|
||
# Phase 152 (task 04): the image position clears in the CANCELLED
|
||
# terminal too — every terminal clears it (the live 3/15 fired
|
||
# through the closure in flight is gone; the FILE counts above
|
||
# keep their final values).
|
||
assert s["images_done"] == 0 and s["images_total"] == 0
|
||
# PLAN §9: the ample-logging line for the converted cancel. Phase
|
||
# 187 (task 02) APPENDS the memory fields (LOCKED A2) — the pre-phase
|
||
# prefix stays byte-identical, the tail is `?` or a positive int.
|
||
assert any(
|
||
re.fullmatch(
|
||
r"sync: cancelled by user state=cancelled files_done=1 files_total=3"
|
||
rf" rss_mb={MB} rss_peak_mb={MB}",
|
||
r.getMessage(),
|
||
)
|
||
for r in caplog.records
|
||
)
|
||
|
||
|
||
def test_unrequested_cancel_is_never_swallowed_and_never_converts(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The shutdown contract pin (the completion criterion): a bare
|
||
``task.cancel()`` WITHOUT the event set (the app-shutdown path —
|
||
no one pressed Cancel) re-raises the ``CancelledError`` with the
|
||
state UNTOUCHED — the task dies mid-run in ``running`` (no
|
||
terminal state, exactly as today's shutdown path does), the event
|
||
stays unset, and nothing is swallowed."""
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, _fast_clone)
|
||
|
||
def act(loop: asyncio.AbstractEventLoop) -> None:
|
||
_pump_until(loop, fake_import.started, why="the run never reached the import")
|
||
assert sync_api._cancel_event.is_set() is False
|
||
task = sync_api._task
|
||
assert task is not None
|
||
task.cancel() # NO endpoint, NO event — the shutdown path
|
||
loop.run_until_complete(task) # raises the CancelledError
|
||
|
||
try:
|
||
thread, errors, out = _run_via_start_sync(act)
|
||
thread.join(10.0)
|
||
finally:
|
||
fake_import.release.set() # free the to_thread park thread (non-daemon)
|
||
assert not thread.is_alive()
|
||
assert "result" not in out # the action raised
|
||
assert len(errors) == 1
|
||
assert isinstance(errors[0], asyncio.CancelledError)
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running" # the task died mid-run — no terminal
|
||
assert s["finished_at"] is None
|
||
assert s["error"] is None
|
||
assert s["detail"] == {}
|
||
assert sync_api._cancel_event.is_set() is False
|
||
|
||
|
||
def test_check_cancelled_raises_only_when_the_event_is_set(
|
||
fresh_sync_status: None,
|
||
) -> None:
|
||
"""The boundary helper is pure polling: a no-op while the event is
|
||
unset, ``asyncio.CancelledError`` when it is set (the run's
|
||
``except CancelledError`` clause converts THAT into the
|
||
``cancelled`` terminal)."""
|
||
assert sync_api._cancel_event.is_set() is False
|
||
sync_api._check_cancelled() # unset: no raise
|
||
sync_api._cancel_event.set()
|
||
with pytest.raises(asyncio.CancelledError):
|
||
sync_api._check_cancelled()
|
||
|
||
|
||
def test_fresh_run_clears_a_stale_cancel_event(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The stale-event pin: run ONE is cancelled (the terminal keeps
|
||
the event SET — only start_sync clears it), then a FRESH run
|
||
starts → ``start_sync`` cleared the event before the task was
|
||
created → the new run completes to ``success`` with no cancel
|
||
request (a stale event cannot bleed into the next run)."""
|
||
first_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, first_import, _fast_clone)
|
||
|
||
def act1(loop: asyncio.AbstractEventLoop) -> dict[str, str]:
|
||
_pump_until(loop, first_import.started, why="run 1 never reached the import")
|
||
return loop.run_until_complete(sync_api.cancel_sync())
|
||
|
||
try:
|
||
thread, errors, out = _run_via_start_sync(act1)
|
||
thread.join(10.0)
|
||
finally:
|
||
first_import.release.set() # free the to_thread park thread
|
||
assert not thread.is_alive()
|
||
assert errors == [], f"run 1 raised: {errors!r}"
|
||
task1 = sync_api._task
|
||
assert task1 is not None
|
||
assert task1.cancelled() # the re-raise ended it cancelled
|
||
assert out.get("result") == {"detail": "cancel requested"}
|
||
assert sync_api.sync_status()["state"] == "cancelled"
|
||
assert sync_api._cancel_event.is_set() # the terminal keeps the event set
|
||
|
||
# The FRESH run — a new gated import (the first one's gates are
|
||
# spent), the same fast clone.
|
||
second_import = _GatedImport(
|
||
ImportSummary(files=1, added=1), threading.Event(), threading.Event()
|
||
)
|
||
monkeypatch.setattr(sync_api, "import_sources", second_import)
|
||
|
||
def act2(loop: asyncio.AbstractEventLoop) -> None:
|
||
_pump_until(loop, second_import.started, why="run 2 never reached the import")
|
||
# start_sync cleared the stale event before the task was
|
||
# created — by the import park it is definitely clear.
|
||
assert sync_api._cancel_event.is_set() is False
|
||
second_import.release.set()
|
||
task = sync_api._task
|
||
assert task is not None
|
||
loop.run_until_complete(task)
|
||
|
||
try:
|
||
thread2, errors2, out2 = _run_via_start_sync(act2)
|
||
thread2.join(10.0)
|
||
finally:
|
||
second_import.release.set() # free the to_thread park thread
|
||
assert not thread2.is_alive()
|
||
assert errors2 == [], f"the fresh run raised: {errors2!r}"
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success" # no cancel request — clean completion
|
||
|
||
|
||
def test_event_alone_cancels_at_the_import_boundary(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The cooperative path WITHOUT ``task.cancel()``: the event is set
|
||
while the run is parked in the clone prelude (simulating a
|
||
registered cancel whose ``_task.cancel()`` hit a blocking
|
||
subprocess and cannot interrupt it) — the span runs to completion
|
||
and the NEXT boundary (immediately before ``import_sources``)
|
||
converts it: the import never runs (the hook never fired), the
|
||
state is ``cancelled``, the counts stay 0/0."""
|
||
clone_parked = threading.Event()
|
||
clone_release = threading.Event()
|
||
clone_calls: list[tuple[str, Path]] = []
|
||
|
||
def parking_clone(url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||
# phase 135: the row's branch pin (None for this file's rows)
|
||
dest = Path(dest)
|
||
dest.mkdir(parents=True, exist_ok=True)
|
||
(dest / "notes.md").write_text("# repo\ncontent for the KB\n", encoding="utf-8")
|
||
clone_calls.append((url, dest))
|
||
clone_parked.set()
|
||
clone_release.wait() # blocking is fine: the clone is a sync call
|
||
return dest
|
||
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, updated=1, unchanged=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, parking_clone)
|
||
|
||
thread, errors = _start_run_via_api()
|
||
try:
|
||
assert clone_parked.wait(5.0), "the run never reached the clone"
|
||
# The event ALONE (no task.cancel — the subprocess cannot be
|
||
# interrupted): the cooperative boundary must catch it.
|
||
sync_api._cancel_event.set()
|
||
finally:
|
||
clone_release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert clone_calls == [("https://git.example.com/repo.git", tmp_path / "bor" / "repo")]
|
||
assert len(errors) == 1
|
||
assert isinstance(errors[0], asyncio.CancelledError)
|
||
assert fake_import.hook_calls == [] # the import never ran
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "cancelled"
|
||
assert s["error"] is None
|
||
assert s["detail"] == {}
|
||
assert s["files_done"] == 0 and s["files_total"] == 0
|
||
assert s["current_file"] is None
|
||
assert s["phase"] is None
|
||
|
||
|
||
def test_event_set_during_import_cancels_before_the_lite_spans(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The overview + summaries boundary: the cancel's event lands
|
||
WHILE the import span runs (the fake import sets it mid-call —
|
||
the endpoint would have set it too) — the run converts at the
|
||
boundary immediately before ``if summary.added + summary.updated
|
||
> 0:``: the overview AND the folder summaries never start, the
|
||
state is ``cancelled``, and the import's final file counts are
|
||
kept."""
|
||
overview_calls: list[Any] = []
|
||
folder_calls: list[Any] = []
|
||
|
||
class _CancelDuringImport:
|
||
def __init__(self, summary: ImportSummary) -> None:
|
||
self.summary = summary
|
||
self.calls = 0
|
||
|
||
async def __call__(
|
||
self,
|
||
sources: list[Path],
|
||
llm: object,
|
||
*,
|
||
prune: bool = False,
|
||
limit: int | None = None,
|
||
session: object = None,
|
||
progress: Callable[[str, str, int, int], None] | None = None,
|
||
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
|
||
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
|
||
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
|
||
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
|
||
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
|
||
) -> ImportSummary:
|
||
self.calls += 1
|
||
if progress is not None:
|
||
progress("repo", "notes/deep.md", 1, 2)
|
||
# The user's cancel lands mid-import (the endpoint's
|
||
# _task.cancel() would interrupt here at the next per-file
|
||
# await; the event ALONE isolates the boundary check — the
|
||
# next boundary is the overview gate).
|
||
sync_api._cancel_event.set()
|
||
return self.summary
|
||
|
||
fake_import = _CancelDuringImport(ImportSummary(files=2, added=1, updated=0))
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, _fast_clone)
|
||
|
||
async def spy_overview(llm: object, session: object = None) -> bool:
|
||
overview_calls.append(llm)
|
||
return False
|
||
|
||
async def spy_folders(
|
||
db: object,
|
||
llm: object,
|
||
*,
|
||
skip: bool = False,
|
||
only_missing: bool = False,
|
||
changed_docs: Sequence[tuple[str, str]] | None = None, # phase 127
|
||
on_progress: Callable[[int, int, str, str], None] | None = None,
|
||
) -> dict[str, int]:
|
||
folder_calls.append(llm)
|
||
return {"generated": 0, "failed": 0, "pruned": 0}
|
||
|
||
monkeypatch.setattr(sync_api, "regenerate_overview", spy_overview)
|
||
monkeypatch.setattr(sync_api, "generate_folder_summaries", spy_folders)
|
||
|
||
thread, errors = _start_run_via_api()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert fake_import.calls == 1 # the import itself ran ...
|
||
assert len(errors) == 1
|
||
assert isinstance(errors[0], asyncio.CancelledError)
|
||
assert overview_calls == [] # ... but the overview never started
|
||
assert folder_calls == [] # ... and neither did the folder summaries
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "cancelled"
|
||
assert s["error"] is None
|
||
assert s["detail"] == {}
|
||
assert s["files_done"] == 1 and s["files_total"] == 2 # final counts kept
|
||
assert s["current_file"] is None
|
||
assert s["phase"] is None
|
||
assert s["current_summary"] is None
|
||
|
||
|
||
def test_event_cancels_at_the_top_of_the_source_loop(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The per-source boundary (two git rows): the event is set while
|
||
the FIRST source's post-clone date walk runs (a blocking span —
|
||
the cooperative consequence: it runs to completion) — the cancel
|
||
converts at the top of iteration TWO, before the second source's
|
||
clone/pull: exactly one clone ran, the import never started, the
|
||
state is ``cancelled``."""
|
||
parked = threading.Event()
|
||
release = threading.Event()
|
||
clone_calls: list[tuple[str, Path]] = []
|
||
|
||
def parking_dates(root: Path) -> dict[str, datetime]:
|
||
parked.set()
|
||
release.wait() # blocking is fine: the date walk is a sync call
|
||
return {}
|
||
|
||
def fast_clone(url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||
# phase 135: the row's branch pin (None for this file's rows)
|
||
dest = Path(dest)
|
||
dest.mkdir(parents=True, exist_ok=True)
|
||
(dest / "notes.md").write_text("# repo\ncontent for the KB\n", encoding="utf-8")
|
||
clone_calls.append((url, dest))
|
||
return dest
|
||
|
||
class _NeverReachedImport:
|
||
def __init__(self) -> None:
|
||
self.calls = 0
|
||
|
||
async def __call__(self, *args: Any, **kwargs: Any) -> ImportSummary:
|
||
self.calls += 1
|
||
return ImportSummary()
|
||
|
||
fake_import = _NeverReachedImport()
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, fast_clone)
|
||
monkeypatch.setattr(sync_api, "file_commit_dates", parking_dates)
|
||
# Two git rows: the cancel must fire at the top of iteration TWO
|
||
# (after the first source's git ops ran to completion).
|
||
monkeypatch.setattr(
|
||
sync_api,
|
||
"effective_sources",
|
||
lambda session: (
|
||
[
|
||
GitSource(url="https://git.example.com/one.git", kind="git"),
|
||
GitSource(url="https://git.example.com/two.git", kind="git"),
|
||
],
|
||
"db",
|
||
),
|
||
)
|
||
|
||
thread, errors = _start_run_via_api()
|
||
try:
|
||
assert parked.wait(5.0), "the run never reached the first date walk"
|
||
sync_api._cancel_event.set()
|
||
finally:
|
||
release.set()
|
||
thread.join(10.0)
|
||
assert not thread.is_alive()
|
||
assert len(errors) == 1
|
||
assert isinstance(errors[0], asyncio.CancelledError)
|
||
# Exactly the first source's git ops ran — the second clone never
|
||
# started (the boundary fired before it).
|
||
assert clone_calls == [("https://git.example.com/one.git", tmp_path / "bor" / "one")]
|
||
assert fake_import.calls == 0
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "cancelled"
|
||
assert s["error"] is None
|
||
assert s["detail"] == {}
|
||
assert s["files_done"] == 0 and s["files_total"] == 0
|
||
assert s["current_file"] is None
|
||
|
||
|
||
# ---------- phase 134 (task 02): the frontend cancel contract ----------
|
||
#
|
||
# The browser behavior is E2E-covered (tests/e2e/test_sync_cancel.py,
|
||
# task 03); here the source-level contract is pinned the same way as
|
||
# the main button above — so a silent regression (a hidden leak, a
|
||
# branch-order swap, a stale re-enable) is caught without a browser.
|
||
|
||
|
||
def test_sync_cancel_button_ships_hidden_and_labeled() -> None:
|
||
"""#sync-cancel-btn SHIPS hidden (anonymous-safe + the idle state
|
||
— sources.js reveals it the same way it reveals #sync-btn) in the
|
||
sync-control row right after the main button, with the visible
|
||
"Cancel" label (the accessible name — WCAG 2.4, never
|
||
icon-only)."""
|
||
text = _text(SHELL_HTML)
|
||
m = re.search(r"<button\b([^>]*)id=\"sync-cancel-btn\"([^>]*)>([^<]*)</button>", text)
|
||
assert m, "the shell must carry the #sync-cancel-btn (RAG view)"
|
||
attrs = m.group(1) + m.group(2)
|
||
assert 'class="sync-cancel-btn"' in attrs
|
||
assert 'type="button"' in attrs
|
||
assert re.search(r"\bhidden\b", attrs), "#sync-cancel-btn must ship hidden"
|
||
assert m.group(3).strip() == "Cancel", "the visible label is the accessible name"
|
||
# Same row: right after the main button closes, before the announcer box.
|
||
main_end = text.find("</button>", text.find('id="sync-btn"'))
|
||
cancel_start = text.find('id="sync-cancel-btn"')
|
||
assert -1 < main_end < cancel_start < text.find('id="sync-progress"')
|
||
|
||
|
||
def test_sync_cancel_btn_css_secondary_pill_and_states() -> None:
|
||
""".sync-cancel-btn is the house SECONDARY pill (the
|
||
.kb-summary-cancel language): >=44px touch floor, --line border,
|
||
the AA ink-soft text pair on the surface, the global 3px
|
||
:focus-visible ring (no per-button override), the :disabled look
|
||
(the request in flight — one request at a time), and the [hidden]
|
||
rule that keeps the attribute authoritative over the display. The
|
||
hover is guarded on :not(:disabled) — a disabled button under the
|
||
cursor never reads as actionable."""
|
||
css = _text(STYLES_CSS)
|
||
block = re.search(r"\.sync-cancel-btn\s*\{([^}]*)\}", css)
|
||
assert block, "styles.css must define .sync-cancel-btn"
|
||
body = block.group(1)
|
||
for prop in (
|
||
"display: inline-flex",
|
||
"min-height: 44px",
|
||
"border-radius: 999px",
|
||
"border: 1px solid var(--line)",
|
||
"color: var(--ink-soft)",
|
||
):
|
||
assert prop in body, f".sync-cancel-btn must carry {prop!r}"
|
||
hover = re.search(r"\.sync-cancel-btn:hover:not\(:disabled\)\s*\{([^}]*)\}", css)
|
||
assert hover, "the hover must be guarded on :not(:disabled)"
|
||
disabled = re.search(r"\.sync-cancel-btn:disabled\s*\{([^}]*)\}", css)
|
||
assert disabled, ".sync-cancel-btn:disabled must be styled"
|
||
assert "opacity: 0.6" in disabled.group(1)
|
||
hidden_rule = re.search(r"\.sync-cancel-btn\[hidden\]\s*\{([^}]*)\}", css)
|
||
assert hidden_rule, "the [hidden] rule must beat the display"
|
||
assert "display: none" in hidden_rule.group(1)
|
||
|
||
|
||
def test_sync_cancel_btn_css_full_width_on_mobile() -> None:
|
||
"""≤640px: the sync-control row already wraps (the 360px case) —
|
||
the Cancel pill stacks full-width under the full-width Sync pill
|
||
(the same mobile rule as the main button, next to it)."""
|
||
css = _text(STYLES_CSS)
|
||
main = css.find(".sync-btn { width: 100%; }")
|
||
assert main != -1, "the main button's mobile full-width rule"
|
||
near = css[main : main + 300]
|
||
assert ".sync-cancel-btn { width: 100%; }" in near, (
|
||
"the Cancel pill must join the main button's mobile full-width rule"
|
||
)
|
||
|
||
|
||
def test_sources_js_owns_the_cancel_button_and_helpers() -> None:
|
||
"""The control ref is null-safe like the main button's, and the
|
||
show/hide helpers carry the exact contract: show = unhide +
|
||
enable + aria-label "Cancel sync" (the stable accessible name);
|
||
hide = hide + disable (a hidden control is never focusable or
|
||
activatable in any state — the never-stale guarantee)."""
|
||
js = _text(SOURCES_JS)
|
||
assert 'querySelector("#sync-cancel-btn")' in js
|
||
show = _body(js, "showSyncCancelBtn")
|
||
assert "syncCancelBtn.hidden = false" in show
|
||
assert "syncCancelBtn.disabled = false" in show
|
||
assert 'syncCancelBtn.setAttribute("aria-label", "Cancel sync")' in show
|
||
hide = _body(js, "hideSyncCancelBtn")
|
||
assert "syncCancelBtn.hidden = true" in hide
|
||
assert "syncCancelBtn.disabled = true" in hide
|
||
|
||
|
||
def test_sources_js_cancel_visible_only_while_a_sync_runs() -> None:
|
||
"""The show/hide contract's two choke points: enterSyncRunningState
|
||
shows the control for kind "sync" (the live click, the re-attach,
|
||
every running tick) and hides it for kind "upload" (the upload job
|
||
is NOT cancellable in this phase — its run keeps the control
|
||
hidden); settleSyncButton hides it on EVERY settle (every
|
||
terminal + both idles pass through it)."""
|
||
js = _text(SOURCES_JS)
|
||
enter = _body(js, "enterSyncRunningState")
|
||
assert 'if (kind === "sync") showSyncCancelBtn()' in enter
|
||
assert "else hideSyncCancelBtn();" in enter
|
||
settle = _body(js, "settleSyncButton")
|
||
assert "hideSyncCancelBtn()" in settle, (
|
||
"every settle (terminal + idle) must hide the Cancel control"
|
||
)
|
||
|
||
|
||
def test_sources_js_cancel_request_is_never_stale() -> None:
|
||
"""requestCancel: the control disables IMMEDIATELY (before the
|
||
fetch — one request at a time, no double fire). Phase 150: the
|
||
box is NOT blanked with the click — the run is still in flight,
|
||
so the live label stays up until the poll's cancelled branch
|
||
settles it (the never-stale contract rides on the control's
|
||
immediate disable); then the branch set — 403 the whoami backstop
|
||
(stop polling, hide the main button, settle idle — the same depth
|
||
as every other 403 in this module), 409 the silent no-op (the
|
||
poll settles), 202 (r.ok) nothing (the poll's cancelled branch
|
||
settles everything), any other non-2xx / network failure the
|
||
phase-55 neutral retry copy + re-enable (the run is still going —
|
||
the owner can retry)."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "requestCancel")
|
||
disable = body.find("syncCancelBtn.disabled = true")
|
||
fetch = body.find('fetch("/api/sync/cancel", { method: "POST" })')
|
||
assert disable != -1 and fetch != -1 and disable < fetch, (
|
||
"the disable must land BEFORE the fetch (never stale)"
|
||
)
|
||
assert 'setSyncProgress("")' not in body, (
|
||
"phase 150: the box is NOT blanked at cancel-request time — the"
|
||
" live label stays until the poll settles"
|
||
)
|
||
assert "r.status === 403" in body
|
||
assert "stopSyncPolling()" in body
|
||
assert "syncBtn.hidden = true" in body
|
||
assert "applySyncIdle()" in body
|
||
assert "r.status === 409" in body
|
||
assert "r.ok" in body
|
||
# The phase-55 neutral retry copy + the re-enable — in the
|
||
# network-failure catch AND the non-2xx fall-through.
|
||
assert body.count("Couldn't cancel the sync — try again.") == 2
|
||
assert body.count("syncCancelBtn.disabled = false") == 2
|
||
|
||
|
||
def test_sources_js_wires_the_cancel_click_next_to_the_main() -> None:
|
||
"""The click listener is armed right after the main button's
|
||
(guarded on the control's presence — a page without the markup is
|
||
a complete no-op, the main button's convention)."""
|
||
js = _text(SOURCES_JS)
|
||
main = js.find('syncBtn.addEventListener("click", startSync)')
|
||
cancel = js.find('syncCancelBtn.addEventListener("click", requestCancel)')
|
||
assert main != -1 and cancel != -1
|
||
assert cancel > main, "the cancel wiring sits next to the main button's"
|
||
|
||
|
||
def test_sources_js_poll_tree_cancelled_branch_in_order() -> None:
|
||
"""The poll tick's decision tree (now 8 branches) gains the
|
||
cancelled branch AFTER the failed branch and BEFORE the
|
||
upload-success branch: stop polling + applySyncCancelled — the
|
||
neutral terminal, not the failure path."""
|
||
js = _text(SOURCES_JS)
|
||
start = js.find("function startSyncPolling")
|
||
end = js.find("async function startSync", start)
|
||
tick = js[start:end]
|
||
failed = tick.find('syncStatus.state === "failed"')
|
||
cancelled = tick.find('syncStatus.state === "cancelled"')
|
||
upload_success = tick.find('uploadStatus.state === "success"')
|
||
assert -1 < failed < cancelled < upload_success, (
|
||
"the cancelled branch must sit between failed and upload success"
|
||
)
|
||
branch = tick[cancelled : cancelled + 300]
|
||
assert "stopSyncPolling()" in branch
|
||
assert "applySyncCancelled(syncStatus)" in branch
|
||
|
||
|
||
def test_sources_js_apply_sync_cancelled_is_neutral() -> None:
|
||
"""applySyncCancelled renders the NEUTRAL result (A1 — cancel is
|
||
intent, not failure): settle the main button retry-ready (which
|
||
also hides the Cancel control), the "Sync cancelled" line on the
|
||
aria-live box (#sync-progress — the sole live region, phase 150:
|
||
the non-running settle no longer writes the box, so there is
|
||
nothing to supersede), the error banner hidden (a failed run's
|
||
banner must not survive), the status frame emitted — NO modal,
|
||
NO banner, NO error styling."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "applySyncCancelled")
|
||
assert 'settleSyncButton("Sync sources")' in body
|
||
# Phase 187 (task 03): the neutral copy keeps its exact wording and
|
||
# the peak clause trails it (a null pair appends the empty string).
|
||
assert 'setSyncProgress("Sync cancelled" + fmtSyncPeakClause(status))' in body
|
||
assert "hideSyncError()" in body
|
||
assert "emitSyncStatus(status)" in body
|
||
assert "showSyncModal" not in body, "cancel is intent — no modal"
|
||
assert "showSyncError" not in body, "cancel is intent — no banner"
|
||
assert "is-error" not in body, "cancel is intent — no error styling"
|
||
|
||
|
||
def test_sources_js_reattaches_a_cancelled_run() -> None:
|
||
"""initSyncButton: a reload during/right after a cancel renders
|
||
the neutral cancelled result (the branch sits after the failed
|
||
branch, before the upload check) — never a stuck running state."""
|
||
js = _text(SOURCES_JS)
|
||
body = _body(js, "initSyncButton")
|
||
assert 'status.state === "cancelled"' in body
|
||
assert "applySyncCancelled(status)" in body
|
||
failed = body.find('status.state === "failed"')
|
||
cancelled = body.find('status.state === "cancelled"')
|
||
assert -1 < failed < cancelled
|
||
|
||
|
||
def test_sources_js_section_header_documents_the_cancel_contract() -> None:
|
||
"""The house pattern: the sync section's header docstring carries
|
||
the decision tree (now 8 branches — the cancelled one is the
|
||
5th) and the Cancel control's show/hide contract."""
|
||
js = _text(SOURCES_JS)
|
||
start = js.find("Sync sources button (Sources page only)")
|
||
assert start != -1, "the sync section marker comment"
|
||
header = js[start : js.find("const syncBtn = root.querySelector", start)]
|
||
assert "5. sync cancelled" in header, "the tree's cancelled branch (the 5th)"
|
||
assert "8. both idle" in header, "the tree is now 8 branches"
|
||
assert "#sync-cancel-btn" in header, "the control's contract is documented"
|
||
|
||
|
||
# ---------- phase 187 (task 03): the run's memory on the status ----------
|
||
#
|
||
# The owner incident (2026-10-15): "how much RAM did that sync use?" was
|
||
# answered by reading systemd unit-teardown lines out of three boots —
|
||
# and the boot that killed the VM had none. Tasks 01/02 made the log
|
||
# carry the number; task 03 puts the SAME pair on the status object, so
|
||
# the page that started the run answers without a shell. The pins below
|
||
# hold the three things that make that honest: the pair is ADDITIVE (the
|
||
# key set is exactly the old set plus two), it is SAMPLED from ONE place
|
||
# (forced at the run start / every phase transition / every terminal,
|
||
# throttled to one read per 5 s in the per-file hook — LOCKED A4), and a
|
||
# reading that does not exist stays null (never 0, never invented).
|
||
|
||
|
||
#: The status payload's key set BEFORE phase 187 — the additive pin's
|
||
#: left-hand side: the payload is exactly this set plus
|
||
#: :data:`_MEMORY_KEYS`, nothing else, forever.
|
||
_PRE_187_STATUS_KEYS = frozenset(
|
||
{
|
||
"state",
|
||
"started_at",
|
||
"finished_at",
|
||
"detail",
|
||
"error",
|
||
"current_file",
|
||
"files_done",
|
||
"files_total",
|
||
"current_chunk",
|
||
"chunks_total",
|
||
"images_done",
|
||
"images_total",
|
||
"phase",
|
||
"current_summary",
|
||
"summaries_done",
|
||
"summaries_total",
|
||
}
|
||
)
|
||
|
||
#: Phase 187 (task 03): the two additive memory keys.
|
||
_MEMORY_KEYS = frozenset({"rss_mb", "rss_peak_mb"})
|
||
|
||
|
||
def test_status_key_set_is_the_old_set_plus_the_memory_pair(
|
||
fresh_sync_status: None,
|
||
) -> None:
|
||
"""The key-set pin (D4: "Additive only — every pre-existing key
|
||
keeps its name, type, and meaning"): the payload is EXACTLY the
|
||
pre-phase-187 set ``∪ {rss_mb, rss_peak_mb}`` — a fifth sub-position
|
||
or a renamed field fails here even when the value assertions pass."""
|
||
keys = set(sync_api.sync_status())
|
||
assert keys == _PRE_187_STATUS_KEYS | _MEMORY_KEYS, keys
|
||
assert _MEMORY_KEYS.isdisjoint(_PRE_187_STATUS_KEYS)
|
||
|
||
|
||
class _FakeClock:
|
||
"""``sync_api.time`` stubbed: the sampler is the module's only
|
||
``time.monotonic()`` reader, so the 5 s guard is assertable without
|
||
sleeping (the test moves the clock, never the wall)."""
|
||
|
||
def __init__(self, start: float = 10_000.0) -> None:
|
||
self.now = start
|
||
|
||
def monotonic(self) -> float:
|
||
return self.now
|
||
|
||
|
||
class _StubMemory:
|
||
"""The two ``app.core.memory`` reads as a scripted pair-per-read
|
||
source: sample *n* returns ``readings[n]`` (the last pair repeats)
|
||
and :attr:`reads` counts the samples, so "sampled at most once per
|
||
5 s" is a number and not a comment.
|
||
|
||
Both helpers are patched on ``app.api.sync`` (the module under test
|
||
imports them by name), which is also what proves the sampler reads
|
||
them from ONE place: a call counter that disagrees with
|
||
``_status`` is a second, unsampled write path."""
|
||
|
||
def __init__(
|
||
self,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
readings: Sequence[tuple[int | None, int | None]],
|
||
) -> None:
|
||
self.readings = list(readings)
|
||
self.reads = 0
|
||
monkeypatch.setattr(sync_api, "rss_mb", self._current)
|
||
monkeypatch.setattr(sync_api, "rss_peak_mb", self._peak)
|
||
|
||
def _pair(self) -> tuple[int | None, int | None]:
|
||
return self.readings[min(self.reads, len(self.readings) - 1)]
|
||
|
||
def _current(self) -> int | None:
|
||
return self._pair()[0]
|
||
|
||
def _peak(self) -> int | None:
|
||
value = self._pair()[1]
|
||
self.reads += 1 # the PAIR is one sample — counted when it completes
|
||
return value
|
||
|
||
|
||
def test_sample_rss_stores_both_readings_from_one_sample(
|
||
fresh_sync_status: None, monkeypatch: pytest.MonkeyPatch
|
||
) -> None:
|
||
"""``_sample_rss`` is the ONE writer of the pair: both fields land
|
||
from the same read, so the status and the lifecycle line can never
|
||
report two different instants."""
|
||
monkeypatch.setattr(sync_api, "time", _FakeClock())
|
||
memory = _StubMemory(monkeypatch, [(412, 1_936)])
|
||
|
||
sync_api._sample_rss(force=True)
|
||
|
||
assert sync_api._status.rss_mb == 412
|
||
assert sync_api._status.rss_peak_mb == 1_936
|
||
assert memory.reads == 1
|
||
|
||
|
||
def test_sample_rss_throttles_the_unforced_call_and_force_always_reads(
|
||
fresh_sync_status: None, monkeypatch: pytest.MonkeyPatch
|
||
) -> None:
|
||
"""LOCKED A4 / D3, pinned on the guard itself: the FIRST call reads
|
||
(never sampled), the next one inside ``_RSS_SAMPLE_INTERVAL_S`` is a
|
||
NO-OP, one past the interval reads again, and ``force=True`` reads
|
||
regardless AND re-arms the window (which is what keeps the 27k-file
|
||
walk at ~600 reads)."""
|
||
clock = _FakeClock(10_000.0)
|
||
monkeypatch.setattr(sync_api, "time", clock)
|
||
memory = _StubMemory(
|
||
monkeypatch, [(100, 200), (101, 201), (102, 202), (103, 203), (104, 204)]
|
||
)
|
||
assert sync_api._RSS_SAMPLE_INTERVAL_S == 5.0, "the phase's sampling window"
|
||
|
||
sync_api._sample_rss() # never sampled → reads
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (100, 200)
|
||
|
||
clock.now += sync_api._RSS_SAMPLE_INTERVAL_S - 0.001
|
||
sync_api._sample_rss() # inside the window → no-op
|
||
assert memory.reads == 1
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (100, 200)
|
||
|
||
clock.now += 0.002 # the window closed
|
||
sync_api._sample_rss()
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (101, 201)
|
||
|
||
sync_api._sample_rss(force=True) # forced, immediately → reads
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (102, 202)
|
||
sync_api._sample_rss() # …and the forced read re-armed the window
|
||
assert memory.reads == 3, "the forced sample closed the window it opened"
|
||
|
||
clock.now += sync_api._RSS_SAMPLE_INTERVAL_S
|
||
sync_api._sample_rss()
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (103, 203)
|
||
assert memory.reads == 4
|
||
|
||
|
||
def test_sample_rss_keeps_none_as_unknown(
|
||
fresh_sync_status: None, monkeypatch: pytest.MonkeyPatch
|
||
) -> None:
|
||
"""A platform that cannot know (no ``/proc``, no ``resource``) stays
|
||
``None`` on the status — never 0, never a plausible number. The wire
|
||
form is JSON null and the UI renders no clause for it (the same
|
||
``?``-is-not-0 rule :func:`app.core.memory.fmt_mb` holds for the
|
||
log)."""
|
||
monkeypatch.setattr(sync_api, "time", _FakeClock())
|
||
_StubMemory(monkeypatch, [(None, None)])
|
||
|
||
sync_api._sample_rss(force=True)
|
||
|
||
assert sync_api._status.rss_mb is None
|
||
assert sync_api._status.rss_peak_mb is None
|
||
body = sync_api.sync_status()
|
||
assert body["rss_mb"] is None and body["rss_peak_mb"] is None
|
||
|
||
|
||
def test_sample_rss_never_publishes_a_peak_below_the_current_reading(
|
||
fresh_sync_status: None, monkeypatch: pytest.MonkeyPatch
|
||
) -> None:
|
||
"""The pair's invariant, pinned at the ONE sampling point (task 04).
|
||
|
||
``VmRSS`` and ``ru_maxrss`` are two kernel counters read at two
|
||
instants, both floored to whole MB, and the high-water refreshes
|
||
LAZILY — so a growing run really does report ``rss_mb`` above
|
||
``rss_peak_mb`` (measured on this repo's own suite at 164 vs 162).
|
||
A "peak" under the process's own current footprint is a FALSE
|
||
number, not a conservative one, so the sampler lifts it to the
|
||
current reading: a lower bound on the truth, never an invention.
|
||
Without this the invariant every consumer (both lifecycle lines, the
|
||
status payload, the page's clause) now relies on would be a coin
|
||
flip — and the phase's own integration gate proved it."""
|
||
monkeypatch.setattr(sync_api, "time", _FakeClock())
|
||
_StubMemory(monkeypatch, [(412, 410)]) # the lazy-kernel case
|
||
|
||
sync_api._sample_rss(force=True)
|
||
|
||
assert sync_api._status.rss_mb == 412
|
||
assert sync_api._status.rss_peak_mb == 412, "the lagging peak is raised"
|
||
assert sync_api.sync_status()["rss_peak_mb"] == 412, "the wire agrees"
|
||
|
||
|
||
def test_sample_rss_does_not_fabricate_an_unknown_peak(
|
||
fresh_sync_status: None, monkeypatch: pytest.MonkeyPatch
|
||
) -> None:
|
||
"""The normalization's edge: an UNKNOWABLE peak stays ``None``.
|
||
|
||
Lifting a lagging reading is a bound on a real measurement; inventing
|
||
a peak from the current reading on a platform where
|
||
:func:`app.core.memory.rss_peak_mb` answered ``None`` would be the
|
||
invented number the phase forbids (``null`` renders no clause, ``0``
|
||
is a value). So the clamp needs BOTH readings to be real."""
|
||
monkeypatch.setattr(sync_api, "time", _FakeClock())
|
||
_StubMemory(monkeypatch, [(412, None)])
|
||
|
||
sync_api._sample_rss(force=True)
|
||
|
||
assert sync_api._status.rss_mb == 412
|
||
assert sync_api._status.rss_peak_mb is None
|
||
|
||
|
||
def test_run_samples_at_start_phase_transitions_and_terminal(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
caplog: pytest.LogCaptureFixture,
|
||
) -> None:
|
||
"""The sampling POINTS, walked through the real runner: run start →
|
||
``import`` → ``overview`` → ``summaries`` → the success terminal, one
|
||
forced read each, and the per-file progress hook's throttled call
|
||
adds NOTHING inside the window the run start just armed (LOCKED A4).
|
||
|
||
The last pair is the number the incident was about: the terminal
|
||
KEEPS it (the phase-64 keep-final-values convention) and the
|
||
``sync: done`` line logs the very same pair the payload reports —
|
||
one reading, two surfaces, no disagreement between the page and the
|
||
log an operator could be argued with."""
|
||
clock = _FakeClock(10_000.0)
|
||
monkeypatch.setattr(sync_api, "time", clock)
|
||
# run start, import, overview, summaries, done (the hook is throttled).
|
||
memory = _StubMemory(
|
||
monkeypatch,
|
||
[(10, 20), (11, 21), (12, 22), (13, 23), (14, 4_321)],
|
||
)
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=3, added=1, unchanged=2),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
with caplog.at_level(logging.INFO, logger="app.api.sync"):
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
# The run start sampled BEFORE any pipeline step: a run that
|
||
# fails in the clone already has a number on the status.
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "running"
|
||
assert (s["rss_mb"], s["rss_peak_mb"]) == (10, 20)
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the run never reached the import"
|
||
# Import phase: the transition forced the second read, and
|
||
# the hook's own call (this fake fires one file) is inside
|
||
# the window — still 2 reads, not 3 (A4 in numbers).
|
||
s = sync_api.sync_status()
|
||
assert s["phase"] == "import"
|
||
assert fake_import.hook_calls == [("repo", "notes/deep.md", 1, 3)]
|
||
assert (s["rss_mb"], s["rss_peak_mb"]) == (11, 21)
|
||
assert memory.reads == 2, "the per-file hook never bypasses the guard"
|
||
fake_import.release.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release.set()
|
||
thread.join(10.0)
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
|
||
# The terminal pair is the FINAL reading, kept next to the counts,
|
||
# and it survives the run (the page shows it after the settle).
|
||
s = sync_api.sync_status()
|
||
assert s["state"] == "success"
|
||
assert (s["rss_mb"], s["rss_peak_mb"]) == (14, 4_321)
|
||
assert s["files_done"] == 1 and s["files_total"] == 3
|
||
|
||
done = [
|
||
r.getMessage()
|
||
for r in caplog.records
|
||
if r.getMessage().startswith("sync: done detail=")
|
||
]
|
||
assert len(done) == 1, done
|
||
assert done[0].endswith("rss_mb=14 rss_peak_mb=4321"), done[0]
|
||
|
||
|
||
def test_per_file_hook_samples_once_the_window_closes(
|
||
fresh_sync_status: None,
|
||
monkeypatch: pytest.MonkeyPatch,
|
||
tmp_path: Path,
|
||
) -> None:
|
||
"""The other half of D3: the number on the page MOVES. The hook
|
||
fires once per file, and once the 5 s window has closed the next
|
||
file's call re-reads — so a long walk reports the memory climbing
|
||
(PLAN §7.4's never-stale contract) at ~600 reads per 27k files
|
||
instead of 27k."""
|
||
clock = _FakeClock(10_000.0)
|
||
monkeypatch.setattr(sync_api, "time", clock)
|
||
memory = _StubMemory(monkeypatch, [(100, 200), (110, 210), (140, 260)])
|
||
clone = _GatedClone(threading.Event(), threading.Event())
|
||
fake_import = _GatedImport(
|
||
ImportSummary(files=1, added=1),
|
||
threading.Event(),
|
||
threading.Event(),
|
||
)
|
||
# The fake fires ONE file, parks, then returns — so the hook is
|
||
# called once mid-run and the test calls it a second time with the
|
||
# window closed (the second file of a real walk).
|
||
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
|
||
|
||
thread, errors = _start_run()
|
||
try:
|
||
assert clone.started.wait(5.0), "the run never reached the clone phase"
|
||
clone.release.set()
|
||
assert fake_import.started.wait(5.0), "the run never reached the import"
|
||
s = sync_api.sync_status()
|
||
assert s["phase"] == "import"
|
||
assert (s["rss_mb"], s["rss_peak_mb"]) == (110, 210) # the transition
|
||
assert memory.reads == 2, "the hook's call inside the window read nothing"
|
||
clock.now += sync_api._RSS_SAMPLE_INTERVAL_S + 1.0
|
||
sync_api._sample_rss() # the NEXT file's hook call
|
||
assert (sync_api._status.rss_mb, sync_api._status.rss_peak_mb) == (140, 260)
|
||
fake_import.release.set()
|
||
finally:
|
||
clone.release.set()
|
||
fake_import.release.set()
|
||
thread.join(10.0)
|
||
assert errors == [], f"the runner raised: {errors!r}"
|
||
assert memory.reads >= 3
|
||
|
||
|
||
# ---------- phase 187 (task 03): the peak clause on the result line ------
|
||
|
||
|
||
def test_sources_js_peak_clause_is_a_trailing_clause_with_a_null_guard() -> None:
|
||
"""The frontend SOURCE pin (the browser behavior is
|
||
tests/e2e/test_sync_peak_memory_visible.py, phase 187 task 04):
|
||
|
||
* ONE builder, reading ``status.rss_peak_mb ?? status.rss_mb`` (the
|
||
high-water mark, the current reading as the fall-back);
|
||
* a non-number (null / undefined / garbage) renders the EMPTY string
|
||
— never ``peak ? MB``, never ``peak null MB``;
|
||
* it is APPENDED to the two result strings that already exist
|
||
(``fmtSyncResult(...)``, ``"Sync cancelled"``) through the single
|
||
box writer — no new control, no new live region, no replaced
|
||
state string;
|
||
* NOTHING else reads it: the running label, the failed box (the
|
||
banner + modal own the error, byte-identical per phase 150) and
|
||
the upload settles never grow a clause;
|
||
* the shell markup is untouched — the box is still one text span."""
|
||
js = _text(SOURCES_JS)
|
||
builder = _body(js, "fmtSyncPeakClause")
|
||
assert "status.rss_peak_mb ?? status.rss_mb" in builder
|
||
assert 'return "";' in builder, "null renders NOTHING"
|
||
assert "` · peak ${peak} MB`" in builder, "the clause's exact shape"
|
||
|
||
success = _body(js, "applySyncSuccess")
|
||
assert (
|
||
"setSyncProgress(fmtSyncResult(status.detail) + fmtSyncPeakClause(status))"
|
||
in success
|
||
)
|
||
cancelled = _body(js, "applySyncCancelled")
|
||
assert 'setSyncProgress("Sync cancelled" + fmtSyncPeakClause(status))' in cancelled
|
||
|
||
for untouched in (
|
||
"enterSyncRunningState",
|
||
"fmtSyncPhaseLabel",
|
||
"fmtSyncLabel",
|
||
"fmtSyncResult",
|
||
"setSyncLabel",
|
||
"setSyncProgress",
|
||
"applySyncFailure",
|
||
"applySyncIdle",
|
||
):
|
||
assert "fmtSyncPeakClause" not in _body(js, untouched), (
|
||
f"{untouched} must not grow a clause — the terminal result "
|
||
"line is the only surface (D4)"
|
||
)
|
||
|
||
html = _text(SHELL_HTML)
|
||
assert '<span class="sync-progress-text" id="sync-progress-text"></span>' in html
|
||
assert 'id="sync-progress-peak"' not in html, "no new element for the clause"
|
||
|
||
|
||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||
def test_sources_js_peak_clause_renders_under_node() -> None:
|
||
"""The REAL extracted ``fmtSyncResult`` + ``fmtSyncPeakClause`` under
|
||
node, composed exactly as ``applySyncSuccess`` composes them: the
|
||
counts line keeps its exact wording and the clause trails it
|
||
(``3 added · 2 unchanged · peak 1936 MB``); a null peak falls back to
|
||
the current reading; BOTH null (idle, a pre-phase-187 run, the
|
||
upload job, a platform that cannot know) render the counts line with
|
||
NO clause — never ``peak null MB``; ``0`` is a VALUE, not missing
|
||
(the ``fmt_mb(0) == "0"`` rule, mirrored); and a negative or
|
||
non-numeric value renders nothing."""
|
||
src = (
|
||
_body(_text(SOURCES_JS), "fmtSyncResult")
|
||
+ "\n"
|
||
+ _body(_text(SOURCES_JS), "fmtSyncPeakClause")
|
||
)
|
||
driver_expr = (
|
||
"const CASES = ["
|
||
"{ detail: { added: 3, unchanged: 2 }, rss_mb: 512, rss_peak_mb: 1936 },"
|
||
"{ detail: { added: 0, unchanged: 1 }, rss_mb: 512, rss_peak_mb: null },"
|
||
"{ detail: { added: 0, unchanged: 1 }, rss_mb: null, rss_peak_mb: null },"
|
||
"{ detail: { added: 1 } },"
|
||
"{ detail: { added: 2, pruned: 4 }, rss_mb: 0, rss_peak_mb: 0 },"
|
||
"{ detail: { added: 1 }, rss_peak_mb: -5 },"
|
||
"{ detail: { added: 1 }, rss_peak_mb: 'big' },"
|
||
"];\n"
|
||
"const LINES = CASES.map("
|
||
" (s) => fmtSyncResult(s.detail) + fmtSyncPeakClause(s)\n"
|
||
");\n"
|
||
"const CANCELLED = CASES.map("
|
||
" (s) => 'Sync cancelled' + fmtSyncPeakClause(s)\n"
|
||
");\n"
|
||
"console.log(JSON.stringify({ LINES, CANCELLED }));"
|
||
)
|
||
proc = subprocess.run(
|
||
["node", "-e", f"{src}\n{driver_expr}"], capture_output=True, text=True, timeout=60
|
||
)
|
||
assert proc.returncode == 0, f"node failed: {proc.stderr}"
|
||
out = json.loads(proc.stdout)
|
||
assert out["LINES"] == [
|
||
"3 added · 2 unchanged · peak 1936 MB",
|
||
"0 added · 1 unchanged · peak 512 MB", # the current-reading fallback
|
||
"0 added · 1 unchanged", # both null → no clause at all
|
||
"1 added", # idle / pre-phase / upload status: no keys, no clause
|
||
"2 added · 4 pruned · peak 0 MB", # 0 is a value, not missing
|
||
"1 added", # a negative reading is not a number to show
|
||
"1 added", # garbage is not a number either
|
||
], out
|
||
assert out["CANCELLED"] == [
|
||
"Sync cancelled · peak 1936 MB",
|
||
"Sync cancelled · peak 512 MB",
|
||
"Sync cancelled",
|
||
"Sync cancelled",
|
||
"Sync cancelled · peak 0 MB",
|
||
"Sync cancelled",
|
||
"Sync cancelled",
|
||
], out
|