phase: 152_sync_progress_image_pass
Build and Push Containers / build-and-push-app (push) Successful in 3m34s
Build and Push Containers / build-and-push-db (push) Successful in 23s

All criteria verified. The CLI path is hook-less (only `app/api/sync.py` passes the hook), and the byte-identity is pinned by `test_on_image_progress_none_run_is_byte_identical`.

## Final Report — Phase 152 final verification pass

- **Verified (no new code needed):** `SyncStatus.images_done/images_total` + reset/clear rules in `app/api/sync.py`; `on_image_progress` threaded through the importer's three PDF vision spans with the continuous per-file counter; the `_image_hook` closure; `sources.js` `· image n/m` label part (box + title + aria).
- **C1** status climbs 1→N + UI shows `· image n/m`: ✅ unit 3/15→15/15, integration 1/15→7/15, E2E 1/4→4/4 (title/aria ride the full string)
- **C2** never frozen: ✅ E2E asserts ≥3 distinct positions, starting at 1, non-decreasing, ≤ total, reaching it
- **C3** cleared at overview/summaries + all terminals; per-file reset via `_hook`; no stale position for image-less files: ✅ unit pins + E2E settle (image part gone in box/title/aria, endpoint 0/0)
- **C4** `/api/sync/status` additive-only: ✅ full-dict pin passes (all pre-existing keys unchanged + the two new 0/0 keys)
- **C5** hook-less caller byte-identical: ✅ CLI passes no hook; `test_on_image_progress_none_run_is_byte_identical` green
- **C6** `uv run pytest`: **3526 passed, exit 0** (95s); dedicated E2E in isolation: **2 passed** (95s); `uv run ruff check . && uv run pyright`: **clean**; coverage: **99% on `app/` (>90% gate)**
- Regression: all five neighbouring E2E suites green in isolation (progress_box 8, chunk_progress 2, label_mobile 4, pdf_support 6, image_documents 4)
- **C7** atomic commit: left to the harness (no git index/history touched per rules)
- **Docs:** README.md — sync-label paragraph already updated this phase, accurately; all `.env.example` vars documented; routes/commands verified — **no changes needed**. AGENTS.md — all references (PLAN, user_stories, test_smoke.py, .env.example, gpg config, .gitignore) verified — **no changes needed**
- **Defects found in prior work:** none (one prior env-leak defect was already fixed inside this phase's task 04, conftest)
- No live-infrastructure touched; no ntfy used. **Next pending phase:** none — `todo/` held only this phase
This commit is contained in:
2026-10-03 01:52:41 -04:00
parent c7cfcd26b5
commit 12fa69b7ea
39 changed files with 3971 additions and 133 deletions
+242 -37
View File
@@ -127,6 +127,28 @@ setSyncProgress and pins the result mode behaviorally (a settle
result line reveals the box, "" empties + hides it); the bare
settle's row now shows the box LEFT ALONE (the settle label rides
the button only — A2).
Phase 152 (task 03) adds the label's FOURTH part — the vision-image
position: fmtSyncPhaseLabel's import fall-through gains "· image
n/m" (rendered while images_total > 0 — a PDF's vision span is in
flight; n the 1-based image about to be described, m the file's
total images, ONE continuous counter across the PDF's three vision
spans), the overview/summaries branches return image: "" (the
position belongs to the import phase only — the status's image
fields are cleared at those transitions, so the branches never read
them), and the bare builder stays image-free (the upload label and
the 202/409 click stay bare). setSyncLabel(mainText, countText =
"", chunkText = "", running = false, imageText = "") appends the
image part LAST to the composed full string — so the box, the
button title, and the running aria-label all carry
"Syncing… <file> (2/3) · chunk 14/38 · image 3/15". The image part
gets NO span of its own (the running button is the spinner — its
parts are hidden, phase 149): it rides ONLY the composed string,
while the chunk part keeps its .sync-label-chunk span; the two
positions are distinct and each disappears independently. The node
harness gains the two in-flight image rows (both parts at once —
the owner's stuck label — and the image-only pictorial PDF) and
pins the composed full string byte-for-byte.
"""
from __future__ import annotations
@@ -298,8 +320,9 @@ def test_fmt_sync_label_returns_the_parts_contract() -> None:
def test_set_sync_label_is_the_single_parts_writer() -> None:
"""setSyncLabel(mainText, countText = "", chunkText = "",
running = false) — Phase 128 (task 01) + Phase 129 (task 04):
the SINGLE point that writes #sync-label. The MAIN part gets
running = false, imageText = "") — Phase 128 (task 01) + Phase
129 (task 04) + Phase 152 (task 03): the SINGLE point that writes
#sync-label. The MAIN part gets
mainText (textContent only — the house rule, no HTML strings);
the COUNT part gets countText when present (textContent +
hidden=false — the "(n/m)" position is visible, NEVER clipped)
@@ -309,11 +332,18 @@ def test_set_sync_label_is_the_single_parts_writer() -> None:
hidden=true — the part renders only while a file's embed batch
is in flight). Then the UNTRUNCATED full label — mainText +
(countText ? " " + countText : "") + (chunkText ? " " + chunkText
: "") — rides the button title (A4 — the screen reader hears the
whole "Syncing… <file> (2/3) · chunk 14/38" there, and in the
box — see below). Phase 149 (task 01): the fourth parameter
: "") + (imageText ? " " + imageText : "") — rides the button
title (A4 — the screen reader hears the
whole "Syncing… <file> (2/3) · chunk 14/38 · image 3/15" there,
and in the box — see below). The image part (phase 152) gets NO
span of its own (the running button is the spinner — its parts
are hidden, phase 149): it rides only the composed full string.
Phase 149 (task 01): the fourth parameter
`running` (the in-flight flag — the running entry passes true,
the settle the default false). Phase 150 (task 02): while
the settle the default false); phase 152 (task 03) adds the
fifth — `imageText` (the bare builders and the non-running calls
pass nothing, so it defaults "" and the part stays out of the
string). Phase 150 (task 02): while
running, the SAME full label goes into the detail box
(#sync-progress — the SOLE aria-live region, the announcer the
screen reader hears) through the single box writer
@@ -324,10 +354,20 @@ def test_set_sync_label_is_the_single_parts_writer() -> None:
running / the ship value "Sync sources" otherwise
(overwrites/restores only — never removes)."""
body = _fn(_js(), "setSyncLabel")
assert (
'function setSyncLabel(mainText, countText = "", chunkText = "", running = false)'
in body
), "phase 149: the in-flight flag is the fourth parameter"
# the exact signature, byte-for-byte (concatenated — the literal
# is 105 columns, over the line cap): main, count, chunk, the
# phase-149 `running` flag (fourth), the phase-152 `imageText`
# (fifth — the bare builders + non-running calls pass nothing,
# so it defaults to the empty string).
expected_sig = (
'function setSyncLabel(mainText, countText = "", chunkText = "",'
' running = false, imageText = "")'
)
assert expected_sig in body, (
"phase 149: the in-flight flag is the fourth parameter; phase"
" 152: the image part is the fifth, defaulting to the empty"
" string (the part stays out of the composed full string)"
)
# the parts: textContent only (the house rule), null-guarded refs
assert "syncLabelMain.textContent = mainText" in body
assert "syncLabelCount.textContent = countText" in body
@@ -353,8 +393,12 @@ def test_set_sync_label_is_the_single_parts_writer() -> None:
"const full =\n"
" mainText +\n"
" (countText ? ` ${countText}` : \"\") +\n"
" (chunkText ? ` ${chunkText}` : \"\");"
) in body, ("the full label is main + ' ' + count + ' ' + chunk (spaces = the CSS gaps)")
" (chunkText ? ` ${chunkText}` : \"\") +\n"
" (imageText ? ` ${imageText}` : \"\");"
) in body, (
"the full label is main + ' ' + count + ' ' + chunk + ' ' +"
" image — the image part LAST (spaces = the CSS gaps)"
)
i_full = body.find("const full")
i_title = body.find("syncBtn.title = full")
i_box = body.find("if (running) setSyncProgress(full);")
@@ -375,7 +419,8 @@ def test_set_sync_label_is_the_single_parts_writer() -> None:
def test_fmt_sync_phase_label_is_the_exact_d2_parts_contract() -> None:
"""fmtSyncPhaseLabel(status) — the SYNC job's phase-aware label
(phase 98 task 02, D2; phase 128 task 01: the PARTS). "overview"
(phase 98 task 02, D2; phase 128 task 01: the PARTS; phase 152
task 03: the IMAGE part). "overview"
→ main exactly "Writing KB overview…" (no folder, no count part);
"summaries" → main "Summarizing folders… <current_summary>" +
count "(n/m)" — the folder part (one leading space +
@@ -385,7 +430,14 @@ def test_fmt_sync_phase_label_is_the_exact_d2_parts_contract() -> None:
a total); anything else (the null prelude, "import") → the
phase-64 fmtSyncLabel parts, byte-identical (same builder, same
fields, same order). The UPLOAD job never enters here — its status
has no phase (D2)."""
has no phase (D2). Phase 152 (task 03): the import fall-through
is the ONLY branch that reads the status's image fields — "·
image n/m" renders while images_total > 0 (a PDF's vision span is
in flight — n the 1-based image about to be described, m the
file's total images) and hides at 0/0; the overview/summaries
branches return image: "" and never read the fields (the position
belongs to the import phase only — the runner clears it at those
transitions)."""
body = _fn(_js(), "fmtSyncPhaseLabel")
assert "function fmtSyncPhaseLabel(status)" in body
# "overview": the exact copy, no folder, no counts.
@@ -421,14 +473,30 @@ def test_fmt_sync_phase_label_is_the_exact_d2_parts_contract() -> None:
)
assert "chunks_total" not in ov, "the overview branch never reads the chunk fields"
assert "chunks_total" not in sum_, "the summaries branch never reads the chunk fields"
# phase 152 (task 03): the overview/summaries labels carry NO
# image part either (the position belongs to the import phase
# only) and never read the status's image fields (the runner
# clears them at those transitions).
assert 'image: ""' in ov, "the overview label carries no image part"
assert 'image: ""' in sum_, (
"the summaries label carries no image part (its (n/m) is its position)"
)
assert "images_total" not in ov, "the overview branch never reads the image fields"
assert "images_total" not in sum_, "the summaries branch never reads the image fields"
# anything else (null prelude, "import"): byte-identical to today's
# sync label — same builder, same fields, same order — plus the
# phase-129 chunk part: "· chunk n/m" ONLY while chunks_total > 0
# (a file's embed batch is in flight — n the 1-based first chunk
# of the batch, m the file's total content chunks, batch
# granularity), "" otherwise (0/0 before the first batch — the
# part hides). The fall-through is the ONLY branch that reads the
# status's chunk fields.
# part hides) + the phase-152 image part: "· image n/m" ONLY
# while images_total > 0 (a PDF's vision span is in flight — n
# the 1-based image about to be described, m the file's total
# images — ONE continuous counter across the PDF's three vision
# spans), "" otherwise (0/0 — the part hides). The fall-through
# is the ONLY branch that reads the status's chunk + image
# fields, and the two parts are independent (both may show,
# each disappears on its own 0/0).
i_destr = body.find("const { main, count } = fmtSyncLabel(")
assert -1 < i_destr < i_fall, "the fall-through destructures the phase-64 parts"
fall = body[i_destr:]
@@ -444,11 +512,22 @@ def test_fmt_sync_phase_label_is_the_exact_d2_parts_contract() -> None:
i_guard = fall.find("status.chunks_total > 0")
i_else = fall.find(": \"\";", i_guard)
assert -1 < i_guard < i_else, "0/0 (before the first batch) → the empty chunk part"
assert "return { main, count, chunk }" in fall
# phase 152 (task 03): the image part — "· image n/m" ONLY while
# images_total > 0, the exact template + the 0/0 → "" guard, and
# the return gains the fourth part.
assert "status.images_total > 0" in fall, (
"the image part renders only while a vision span is in flight"
)
assert "`· image ${status.images_done}/${status.images_total}`" in fall
i_iguard = fall.find("status.images_total > 0")
i_ielse = fall.find(": \"\";", i_iguard)
assert -1 < i_iguard < i_ielse, "0/0 (before the first image) → the empty image part"
assert "return { main, count, chunk, image }" in fall
# the bare builder (the upload job + the 202/409 click) returns NO
# chunk part — those labels stay bare.
# chunk OR image part — those labels stay bare.
bare = _fn(_js(), "fmtSyncLabel")
assert "chunk" not in bare, "fmtSyncLabel stays chunk-free (upload + the click stay bare)"
assert "image" not in bare, "fmtSyncLabel stays image-free (upload + the click stay bare)"
def test_enter_running_state_is_phase_aware_and_writes_the_parts() -> None:
@@ -460,13 +539,16 @@ def test_enter_running_state_is_phase_aware_and_writes_the_parts() -> None:
endpoint has no phase) and the 202/409 click keep the bare
fmtSyncLabel parts, byte-identical. Phase 128 (task 01): the entry
itself no longer writes the label — it hands the parts to the
SINGLE writer (setSyncLabel(main, count, chunk) — phase 129:
the chunk part destructures with a "" default), which lands the
SINGLE writer (setSyncLabel(main, count, chunk, running, image) —
phase 129: the chunk part destructures with a "" default, phase
152: the image part does too), which lands the
main part, the pinned count part (hidden when absent), the pinned
chunk part (phase 129 — hidden when absent), and the FULL
untruncated label on the button title + #sync-progress (A4 — set
for EVERY running label; CSS ellipsizes the main part only).
Phase 150 (task 02): the box write rides in the writer too
for EVERY running label; CSS ellipsizes the main part only —
phase 152: with the image part appended last when a PDF's vision
span is in flight; it gets no span of its own). Phase 150 (task
02): the box write rides in the writer too
(setSyncLabel → setSyncProgress, running mode)."""
body = _fn(_js(), "enterSyncRunningState")
assert (
@@ -487,13 +569,16 @@ def test_enter_running_state_is_phase_aware_and_writes_the_parts() -> None:
# the parts are handed to the single writer (A4 lives in it) —
# phase 129: the chunk part destructures with a "" default (the
# bare builders — the upload job, the 202/409 click — return no
# chunk part, so the part stays hidden there); phase 149: the
# chunk part, so the part stays hidden there); phase 152: the
# image part does too (it rides only the composed full string —
# no span); phase 149: the
# call passes `running = true` (the detail box reveals + the
# full-label aria-label ride in the writer).
assert "const { main, count, chunk = \"\" } =" in body, (
"the chunk part destructures with the '' default (bare builders return none)"
assert 'const { main, count, chunk = "", image = "" } =' in body, (
"the chunk + image parts destructure with the '' defaults"
" (bare builders return none)"
)
i_set = body.find("setSyncLabel(main, count, chunk, true)")
i_set = body.find("setSyncLabel(main, count, chunk, true, image)")
assert -1 < i_fall < i_set, "the parts are written after the split (running = true)"
assert body.count("setSyncLabel(") == 1, "one writer call per entry"
assert "syncBtn.title =" not in body, ("the title write lives in setSyncLabel (A4)")
@@ -713,6 +798,25 @@ enterSyncRunningState("sync", "src/a/b.md", 2, 3, {
current_chunk: 14, chunks_total: 38,
});
show("import-inflight");
// 1b. Phase 152 (task 03): import, a PDF's vision span in flight
// (the owner's stuck label — the last chunk position 125/126 is
// STILL on the status + the climbing image position 3/15): the
// composed full string carries BOTH parts (chunk keeps its span,
// the image part rides the string only — no span):
enterSyncRunningState("sync", "src/a/b.md", 2, 3, {
phase: "import", current_file: "src/a/b.md", files_done: 2, files_total: 3,
current_chunk: 125, chunks_total: 126, images_done: 3, images_total: 15,
});
show("import-image-span");
// 1c. Phase 152 (task 03): import, a purely pictorial PDF (no text
// chunks — the chunk position never left 0/0) in its vision
// span: the image part shows ALONE (the chunk part stays hidden
// + empty — the two parts are independent):
enterSyncRunningState("sync", "src/a/b.md", 2, 3, {
phase: "import", current_file: "src/a/b.md", files_done: 2, files_total: 3,
current_chunk: 0, chunks_total: 0, images_done: 7, images_total: 7,
});
show("import-image-only");
// 2. import before the first batch (0/0) — the chunk part hides:
enterSyncRunningState("sync", "src/a/b.md", 2, 3, {
phase: "import", current_file: "src/a/b.md", files_done: 2, files_total: 3,
@@ -893,6 +997,96 @@ test-document-viewer pattern). The pinned "· chunk n/m" part (the
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_image_part_renders_only_in_the_import_phase_under_node() -> None:
"""Phase 152 (task 03): behavioral check (node) of the REAL label
path — the extracted fmtSyncPhaseLabel / setSyncLabel /
enterSyncRunningState run against the stub element refs. The
pinned "· image n/m" part (the task's fixture values):
* import, a PDF's vision span in flight (images_total 15 > 0)
WITH the last chunk position still on the status (125/126):
the composed full string carries BOTH parts — the box, the
button title, and the running aria-label all read exactly
"Syncing… src/a/b.md (2/3) · chunk 125/126 · image 3/15"
(the owner's stuck label, now moving); the chunk part KEEPS
its span ("· chunk 125/126" visible) and the image part gets
NO span of its own (it rides the composed string only — the
running button is the spinner, phase 149);
* import, a purely pictorial PDF (the chunk position never left
0/0) in its vision span: the image part shows ALONE —
"Syncing… src/a/b.md (2/3) · image 7/7" — and the chunk part
stays hidden + empty (the two parts are distinct and each
disappears independently on its own 0/0);
* every OTHER running state (the pre-batch import, the prelude,
the 202/409 click, the upload run, the overview, the
summaries — the status's image fields are 0/0 or cleared at
those transitions) leaves the composed string image-free.
The box / Cancel-control / is-running contracts of the running
state apply to the new rows exactly as to every other running
state (the harness's other pins cover them per tag)."""
rows = _chunk_harness_rows()
by_tag = {row[0]: row for row in rows}
# row = [tag, title, announcer (the box's text — the sole live
# region, phase 150), main, count, countHidden, chunk,
# chunkHidden, cancelHidden, cancelDisabled, progressText,
# progressHidden, btnRunning, iconSpinning, ariaLabel]
# 1b. both parts at once — the composed full string, byte-for-byte
# (the image part LAST, after the chunk part), on the box, the
# title, and the running aria-label (A4) — the chunk span
# keeps its own part.
full = "Syncing… src/a/b.md (2/3) · chunk 125/126 · image 3/15"
row = by_tag["import-image-span"]
assert row[1] == full, row[1]
assert row[2] == full, "the announcer (the box) hears the whole thing (A4)"
assert row[14] == full, "the running button's accessible name is the full label"
assert row[10] == full, "the detail box carries the EXACT full label"
assert row[6] == "· chunk 125/126" and row[7] is False, (
"the chunk part keeps its own span alongside the image part"
)
# 1c. the image part alone (the chunk position never left 0/0 —
# the two parts are independent).
row = by_tag["import-image-only"]
assert row[1] == "Syncing… src/a/b.md (2/3) · image 7/7", row[1]
assert row[2] == row[1] and row[14] == row[1], row
assert row[6] == "" and row[7] is True, (
"the chunk part stays hidden + empty (its own 0/0)"
)
# The pre-batch import (0/0 on BOTH positions) and every other
# running state: the composed string is image-free (the part
# hides on its own 0/0 — the overview/summaries transitions
# clear the position server-side, the bare builders carry none).
for tag, expected_title in (
("import-inflight", "Syncing… src/a/b.md (2/3) · chunk 14/38"),
("import-prebatch", "Syncing… src/a/b.md (2/3)"),
("prelude", "Syncing…"),
("click", "Syncing…"),
("upload", "Importing"),
("overview", "Writing KB overview…"),
("summaries", "Summarizing folders… src/alpha (2/3)"),
):
row = by_tag[tag]
assert row[1] == expected_title, (tag, row)
assert "· image" not in row[1], (
f"{tag}: the image part shows only while images_total > 0"
)
# The running-state contracts apply to the new rows too: box
# visible + is-running + spinning, the Cancel control shown +
# enabled (a sync run in flight).
for tag in ("import-image-span", "import-image-only"):
row = by_tag[tag]
assert row[11] is False, f"{tag}: the box must be visible while in flight"
assert row[12] is True and row[13] is True, f"{tag}: is-running + spinning"
assert row[8] is False and row[9] is False, (
f"{tag}: the Cancel control is shown + enabled while a sync runs"
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_progress_box_and_running_button_contract_under_node() -> None:
"""Phase 149 (task 01) — the detail box + the icon-only running
@@ -995,14 +1189,16 @@ def test_progress_box_and_running_button_contract_under_node() -> None:
def test_every_other_state_passes_no_chunk_text() -> None:
"""Source-level pin of the render-only-while-chunks_total>0
contract: the import fall-through is the ONLY place in sources.js
that reads the status's chunk fields (the summaries/overview
branches and the bare builders never touch them), and every
NON-running label emission passes no chunk text (the part hides):
the settle/idle labels go through setSyncLabel(label) — the only
running-flagged call site (phase 149: `running = true`) is the
running entry."""
"""Source-level pin of the render-only-while-chunks_total>0 +
render-only-while-images_total>0 contracts: the import
fall-through is the ONLY place in sources.js that reads the
status's chunk OR image fields (the summaries/overview branches
and the bare builders never touch them), and every
NON-running label emission passes no chunk/image text (both
parts hide): the settle/idle labels go through setSyncLabel(label)
— the only running-flagged call site (phase 149: `running = true`)
is the running entry (phase 152: passing the image part as its
fifth argument)."""
js = _js()
body = _fn(js, "fmtSyncPhaseLabel")
i_fall = body.find("fmtSyncLabel(")
@@ -1010,12 +1206,21 @@ def test_every_other_state_passes_no_chunk_text() -> None:
assert "chunks_total" not in pre and "current_chunk" not in pre, (
"the overview/summaries branches never read the chunk fields"
)
assert "images_total" not in pre and "images_done" not in pre, (
"the overview/summaries branches never read the image fields (phase 152)"
)
fall = body[i_fall:]
assert "status.chunks_total" in fall and "status.current_chunk" in fall
assert js.count("setSyncLabel(main, count, chunk, true)") == 1, (
"the ONLY running-flagged call site is the running entry (phase 149)"
assert "status.images_total" in fall and "status.images_done" in fall, (
"the import fall-through is the ONLY reader of the image fields (phase 152)"
)
assert js.count("setSyncLabel(main, count, chunk, true, image)") == 1, (
"the ONLY running-flagged call site is the running entry (phase 149"
" — phase 152: with the image part)"
)
assert "setSyncLabel(label)" in js, (
"the settle/idle labels stay two-arg (no chunk/image)"
)
assert "setSyncLabel(label)" in js, "the settle/idle labels stay two-arg (no chunk)"
# ---------- the two-job tick (startSyncPolling) ----------
+508 -1
View File
@@ -18,7 +18,8 @@ import base64
import hashlib
import logging
import os
from collections.abc import Callable
import textwrap
from collections.abc import Callable, Sequence
from datetime import UTC, datetime
from pathlib import Path
@@ -41,6 +42,11 @@ from app.rag.importer import (
match_extension,
)
from app.rag.llm import EmbeddingError, LLMError
from app.rag.pdf import (
extract_pdf_images_with_stats,
extract_pdf_page_texts,
pictorial_pages,
)
from tests.fakes import FakeEmbedder
#: The original seven A9 formats as dotted suffixes (pre-phase-47 default
@@ -2088,3 +2094,504 @@ def test_fallback_embedded_images_still_join_the_pipeline(
assert image_doc.content != d1 and image_doc.content != d2
finally:
_cleanup_source(db, "both")
# ---------- phase 152 (task 02): on_image_progress plumbing ----------
#: A text block comfortably over the phase-140 100-char sparseness bar
#: (a USABLE layer, zero pictorial candidates even with the toggle on
#: — the "no vision span" class). Inserted wrapped at 60 chars: a
#: single long line would be CLIPPED at the page edge on extraction,
#: while the wrapped multi-line read round-trips (the phase-140
#: fixture's class property).
_P152_DENSE_TEXT = (
"The quadlet unit file defines the container interface, the image, "
"the network, and the restart policy. Every key maps to a docker "
"flag in the generated service file, and comments use the usual "
"semicolon form throughout the whole document."
)
assert len(_P152_DENSE_TEXT.strip()) > 100 # the class property, not a typo
def _p152_solid_png(value: int) -> bytes:
"""A 40x30 solid-color PNG (channel *value*): every value used
below stays under the phase-151 blank gate's 235 mean floor (kept,
never blank-skipped), and distinct values are distinct images (the
dedupe keeps every one)."""
pix = fitz.Pixmap(fitz.csRGB, fitz.IRect(0, 0, 40, 30), 0)
pix.clear_with(value)
return pix.tobytes("png")
def _build_image_pdf(
path: Path,
values: Sequence[int],
dense: bool = True,
repeat_first: bool = False,
) -> None:
"""A REAL one-page PDF: *dense* a text block over the 100-char bar
(a USABLE layer — the fallback never fires; zero pictorial
candidates even with the toggle on) carrying *values* the
solid-color embedded images (one placement each, distinct).
``dense=False`` inserts the sparse ``BESTA`` marker instead (a
pictorial candidate under the default 100-char bar).
``repeat_first`` adds a SECOND placement of the first image (the
same stream — one xref, the parser's repeated-placement class,
phase 151: the dedupe drops it, the position count is
post-dedupe)."""
doc = fitz.open()
page = doc.new_page()
text = _P152_DENSE_TEXT if dense else "BESTA"
for i, line in enumerate(textwrap.wrap(text, 60)):
page.insert_text((72, 72 + i * 14), line)
y = 300
for value in values:
page.insert_image(fitz.Rect(72, y, 112, y + 30), stream=_p152_solid_png(value))
y += 40
if repeat_first:
page.insert_image(
fitz.Rect(150, 300, 190, 330), stream=_p152_solid_png(values[0])
)
doc.save(path)
doc.close()
def _build_pictorial_image_pdf(path: Path, values: Sequence[int]) -> None:
"""A REAL two-page PDF — the ONE continuous counter's two-span
case: page 1 is SPARSE (``BESTA`` — a pictorial candidate under
the default 100-char bar), page 2 is dense (never a candidate) and
carries *values* the solid-color embedded images. Both the
phase-140 pictorial pass (before the document commit) and the
phase-126 embedded-image pass (after it) fire on it."""
doc = fitz.open()
doc.new_page().insert_text((72, 72), "BESTA")
page = doc.new_page()
for i, line in enumerate(textwrap.wrap(_P152_DENSE_TEXT, 60)):
page.insert_text((72, 72 + i * 14), line)
y = 300
for value in values:
page.insert_image(fitz.Rect(72, y, 112, y + 30), stream=_p152_solid_png(value))
y += 40
doc.save(path)
doc.close()
def _build_garbage_image_pdf(path: Path, values: Sequence[int]) -> None:
"""A REAL two-page garbage-text PDF (every page XML-ish — the
phase-133 gate fires, the page-screenshot fallback runs) with
*values* the solid-color embedded images on page 2 — the fallback
span + the embedded span, ONE continuous counter."""
doc = fitz.open()
for p in range(1, 3):
page = doc.new_page()
for i in range(10):
page.insert_text((72, 72 + i * 14), f"<foo:p{p}>{i}</foo:p{p}>")
y = 400
for value in values:
doc[1].insert_image(
fitz.Rect(72, y, 112, y + 30), stream=_p152_solid_png(value)
)
y += 40
doc.save(path)
doc.close()
def _p152_summary_fields(s: ImportSummary) -> tuple:
"""One run's observable outcome (the byte-identity pin's left
side): every count a hook could plausibly move + the changed
paths' RELS (the source names are equal by construction — the two
roots carry the same directory name)."""
return (
s.files, s.added, s.updated, s.unchanged, s.pruned, s.errors,
s.chunks, s.embed_batches, s.summaries, s.summary_errors,
s.images_failed, s.pdfs_screenshotted, s.pdfs_pictorial,
s.pdf_pictorial_pages, s.pdf_images_deduped, s.pdf_images_blank_skipped,
tuple(rel for _, rel in s.changed_paths),
)
def _p152_rows(db, source: str) -> list[tuple]:
"""A source's document rows (the byte-identity pin's right side),
identity-blind: the fields a hook could plausibly move."""
docs = db.scalars(select(Document).where(Document.source == source)).all()
return sorted(
(d.path, d.title, d.content, d.content_hash, d.is_image, d.summary,
d.image_context)
for d in docs
)
def test_on_image_progress_embedded_pass_climbs_one_per_image(
db, tmp_path: Path
) -> None:
"""The phase-152 core pin: a PDF with N DISTINCT embedded images
(one placement each — post-dedupe, post-blank-gate: the work
actually about to happen) fires the hook ONCE per image, 1-based,
with the span's total, at the START of each image's pipeline step:
``(1, N) … (N, N)``. A repeated placement of the first image (the
parser's dedupe, phase 151) adds NO position (the total is the
post-dedupe count), a clean usable text layer burns no fallback or
pictorial span (the embedded pass's totals stand alone), and the
phase-147 fast path on a re-sync (raw bytes unchanged) burns no
vision call and fires the hook never."""
root = tmp_path / "imgprog"
root.mkdir()
values = (90, 130, 170, 210) # 4 distinct images, all under the blank floor
_build_image_pdf(root / "report.pdf", values, repeat_first=True)
n = len(values)
llm = _PageVisionLLM()
llm.settings = _pdf_settings(tmp_path, images=True)
events: list[tuple[int, int]] = []
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
# The PDF + its N image docs; the repeat placement deduped.
assert (summary.files, summary.added) == (1, 1 + n)
assert summary.pdf_images_deduped == 1
assert (summary.images_failed, summary.pdfs_screenshotted,
summary.pdfs_pictorial) == (0, 0, 0)
# The embedded pass is the ONLY vision span: N slow calls.
assert llm.vision_calls == n
# The core pin: once per image, 1-based, the span's total.
assert events == [(i, n) for i in range(1, n + 1)]
# The phase-147 fast path: a re-sync (bytes unchanged) burns no
# vision call and fires the hook never — the events list grows
# not at all.
summary2 = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
assert summary2.unchanged == 1
assert llm.vision_calls == n
assert len(events) == n
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_pictorial_then_embedded_is_one_counter(
db, tmp_path: Path
) -> None:
"""ASSUMPTION A3: the spans of ONE file are ONE continuous counter
— the embedded pass starts where the pictorial pass ended (``base
= images_done_so_far``): the positions climb ``1…C+N`` with NO
restart and never above the combined total (the pictorial span's
total is its candidate-page count, the embedded span's total the
combined count)."""
root = tmp_path / "imgprog-two"
root.mkdir()
values = (90, 130, 170)
_build_pictorial_image_pdf(root / "ikea.pdf", values)
# The span inputs, computed IN the test (no hardcoded lists — the
# phase-140 idiom): the deterministic gate's candidates and the
# post-dedupe/post-gate image count.
pdf = root / "ikea.pdf"
page_texts = extract_pdf_page_texts(pdf)
n_cand = len(pictorial_pages(page_texts, min_chars=100))
images, _ = extract_pdf_images_with_stats(pdf)
n_img = len(images)
assert (n_cand, n_img) == (1, len(values))
llm = _PageVisionLLM()
llm.settings = Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
import_extensions="md,txt,pdf",
images=True,
image_dir=str(tmp_path / "images"),
pdf_pictorial=True, # default on — the pass runs on the sparse page
)
events: list[tuple[int, int]] = []
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
assert (summary.files, summary.added) == (1, 1 + n_img)
assert (summary.pdfs_pictorial, summary.pdf_pictorial_pages) == (1, 1)
# The continuous counter: 1…C (total C) then C+1…C+N (total
# C+N) — no restart, never above the combined total.
assert events == (
[(i, n_cand) for i in range(1, n_cand + 1)]
+ [(n_cand + i, n_cand + n_img) for i in range(1, n_img + 1)]
)
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_fallback_span_climbs_into_embedded(
db, tmp_path: Path
) -> None:
"""The owner's stuck-label class: a garbage-text PDF's per-page
screenshot describes (the phase-133 fallback span — the span that
froze the label for minutes) report the rendered-page count
(``1…M`` with total ``M``) and hand their reported count to the
embedded pass, which continues the SAME counter (``M+1`` with
total ``M+N``) — every slow call has a position before it."""
root = tmp_path / "imgprog-fb"
root.mkdir()
_build_garbage_image_pdf(root / "garbage.pdf", (90,))
llm = _PageVisionLLM()
llm.settings = _pdf_settings(tmp_path, images=True)
events: list[tuple[int, int]] = []
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
assert summary.pdfs_screenshotted == 1
assert (summary.files, summary.added, summary.images_failed) == (1, 2, 0)
# 2 fallback pages (total 2) + 1 embedded image continuing at
# 3 (total 3) — ONE counter across the two spans.
assert events == [(1, 2), (2, 2), (3, 3)]
# Every slow call (2 pages + 1 image) was preceded by a
# position — the label never sits frozen on a busy run.
assert llm.vision_calls == 3
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_text_only_pdf_never_fires(
db, tmp_path: Path
) -> None:
"""A clean-text PDF with NO embedded images fires the hook NEVER,
with BOTH default-on toggles: the garbage gate stays closed (no
fallback span), the pictorial pass's deterministic gate finds zero
candidates (a no-op — the clean-text byte-identity class), and the
embedded pass's list is empty. The skipped spans report nothing
and advance the counter not at all (there is no vision call at
all)."""
root = tmp_path / "imgprog-none"
root.mkdir()
_build_image_pdf(root / "clean.pdf", ()) # dense text, zero images
llm = _PageVisionLLM()
llm.settings = Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
import_extensions="md,txt,pdf",
images=True,
image_dir=str(tmp_path / "images"),
pdf_pictorial=True, # default on — the pass runs and no-ops
)
events: list[tuple[int, int]] = []
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
assert (summary.files, summary.added) == (1, 1)
assert (summary.pdfs_screenshotted, summary.pdfs_pictorial) == (0, 0)
assert llm.vision_calls == 0 # no span ran — zero slow calls
assert events == []
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_toggle_off_reports_nothing(
db, tmp_path: Path
) -> None:
"""The ``images`` toggle OFF skips the embedded span entirely (the
prune guard keeps any existing image docs) — a PDF that WOULD
carry N embedded images reports nothing (a skipped span advances
the counter not at all) and indexes as text only."""
root = tmp_path / "imgprog-off"
root.mkdir()
_build_image_pdf(root / "off.pdf", (90, 130))
llm = _PageVisionLLM()
llm.settings = _pdf_settings(tmp_path, images=False)
events: list[tuple[int, int]] = []
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
assert (summary.files, summary.added) == (1, 1)
assert llm.vision_calls == 0
assert events == []
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_none_run_is_byte_identical(
db, tmp_path: Path
) -> None:
"""The regression pin: a hook-less run (the CLI/chat default —
``on_image_progress=None``, every call site a guarded no-op) lands
byte-identical to a run WITH the hook: the same documents (path,
title, content, digest, image flag, summary, context) and the same
summary counters. The hook observes; it never changes the run."""
# Both roots carry the SAME directory name — the source (the
# walk's root.name) is the document's provenance and part of the
# stored image context, so the comparison needs it equal.
root_a = tmp_path / "a" / "same_src"
root_a.mkdir(parents=True)
root_b = tmp_path / "b" / "same_src"
root_b.mkdir(parents=True)
values = (90, 130, 170, 210)
_build_image_pdf(root_a / "same.pdf", values)
_build_image_pdf(root_b / "same.pdf", values)
llm_a, llm_b = _PageVisionLLM(), _PageVisionLLM()
llm_a.settings = _pdf_settings(tmp_path, images=True)
llm_b.settings = _pdf_settings(tmp_path, images=True)
events: list[tuple[int, int]] = []
try:
# Run A (hook-less) first, snapshot its rows, clear the shared
# source name, then run B (hooked) — the rows line up 1:1.
summary_a = asyncio.run(import_sources([root_a], llm_a, session=db))
rows_a = _p152_rows(db, "same_src")
_cleanup_source(db, "same_src")
summary_b = asyncio.run(
import_sources(
[root_b], llm_b, session=db,
on_image_progress=lambda pos, total: events.append((pos, total)),
)
)
rows_b = _p152_rows(db, "same_src")
# The hook ran in run B (N image positions, the span's total)
# — and changed nothing observable.
assert events == [(i, len(values)) for i in range(1, len(values) + 1)]
assert _p152_summary_fields(summary_a) == _p152_summary_fields(summary_b)
assert rows_a == rows_b
finally:
_cleanup_source(db, "same_src")
class _InterleavingVisionLLM(_PageVisionLLM):
"""A ``_PageVisionLLM`` variant that records each vision call in
a SHARED event sequence (phase 152, task 04): the importer fires
the ``on_image_progress`` hook BEFORE the
``_index_one_embedded_image`` describe seam, so the sequence reads
hook, vision, hook, vision, … — the interleaving pin (the label
moves at the START of each slow call, never after it)."""
def __init__(self, events: list[tuple]) -> None:
super().__init__()
self.events = events
async def chat(self, messages, model=None): # pyright: ignore[reportIncompatibleMethodOverride]
reply = await super().chat(messages, model)
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
if isinstance(user, list):
self.events.append(("vision", self.vision_calls))
return reply
def test_on_image_progress_hook_fires_before_each_describe_call(
db, tmp_path: Path
) -> None:
"""Phase 152 (task 04): the hook fires BEFORE the corresponding
describe call (the contract: at the START of each slow call, never
after it) — the interleaving pinned by recording events in the
fake: for a PDF with N embedded images the sequence reads
hook(1,N), vision(1), hook(2,N), vision(2), …, hook(N,N),
vision(N) (the fake's vision call IS the describe seam the hook
precedes — the label moves while the describe is about to start,
so a busy run never reads frozen)."""
root = tmp_path / "imgprog-order"
root.mkdir()
values = (90, 130, 170)
_build_image_pdf(root / "order.pdf", values)
n = len(values)
events: list[tuple] = []
llm = _InterleavingVisionLLM(events)
llm.settings = _pdf_settings(tmp_path, images=True)
try:
summary = asyncio.run(
import_sources(
[root], llm, session=db,
on_image_progress=lambda pos, total: events.append(("image", pos, total)),
)
)
assert (summary.files, summary.added) == (1, 1 + n)
assert llm.vision_calls == n # one describe per image
# The interleaving: hook BEFORE each describe call, in order.
expected: list[tuple] = []
for i in range(1, n + 1):
expected.append(("image", i, n))
expected.append(("vision", i))
assert events == expected
finally:
_cleanup_source(db, root.name)
def test_on_image_progress_blank_gate_excludes_skipped_images_from_the_total(
db, tmp_path: Path
) -> None:
"""Phase 152 (task 04, the phase-151 interaction): images skipped
by the dedupe/blank gate are NOT counted in the hook's total —
the total is the work actually about to happen (the pass's
post-dedupe, post-gate list). A PDF with 2 real + 1 near-white
image (250 — the blank gate's class, mean ≥ 235): the gate
(default ON) drops the near-white one BEFORE the pass — the hook
fires for the 2 kept with total 2 (the skipped image burns no
vision call, no position); with the kill switch OFF
(``pdf_image_skip_blank=False`` — the pre-phase-151 behavior) the
near-white image is indexed too — the hook fires for all 3 with
total 3 (the total is the work about to happen under the knob's
state)."""
values_a = (90, 130, 250) # 250 = near-white (the blank gate's class)
values_b = (110, 150, 250) # distinct bytes — a different document set
# Both roots carry the SAME directory name — the source is part of
# the stored image context, so the runs line up 1:1 (the house
# cleanup idiom separates them).
root_a = tmp_path / "a" / "same_src"
root_a.mkdir(parents=True)
root_b = tmp_path / "b" / "same_src"
root_b.mkdir(parents=True)
_build_image_pdf(root_a / "blank.pdf", values_a)
_build_image_pdf(root_b / "blank.pdf", values_b)
llm_a = _PageVisionLLM()
llm_a.settings = _pdf_settings(tmp_path, images=True) # skip_blank default ON
events_a: list[tuple[int, int]] = []
try:
summary_a = asyncio.run(
import_sources(
[root_a], llm_a, session=db,
on_image_progress=lambda pos, total: events_a.append((pos, total)),
)
)
# The blank gate dropped the near-white image BEFORE the
# pass — the total is the 2 actually about to happen.
assert summary_a.pdf_images_blank_skipped == 1
assert (summary_a.files, summary_a.added) == (1, 3)
assert llm_a.vision_calls == 2
assert events_a == [(1, 2), (2, 2)]
# The knob OFF: the near-white image is indexed too — the
# total is all 3 (the pre-phase behavior, the work about to
# happen under the knob's state). Fresh source (the house
# cleanup idiom — run B's rows would otherwise be updates of
# run A's rows, same synthetic image paths).
_cleanup_source(db, "same_src")
llm_b = _PageVisionLLM()
llm_b.settings = Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
import_extensions="md,txt,pdf",
images=True,
image_dir=str(tmp_path / "images"),
pdf_pictorial=False,
pdf_image_skip_blank=False,
)
events_b: list[tuple[int, int]] = []
summary_b = asyncio.run(
import_sources(
[root_b], llm_b, session=db,
on_image_progress=lambda pos, total: events_b.append((pos, total)),
)
)
assert summary_b.pdf_images_blank_skipped == 0
assert (summary_b.files, summary_b.added) == (1, 4)
assert llm_b.vision_calls == 3
assert events_b == [(1, 3), (2, 3), (3, 3)]
finally:
_cleanup_source(db, "same_src")
+567 -15
View File
@@ -110,6 +110,56 @@ poll tree's new 5th branch (after failed, before the upload branches
— the 7-branch tree is now 8), the neutral ``applySyncCancelled``
(no modal, no banner, no error styling), and the
``initSyncButton`` re-attach of a ``cancelled`` run.
Phase 152 (task 01): the status contract gains the in-flight file's
VISION-image position — ``images_done`` (the 1-based index of the
image about to be described) + ``images_total`` (the file's total
images for this run — the three PDF vision spans share ONE continuous
counter per file, the importer's ``on_image_progress`` hook that
task 02/03 wire). The idle dict pins them 0/0; the per-file reset is
the pairing rule, extended — a ``progress`` call names the file AND
clears the image position, pinned through the runner's own ``_hook``
closure (the two-file fake below: the position fired through the
runner's own ``_image_hook`` closure while file A is in flight is
0/0 when file B is named — a file with no images must not inherit
the previous PDF's position); the clear rule mirrors the chunk
position — 0/0 at the overview/summaries transitions and in EVERY
terminal (success, failed, and cancelled — each pinned with a live
position in flight; task 04 moves every live position below onto the
runner's OWN closures, the phase-64/129 idiom), while the FILE
counts keep their final values (the convention split: a
sub-position of a file that's done clears with the file).
Phase 152 (task 03): the runner's THIRD paired closure —
``_image_hook`` (beside ``_hook`` and ``_chunk_hook``) — lands the
importer's ``on_image_progress`` pairs on the status, and the import
call now passes ``on_image_progress=_image_hook`` (the wiring pin:
the fakes receive the kwarg and record the live closure). The
frontend gains the "· image n/m" label PART: the running entry
destructures ``image = ""`` and the writer call becomes
``setSyncLabel(main, count, chunk, true, image)`` — the image part
rides ONLY the composed full string (box + title + running
aria-label — no new button span; the running button is the
spinner), the chunk part keeps its own span, and the bare builders
+ non-running calls pass nothing (both parts default "").
Phase 152 (task 04): the position pins move onto the runner's OWN
closures (the phase-64/129 idiom — the fakes fire the image pairs
through the ``on_image_progress`` kwarg, the ``_image_hook`` under
test, instead of standing in on the status object): a mid-run pair
(3/15, then 15/15) climbs on the status BETWEEN ticks while
``current_file`` and the chunk position keep their own values (the
THREE positions are independent sub-positions of the file), the
per-file reset, the overview/summaries transition clears, and the
FAILED + CANCELLED terminals all fire their live positions through
the closure too. The frontend source pins the ``fmtSyncPhaseLabel``
image part (present iff ``images_total > 0`` — the exact guard +
template), the ``overview`` / ``summaries`` branches' ``image: ""``
(unchanged labels), and ``setSyncLabel``'s composed ``full`` string
byte-for-byte (a status carrying both positions reads "… (2540/2540)
· chunk 125/126 · image 3/15" on the box + title + running
aria-label — the runtime string is behaviorally pinned in
tests/unit/test_frontend_sync_upload.py's node check).
"""
from __future__ import annotations
@@ -393,13 +443,16 @@ def test_sources_js_running_state_is_never_stale() -> None:
Phase 128 (task 01): the entry hands the label PARTS (the
phase-aware split — sync + status → fmtSyncPhaseLabel, the
fall-through → the byte-identical fmtSyncLabel) to the SINGLE
writer, setSyncLabel(main, count, chunk — phase 129: the chunk
part destructures with a "" default), which lands the main part,
writer, setSyncLabel(main, count, chunk, running, image — phase
129: the chunk part destructures with a "" default, phase 152:
the image part does too), which lands the main part,
the pinned count part (hidden when absent), the pinned chunk part
(phase 129 — hidden when absent), and the FULL untruncated label
on the button title + #sync-progress (A4) — the direct span /
title / announcer writes are GONE from the entry (they live in
setSyncLabel + setSyncProgress). Phase 149 (task 01): the writer call passes
on the button title + #sync-progress (A4 — phase 152: with the
image part appended after the chunk part when a PDF's vision span
is in flight; the image part gets no span of its own) — the
direct span / title / announcer writes are GONE from the entry
(they live in setSyncLabel + setSyncProgress). Phase 149 (task 01): the writer call passes
`running = true` (the detail box reveals with the full label),
and the button's accessible name rides in the writer too (the
full live label — the icon-only running button keeps a name);
@@ -414,8 +467,10 @@ def test_sources_js_running_state_is_never_stale() -> None:
assert "fmtSyncLabel(kind, currentFile, done, total)" in body
# …and the single writer lands the parts (A4 in it — the title is
# set for EVERY running label: the file-only rule is adjusted
# away) with the phase-149 `running` flag (the box reveals).
assert "setSyncLabel(main, count, chunk, true)" in body
# away) with the phase-149 `running` flag (the box reveals) +
# the phase-152 image part (rides only the composed full string —
# no span).
assert "setSyncLabel(main, count, chunk, true, image)" in body
assert body.count("setSyncLabel(") == 1, "the entry writes ONLY through the parts writer"
assert "syncBtn.title =" not in body, ("the title write lives in setSyncLabel (A4)")
assert "syncLabel.textContent" not in body, ("no direct span write — the parts")
@@ -433,6 +488,70 @@ def test_sources_js_running_state_is_never_stale() -> None:
)
def test_fmt_sync_phase_label_image_part_is_pinned_source_level() -> None:
"""Phase 152 (task 04): the frontend SOURCE pins the "· image
n/m" label part (the browser behavior is pinned in
tests/unit/test_frontend_sync_upload.py's node check — the REAL
extracted functions run against stub refs; this pin keeps the
template itself from drifting without a browser):
* the import fall-through is the ONLY reader of the status's
image fields, guarded on ``images_total > 0`` (the part is
present IFF ``images_total > 0`` — 0/0 hides it, the exact
guard + template + empty false-branch);
* the ``overview`` / ``summaries`` branches return ``image: ""``
(the position belongs to the import phase only — those labels
stay unchanged by the phase);
* the running entry destructures the part with a `""` default and
hands it to the single writer as the FIFTH argument (after the
`running` flag);
* ``setSyncLabel``'s composed ``full`` string appends the image
part LAST (after the chunk part) — so a status carrying both
positions reads exactly "Syncing… <file> (2540/2540) · chunk
125/126 · image 3/15" on the box, the title, and the running
aria-label (byte-for-byte; the docstring's worked example
carries the shape).
"""
js = _text(SOURCES_JS)
builder = _body(js, "fmtSyncPhaseLabel")
# The import fall-through: present IFF images_total > 0 — the
# exact guard + template (the phase-129 chunk part's sibling,
# same shape).
assert (
"status.images_total > 0\n"
" ? `· image ${status.images_done}/${status.images_total}`\n"
" : \"\";"
) in builder
assert "return { main, count, chunk, image };" in builder
# The overview branch: the unchanged label + the part absent.
assert (
'return { main: "Writing KB overview…", count: "", chunk: "", image: "" };'
) in builder
# The summaries branch: the unchanged label + the part absent
# (the phase's own (n/m) is its position).
assert "image: \"\", // the image position belongs to the import phase only" in builder
assert "`Summarizing folders…${folder}`" in builder
# The running entry: the part destructures with a `""` default
# and rides the writer call as the fifth argument.
entry = _body(js, "enterSyncRunningState")
assert 'const { main, count, chunk = "", image = "" }' in entry
assert "setSyncLabel(main, count, chunk, true, image)" in entry
# The writer: the composed full string — the image part LAST,
# after the chunk part (the "… · chunk 125/126 · image 3/15"
# shape the docstring's worked example carries).
writer = _body(js, "setSyncLabel")
assert (
"const full =\n"
" mainText +\n"
" (countText ? ` ${countText}` : \"\") +\n"
" (chunkText ? ` ${chunkText}` : \"\") +\n"
" (imageText ? ` ${imageText}` : \"\");"
) in writer
assert "· chunk 125/126 · image 3/15" in js, (
"the worked example (both positions at once) is documented"
)
def test_sources_js_terminal_states() -> None:
"""Terminal rendering: success → enabled + 'Synced HH:MM' (local
time of finished_at); failed → enabled + retry-ready 'Sync sources'
@@ -927,7 +1046,20 @@ class _GatedImport:
``_chunk_hook`` closure: pair one lands before the main park
(readable mid-run), and every later pair lands after the main
park + its own park (``chunk_release``), so each position is
readable on the status between ticks."""
readable on the status between ticks.
Phase 152 (task 03): the runner now ALWAYS passes the
image-position closure (task 03's wiring) — recorded for the
wiring pins.
Phase 152 (task 04): optionally ALSO fires the in-flight file's
image-position pairs (``image_pairs``) through the runner's OWN
``_image_hook`` closure (pair one before the main park —
readable mid-run; every later pair after the main park + its own
park ``image_release`` — the phase-129 chunk idiom, mirrored for
the vision-image position). With no pairs the fake never fires
it (the status image fields stay 0/0, like the chunk fields of a
no-batch import)."""
def __init__(
self,
@@ -937,6 +1069,8 @@ class _GatedImport:
fail: BaseException | None = None,
chunk_pairs: Sequence[tuple[int, int]] = (),
chunk_release: threading.Event | None = None,
image_pairs: Sequence[tuple[int, int]] = (), # phase 152 (task 04)
image_release: threading.Event | None = None, # phase 152 (task 04)
) -> None:
self.summary = summary
self.started = started
@@ -947,6 +1081,12 @@ class _GatedImport:
self.chunk_pairs = list(chunk_pairs)
self.chunk_release = chunk_release
self.chunk_hook_calls: list[tuple[int, int]] = []
# Phase 152 (task 03): the image-position hook the runner
# passes (recorded for the wiring pins).
self.image_hook: Callable[[int, int], None] | None = None
self.image_pairs = list(image_pairs)
self.image_release = image_release
self.image_hook_calls: list[tuple[int, int]] = []
async def __call__(
self,
@@ -958,6 +1098,7 @@ class _GatedImport:
session: object = None,
progress: Callable[[str, str, int, int], None] | None = None,
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
@@ -966,6 +1107,16 @@ class _GatedImport:
if progress is not None:
progress("repo", "notes/deep.md", 1, 3)
self.hook_calls.append(("repo", "notes/deep.md", 1, 3))
# Phase 152 (task 03): the runner now ALWAYS passes the
# image-position closure (task 03's wiring) — recorded for the
# wiring pins.
self.image_hook = on_image_progress
# Phase 152 (task 04): image pair one lands BEFORE the main
# park (readable mid-run — the phase-129 chunk-pair-one
# idiom, mirrored for the vision-image position).
if on_image_progress is not None and self.image_pairs:
on_image_progress(*self.image_pairs[0])
self.image_hook_calls.append(self.image_pairs[0])
if on_chunk_progress is not None and self.chunk_pairs:
# Phase 129 (task 03): the file's chunk-position pairs,
# fired through the runner's OWN hook closure (the closure
@@ -986,6 +1137,17 @@ class _GatedImport:
else:
self.started.set()
await asyncio.to_thread(self.release.wait) # park without freezing the worker loop
# Phase 152 (task 04): every later image pair lands after the
# main park + its own park (``image_release``), so each
# position is readable on the status between ticks. A wiring
# regression to None skips this branch and the pair tests
# time out.
if on_image_progress is not None:
for pair in self.image_pairs[1:]:
on_image_progress(*pair)
self.image_hook_calls.append(pair)
if self.image_release is not None:
await asyncio.to_thread(self.image_release.wait)
if self.fail is not None:
raise self.fail
return self.summary
@@ -1088,10 +1250,10 @@ def test_idle_status_pins_full_shape_including_progress_keys(
fresh_sync_status: None,
) -> None:
"""Idle: the three phase-64 progress keys ride along as null/0/0,
the two phase-129 chunk keys as 0/0, the four phase-98 phase keys
as null/0/0/0, and EVERY pre-existing key is unchanged — the full
response dict is pinned, so the current UI and every existing
consumer keep working."""
the two phase-129 chunk keys as 0/0, the two phase-152 image keys
as 0/0, the four phase-98 phase keys as null/0/0/0, and EVERY
pre-existing key is unchanged — the full response dict is pinned,
so the current UI and every existing consumer keep working."""
assert sync_api.sync_status() == {
"state": "idle",
"started_at": None,
@@ -1105,6 +1267,10 @@ def test_idle_status_pins_full_shape_including_progress_keys(
# (the dataclass defaults, no special-casing).
"current_chunk": 0,
"chunks_total": 0,
# Phase 152 (task 01): the image-position keys — 0/0 idle
# (the dataclass defaults, no special-casing).
"images_done": 0,
"images_total": 0,
# Phase 98 (task 01): the phase-machine keys — null/0/0/0 idle.
"phase": None,
"current_summary": None,
@@ -1145,13 +1311,19 @@ def test_mid_run_status_reports_current_file(
assert s["files_done"] == 0 and s["files_total"] == 0
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
# Phase 152 (task 01): the image position is 0/0 in the
# prelude too (no vision span yet).
assert s["images_done"] == 0 and s["images_total"] == 0
clone.release.set()
assert fake_import.started.wait(5.0), "the run never reached the import"
# Import phase: the hook's file is live on the status, and the
# phase machine names the phase (the summary span has not
# started — its counters are still 0/0); no embed batch has
# fired the chunk hook (this canned import passes no pairs),
# so the chunk position stays 0/0.
# so the chunk position stays 0/0; the image position stays
# 0/0 too (this canned import fires no image hook — task 03
# wires the runner's image closure, the importer's span is
# what would feed it).
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "import"
@@ -1161,6 +1333,7 @@ def test_mid_run_status_reports_current_file(
assert s["current_summary"] is None
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
assert s["images_done"] == 0 and s["images_total"] == 0
fake_import.release.set()
finally:
clone.release.set()
@@ -1178,7 +1351,8 @@ def test_mid_run_status_reports_current_file(
# folder step never fired the summary hook, so the summary counts
# stay 0/0 (the keep-final-counts convention keeps what fired);
# the chunk position clears with the file (phase 129 — it is a
# sub-position of a done file, never a keep-final counter).
# sub-position of a done file, never a keep-final counter), and
# so does the image position (phase 152 — same convention).
s = sync_api.sync_status()
assert s["state"] == "success"
assert s["phase"] is None
@@ -1187,6 +1361,7 @@ def test_mid_run_status_reports_current_file(
assert s["files_done"] == 1 and s["files_total"] == 3
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
assert s["images_done"] == 0 and s["images_total"] == 0
assert s["error"] is None
assert s["started_at"] is not None and s["finished_at"] is not None
@@ -1215,6 +1390,7 @@ def test_failed_terminal_clears_current_file_keeps_counts(
"failed: connection refused"
),
chunk_pairs=[(1, 38)], # phase 129: the live pair before the raise
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair before the raise
)
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
@@ -1231,6 +1407,13 @@ def test_failed_terminal_clears_current_file_keeps_counts(
assert s["current_file"] == "repo/notes/deep.md"
assert s["files_done"] == 1 and s["files_total"] == 3
assert s["current_chunk"] == 1 and s["chunks_total"] == 38
# Phase 152 (task 04): a live image position in flight when
# the run dies — fired through the runner's OWN _image_hook
# closure (image_pairs[0] — the importer's span would do the
# same); the failed terminal must clear it (the
# sub-position of an abandoned file, like the chunk
# position).
assert s["images_done"] == 3 and s["images_total"] == 15
fake_import.release.set()
finally:
clone.release.set()
@@ -1238,8 +1421,9 @@ def test_failed_terminal_clears_current_file_keeps_counts(
thread.join(10.0)
assert not thread.is_alive()
assert errors == []
# Wiring: the pair fired through the runner's own closure.
# Wiring: the pairs fired through the runner's OWN closure.
assert fake_import.chunk_hook_calls == [(1, 38)]
assert fake_import.image_hook_calls == [(3, 15)]
s = sync_api.sync_status()
assert s["state"] == "failed"
assert s["current_file"] is None # cleared in the terminal state
@@ -1251,6 +1435,11 @@ def test_failed_terminal_clears_current_file_keeps_counts(
# terminal too — the convention split, pinned (the FILE counts
# above keep their final values, the sub-position does not).
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
# Phase 152 (task 04): the image position clears in the FAILED
# terminal too — the convention split, pinned (the live 3/15 fired
# through the closure mid-run is gone, the FILE counts keep their
# final values).
assert s["images_done"] == 0 and s["images_total"] == 0
assert s["detail"] == {}
error = s["error"] or ""
assert "*****@aipi.example.com" in error # credentials masked
@@ -1468,6 +1657,357 @@ def test_chunk_position_clears_at_phase_transitions_and_terminal(
assert s["files_done"] == 1 and s["files_total"] == 3
# ---------- phase 152 (task 01 + task 04): the in-flight file's -----
# ---------- vision-image position (images_done / images_total) -------
#
# Same seams as the phase-129 block above. Task 04 pins every live
# position through the runner's OWN closures (the phase-64/129 idiom):
# the fakes fire the image pairs through the ``on_image_progress``
# kwarg — the ``_image_hook`` under test (task 03's wiring: the
# importer's span's exact path) — and drive the per-file ``_hook``
# reset (through the ``progress`` kwarg the fake import receives) and
# the transition/terminal clears.
def test_mid_run_status_reports_image_position_between_ticks(
fresh_sync_status: None,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""Phase 152 (task 04): the status carries the in-flight file's
VISION-image position — the pairs fire through the runner's OWN
``_image_hook`` closure (task 03's wiring — the importer's
``on_image_progress`` span's exact path), and the position CLIMBS
between ticks (3/15 → 15/15 — the PDF's vision span moving, the
owner's stuck label, now moving) while ``current_file`` and the
chunk position (14/38, a concurrent embed batch) keep their own
values — the THREE positions are independent sub-positions of the
file; the prelude reports 0/0, and the success terminal clears
BOTH sub-positions (image + chunk) to 0/0 while the FILE counts
keep their final values (the convention split)."""
clone = _GatedClone(threading.Event(), threading.Event())
image_release = threading.Event()
fake_import = _GatedImport(
ImportSummary(files=3, added=1, updated=1, unchanged=1),
threading.Event(),
threading.Event(),
chunk_pairs=[(14, 38)], # a concurrent embed batch (independent)
image_pairs=[(3, 15), (15, 15)], # two describe calls of a 15-image span
image_release=image_release,
)
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
thread, errors = _start_run()
try:
assert clone.started.wait(5.0), "the run never reached the clone phase"
# Prelude: every position 0/0 (no file, no batch, no image
# yet — alongside the bare-label pins of the phase-64 test).
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["current_file"] is None
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
assert s["images_done"] == 0 and s["images_total"] == 0
clone.release.set()
assert fake_import.started.wait(5.0), "the run never reached the import"
# Import phase: the file is live (the progress hook's call),
# the chunk position 14/38 (the embed batch), and the image
# position 3/15 (the vision span) — the THREE positions are
# independent sub-positions of the same file.
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "import"
assert s["current_file"] == "repo/notes/deep.md"
assert s["files_done"] == 1 and s["files_total"] == 3
assert s["current_chunk"] == 14 and s["chunks_total"] == 38
assert s["images_done"] == 3 and s["images_total"] == 15
# Release the main park: the import fires image pair two
# (15/15 — the span's last describe call) and parks on
# image_release.
fake_import.release.set()
_poll_until(
lambda: sync_api.sync_status()["images_done"] == 15,
what="the second image pair on the status",
)
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "import"
# The image position CLIMBED (3 → 15 — monotonic, no
# backwards jump, never above the total) ... the FILE and
# chunk positions keep their own values (independent).
assert s["images_done"] == 15 and s["images_total"] == 15
assert s["current_file"] == "repo/notes/deep.md"
assert s["files_done"] == 1 and s["files_total"] == 3
assert s["current_chunk"] == 14 and s["chunks_total"] == 38
image_release.set()
finally:
clone.release.set()
fake_import.release.set()
image_release.set()
thread.join(10.0)
assert not thread.is_alive()
assert errors == [], f"the runner raised: {errors!r}"
# Wiring: every pair fired through the runner's OWN closure (the
# status reads above came through it; a wiring regression to
# None would have skipped the firing branch and timed out).
assert fake_import.chunk_hook_calls == [(14, 38)]
assert fake_import.image_hook_calls == [(3, 15), (15, 15)]
# Success terminal: BOTH sub-positions clear (0/0) ...
s = sync_api.sync_status()
assert s["state"] == "success"
assert s["phase"] is None
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
assert s["images_done"] == 0 and s["images_total"] == 0
# ... while the FILE counts keep their final values (the
# convention split, pinned alongside the phase-64 terminal pin).
assert s["current_file"] is None
assert s["files_done"] == 1 and s["files_total"] == 3
class _TwoFileImport:
"""The phase-152 (task 01 + task 04) fake: fires the runner's
OWN progress hook for file A, then file A's image position through
the runner's OWN ``on_image_progress`` closure (task 03's wiring —
what the importer's span would do), parks (the test reads the
live position on the status), fires the progress hook again for
file B (the runner's ``_hook`` under test must reset the image
position to 0/0 — the pairing rule, extended), parks again, and
completes."""
def __init__(self, summary: ImportSummary) -> None:
self.summary = summary
self.parked_a = threading.Event() # parked after file A's progress
self.release_a = threading.Event() # test releases → file B's progress
self.parked_b = threading.Event() # parked after file B's progress
self.release_b = threading.Event()
self.hook_calls: list[tuple[str, str, int, int]] = []
self.image_hook_calls: list[tuple[int, int]] = [] # phase 152 (task 04)
async def __call__(
self,
sources: list[Path],
llm: object,
*,
prune: bool = False,
limit: int | None = None,
session: object = None,
progress: Callable[[str, str, int, int], None] | None = None,
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
) -> ImportSummary:
if progress is not None:
progress("repo", "docs/a.pdf", 1, 2)
self.hook_calls.append(("repo", "docs/a.pdf", 1, 2))
# Phase 152 (task 04): file A's vision span is in flight —
# its image position (3/15) fires through the runner's OWN
# on_image_progress closure (task 03's wiring), then the fake
# parks.
if on_image_progress is not None:
on_image_progress(3, 15)
self.image_hook_calls.append((3, 15))
self.parked_a.set()
await asyncio.to_thread(self.release_a.wait)
if progress is not None:
# The SECOND progress call names file B — the closure under
# test must reset the image position the closure fired
# while file A was in flight (a file with no images must
# not inherit the previous PDF's position).
progress("repo", "docs/b.md", 2, 2)
self.hook_calls.append(("repo", "docs/b.md", 2, 2))
self.parked_b.set()
await asyncio.to_thread(self.release_b.wait)
return self.summary
def test_image_position_resets_per_file(
fresh_sync_status: None,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""Phase 152 (task 01 + task 04): the pairing rule, extended — a
``progress`` call names the file AND clears the image position;
every image position until the next ``progress`` call belongs to
it. File A's live position (3/15, fired through the runner's OWN
``_image_hook`` closure while parked on file A — task 04: the
phase-64/129 idiom, the importer's image hook's exact path) is
0/0 the moment the runner's own ``_hook`` names file B: a file
with no images must not inherit the previous PDF's position
(unlike the chunk position, not every file embeds chunks). The
success terminal keeps the position at 0/0 and the FILE counts
keep their final values (the convention split)."""
clone = _GatedClone(threading.Event(), threading.Event())
fake_import = _TwoFileImport(ImportSummary(files=2, added=1, updated=1))
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
thread, errors = _start_run()
try:
assert clone.started.wait(5.0), "the run never reached the clone phase"
clone.release.set()
assert fake_import.parked_a.wait(5.0), "the run never parked on file A"
# File A is named by the runner's own _hook (a wiring
# regression to None would leave current_file null here).
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "import"
assert s["current_file"] == "repo/docs/a.pdf"
assert s["files_done"] == 1 and s["files_total"] == 2
# File A's vision span is in flight: its image position is
# live on the status — fired through the runner's OWN
# _image_hook closure (the fake fired it before parking;
# a wiring regression to None would leave it 0/0 here).
assert s["images_done"] == 3 and s["images_total"] == 15
# Release file A: the runner's _hook names file B — the image
# position must reset to 0/0 in the same call (the pairing
# rule, extended; the closure under test, driven through the
# fake's progress kwarg).
fake_import.release_a.set()
assert fake_import.parked_b.wait(5.0), "the run never parked on file B"
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "import"
assert s["current_file"] == "repo/docs/b.md"
assert s["files_done"] == 2 and s["files_total"] == 2
# RESET per file: file B (no images) does not inherit file
# A's 3/15.
assert s["images_done"] == 0 and s["images_total"] == 0
fake_import.release_b.set()
finally:
clone.release.set()
fake_import.release_a.set()
fake_import.release_b.set()
thread.join(10.0)
assert not thread.is_alive()
assert errors == [], f"the runner raised: {errors!r}"
# Wiring: BOTH progress calls went through the runner's own
# closure (the reset above was its doing), and the image pair
# went through the runner's own _image_hook closure.
assert fake_import.hook_calls == [
("repo", "docs/a.pdf", 1, 2),
("repo", "docs/b.md", 2, 2),
]
assert fake_import.image_hook_calls == [(3, 15)]
# Success terminal: the image position stays 0/0 (cleared in the
# terminal too — every terminal clears it) and the FILE counts
# keep their final values (the convention split).
s = sync_api.sync_status()
assert s["state"] == "success"
assert s["phase"] is None
assert s["current_file"] is None
assert s["files_done"] == 2 and s["files_total"] == 2
assert s["images_done"] == 0 and s["images_total"] == 0
def test_image_position_clears_at_phase_transitions_and_terminal(
fresh_sync_status: None,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""Phase 152 (task 01 + task 04): the image position is CLEARED
(0/0) at every point the import phase ends. This run's summary
changes the KB, so BOTH post-import transitions run and are
pinned: the import→``"overview"`` transition and the
import→``"summaries"`` transition (the changed-KB branch) each
clear the live position (3/15, fired mid-import through the
runner's OWN ``_image_hook`` closure — task 04: the
phase-64/129 idiom, the importer's span's exact path) to 0/0,
and the success terminal keeps it 0/0 — through the whole span
the FILE counts keep their final values (the convention split:
the file counts survive for the result line's context, the image
position is a sub-position of a file that's done)."""
clone = _GatedClone(threading.Event(), threading.Event())
import_release = threading.Event()
overview_started = threading.Event()
overview_release = threading.Event()
folders_started = threading.Event()
folders_release = threading.Event()
fake_import = _GatedImport(
ImportSummary(files=3, added=1, updated=1, unchanged=1),
threading.Event(),
import_release,
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair to clear
)
_patch_sync_seams(monkeypatch, tmp_path, fake_import, clone)
# Park the post-import steps (re-monkeypatched over the seams
# helper's no-ops): the test reads the status DURING the
# "overview" phase and DURING the "summaries" phase — the two
# import-phase-ending transitions the task pins (the same shape
# as the phase-129 chunk-position transition test above).
async def gated_overview(llm: object, session: object = None) -> bool:
overview_started.set()
await asyncio.to_thread(overview_release.wait)
return True
async def gated_folders(
db: object,
llm: object,
*,
skip: bool = False,
only_missing: bool = False,
changed_docs: Sequence[tuple[str, str]] | None = None, # phase 127
on_progress: Callable[[int, int, str, str], None] | None = None,
) -> dict[str, int]:
folders_started.set()
await asyncio.to_thread(folders_release.wait)
return {"generated": 0, "failed": 0, "pruned": 0}
monkeypatch.setattr(sync_api, "regenerate_overview", gated_overview)
monkeypatch.setattr(sync_api, "generate_folder_summaries", gated_folders)
thread, errors = _start_run()
try:
assert clone.started.wait(5.0), "the run never reached the clone phase"
clone.release.set()
assert fake_import.started.wait(5.0), "the run never reached the import"
# Mid-import: the file is live; its image position went live
# too — fired through the runner's OWN _image_hook closure
# (image_pairs[0] — the importer's span would do the same).
s = sync_api.sync_status()
assert s["phase"] == "import"
assert s["images_done"] == 3 and s["images_total"] == 15
# Release the import: it returns (added+updated > 0) → the
# phase machine moves to "overview" and the position clears.
import_release.set()
assert overview_started.wait(5.0), "the run never reached the overview"
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "overview"
# CLEARED at the import→overview transition (0/0) ... the file
# counts KEEP their final values (the convention split).
assert s["images_done"] == 0 and s["images_total"] == 0
assert s["files_done"] == 1 and s["files_total"] == 3
overview_release.set()
# ... and the changed-KB branch moves to "summaries" (the
# second import-phase-ending transition — cleared there too,
# idempotently) before parking.
assert folders_started.wait(5.0), "the run never reached the summaries"
s = sync_api.sync_status()
assert s["state"] == "running"
assert s["phase"] == "summaries"
assert s["images_done"] == 0 and s["images_total"] == 0
assert s["files_done"] == 1 and s["files_total"] == 3
folders_release.set()
finally:
clone.release.set()
import_release.set()
overview_release.set()
folders_release.set()
thread.join(10.0)
assert not thread.is_alive()
assert errors == [], f"the runner raised: {errors!r}"
assert fake_import.image_hook_calls == [(3, 15)]
# Success terminal: still 0/0 (the terminal clears too — same
# spots as current_file), file counts kept.
s = sync_api.sync_status()
assert s["state"] == "success"
assert s["phase"] is None
assert s["current_file"] is None
assert s["images_done"] == 0 and s["images_total"] == 0
assert s["files_done"] == 1 and s["files_total"] == 3
# ---------- phase 134 (task 01): the cooperative cancel (5th state) ----------
#
# The cancel runs are driven through the REAL ``start_sync`` on a
@@ -1629,11 +2169,17 @@ def test_cancel_endpoint_stops_the_in_flight_run(
ImportSummary(files=3, added=1, updated=1, unchanged=1),
threading.Event(),
threading.Event(),
image_pairs=[(3, 15)], # phase 152 (task 04): the live pair before the cancel
)
_patch_sync_seams(monkeypatch, tmp_path, fake_import, _fast_clone)
def act(loop: asyncio.AbstractEventLoop) -> dict[str, str]:
assert fake_import.started.wait(5.0), "the run never reached the import"
# Phase 152 (task 04): the live image position in flight when
# the cancel converts was fired through the runner's OWN
# _image_hook closure (image_pairs[0] — the fake fired it
# before parking, like the importer's span would); the
# cancelled terminal must clear it.
return loop.run_until_complete(sync_api.cancel_sync())
try:
@@ -1665,6 +2211,11 @@ def test_cancel_endpoint_stops_the_in_flight_run(
assert s["files_done"] == 1 and s["files_total"] == 3 # final counts kept
assert s["summaries_done"] == 0 and s["summaries_total"] == 0
assert s["current_chunk"] == 0 and s["chunks_total"] == 0
# Phase 152 (task 04): the image position clears in the CANCELLED
# terminal too — every terminal clears it (the live 3/15 fired
# through the closure in flight is gone; the FILE counts above
# keep their final values).
assert s["images_done"] == 0 and s["images_total"] == 0
# PLAN §9: the ample-logging line for the converted cancel.
assert any(
r.getMessage()
@@ -1879,6 +2430,7 @@ def test_event_set_during_import_cancels_before_the_lite_spans(
session: object = None,
progress: Callable[[str, str, int, int], None] | None = None,
on_chunk_progress: Callable[[int, int], None] | None = None, # phase 129
on_image_progress: Callable[[int, int], None] | None = None, # phase 152
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106