chore(agents): phase roadmap from TODO.md, 3 phases
Converts the three open TODO items into executable phases (Protocol B append, numbered after the existing 149): - 150_drop_sync_result — retire span.sync-result; #sync-progress becomes the single sync surface and the sole aria-live region (TODO L5) - 151_pdf_image_extraction — dedupe PDF image occurrences (46 placements -> 15 unique in podman-systemd.pdf), composite /SMask so masked images are not blank, and skip near-white decorative underlays (TODO L4) - 152_sync_progress_image_pass — report the PDF vision-image position so the sync label stops freezing at the last chunk (TODO L3) Both reported bugs were reproduced against test_sources/podman-systemd.pdf before authoring; root causes are recorded in the phase overviews. The TODO file is cleared now that its items live in .agents/phases/. LLM-Generated: true
This commit is contained in:
@@ -0,0 +1,48 @@
|
||||
# Phase 152 — the sync label keeps moving while images are described and embedded
|
||||
|
||||
**Source:** `TODO.md` L3 — "The sync status on the RAG page gets \"stuck\" when it's embedding images. For example, when syncing test_sources/podman-systemd.pdf the status was stuck on `Syncing… podman-systemd.unit — Podman documentation/podman-systemd.unit — Podman documentation.pdf (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a (TODO-derived phase, Protocol B append).
|
||||
**Context:** the sync status machine `app/api/sync.py` — `SyncStatus` (L276–300: `state`, `current_file`/`files_done`/`files_total` (phase 64), `phase`/`current_summary`/`summaries_done`/`summaries_total` (phase 98), `current_chunk`/`chunks_total` (phase 129)), the `GET /api/sync/status` payload (L330–360), and `_run_sync`'s paired progress closures `_hook` (L553 — names the file) and `_chunk_hook` (L563 — the in-flight file's embed-batch position, fed by `LLMClient.embed`'s `on_progress`). The importer threads the chunk hook as `import_sources(on_chunk_progress=…)` → `_index_file(on_chunk=…)` → `_index_pdf_file(on_chunk=…)` → `_replace_content_chunks(on_chunk=…)`. The importer's own docstring states the gap this phase closes (L684–690): *"Only the content-chunk embeds report … the PDF's embedded-image docs (separate documents, not the walked file) do not report either."*
|
||||
|
||||
**Reproduced:** the owner's `podman-systemd.pdf` is the LAST file of a 2,540-file run. Its text chunks finish (the label's last write: `· chunk 125/126`), then the PDF's vision spans run — `_index_pdf_embedded_images` (46 image occurrences pre-phase-151, each a `describe_image` chat call + an embed + a summary), `_pictorial_pdf_pass` (L1715 — one render + one describe call per sparse page), `_fallback_page_screenshots` (L1635 — one describe call per page of a garbage-text PDF). **None of them reports progress**, so the label sits frozen on the last chunk position for minutes while the run is very much alive. The phase-149 box made the freeze plainly visible (it is now the big surface the user reads).
|
||||
|
||||
**Locked (owner decision, chat):** the fix is a real progress signal for the vision spans (not a spinner/heartbeat that hides the stall), reported as a per-image position alongside the existing chunk position.
|
||||
|
||||
## Objective
|
||||
During a PDF's image spans the status reports `image n/m`, the UI renders it, and the label advances once per vision call — a long sync is visibly progressing, and a genuinely stalled run is distinguishable from a busy one.
|
||||
|
||||
## Dependencies
|
||||
- `151_pdf_image_extraction` (todo) — its dedupe + blank gate shrink the span this phase reports on, and its `extract_pdf_images_with_stats` is the call that establishes the span's `total`. Report order matters: the totals this phase reports are post-dedupe/post-gate counts.
|
||||
- `150_drop_sync_result` (todo) — the label surface is `#sync-progress` (the phase-149 box is the only place the running label renders; the button is the spinner).
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **The hook (task 02):** `import_sources(..., on_image_progress: Callable[[int, int], None] | None = None)`, threaded exactly like `on_chunk_progress`: `import_sources` → `_index_file(on_image=…)` → `_index_pdf_file(on_image=…)` → the three vision spans (`_index_pdf_embedded_images`, `_pictorial_pdf_pass`, `_fallback_page_screenshots`). Contract (the phase-129 pairing rule, extended): the hook is called **immediately before each vision describe call** with `(position_1based, total)` where `total` is that span's count — the extracted-image count for the embedded pass (post-dedupe, post-blank-gate: the work actually about to happen), the candidate-page count for the pictorial pass, the rendered-page count for the screenshot fallback. The three spans of ONE file are reported as ONE continuous counter: the pass that runs second starts where the first ended (`base = images_done_so_far`), so the label climbs `1/N … N/N` across the file's whole vision span instead of restarting. `None` (every existing caller — the CLI, the chat path) is a zero-cost guarded no-op and the run is byte-identical (the phase-129 rule).
|
||||
- **The status fields (task 01):** `SyncStatus` gains `images_done: int = 0` / `images_total: int = 0`, reported by `GET /api/sync/status` as `images_done` / `images_total`. Clear/reset rules mirror the chunk position (a sub-position of a file): cleared at the `phase = "overview"` / `phase = "summaries"` transitions and in every terminal (success/failed/cancelled), and **reset to 0/0 by the per-file `_hook`** — the pairing rule is "a `progress` call names the file and clears the image position; every image position until the next `progress` call belongs to it". (The chunk position does not need this because every file embeds chunks; a file with no images must not inherit the previous PDF's position.)
|
||||
- **The label (task 03):** `sources.js` `fmtSyncPhaseLabel` appends `· image n/m` when `status.images_total > 0` (the phase-129 chunk-part language: a position reads as data, mono in the box). `setSyncLabel` gains a fifth parameter `imageText = ""` appended to `full` (after the chunk part) — it rides the title, the box, and the running `aria-label`. **No new button span** is added: while running the label parts are hidden (phase 149 makes the button the spinner), so the parts DOM stays as-is and the position lives only in the composed `full` string. The chunk part keeps its own slot (the two positions are distinct and both may show: `… (2540/2540) · chunk 125/126 · image 3/15`).
|
||||
- **ASSUMPTION A1 (owner-locked):** a real per-image position, not an elapsed-time heartbeat.
|
||||
- **ASSUMPTION A2:** counts only, not the image identity — the file label already names the PDF and the synthetic `…#page=N#img=i.png` path is noise in a label.
|
||||
- **ASSUMPTION A3:** the three PDF vision spans share ONE counter per file (continuous, not three restarts) — the user cares that the file is progressing, not which span it is in.
|
||||
- **ASSUMPTION A4:** standalone image files (one image per file) get no sub-position — `1/1` per file would be noise; the file count already covers them.
|
||||
- **ASSUMPTION A5:** no new `phase` value — the image spans are inside `phase: "import"` (the phase machine stays as phase 98 defined it).
|
||||
|
||||
## Tasks
|
||||
1. `01_status_image_fields.md` — `SyncStatus.images_done/images_total`, the reset/clear rules, the `/api/sync/status` payload + docstring.
|
||||
2. `02_importer_image_hook.md` — `on_image_progress` threaded through the importer's three PDF vision spans with the continuous per-file counter.
|
||||
3. `03_label_and_wiring.md` — the `_image_hook` closure in `_run_sync`, the `sources.js` label part + title/aria, the phase-129-style comment updates.
|
||||
4. `04_unit_tests.md` — the status machine pins (idle 0/0, mid-run position, the per-file reset, every terminal clears) + the importer hook pairing.
|
||||
5. `05_e2e_image_progress.md` — the dedicated Playwright suite: the label advances through a PDF's image span (slow-LLM proxy), never frozen.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_sync_button.py`'s full-status-dict pin gains the two keys (idle `0/0`); the terminal-clears and per-file-reset rules pinned through the runner's own closures (the phase-64/129 idiom); the importer hook pinned with `tests/fakes.py::FakeEmbedder` — a PDF with N images calls the hook N times with `(1,N)…(N,N)`, the three spans continue one counter, `None` costs nothing (the hook-less run is byte-identical).
|
||||
- Integration: a sync run over a fixture PDF reports a monotonically increasing `images_done` across polls (no backwards jumps, never above `images_total`).
|
||||
- Coverage: **>90%** on new/modified code (`app/api/sync.py`, `app/rag/importer.py`, the JS is pinned by the source-level suites + E2E).
|
||||
- E2E: `uv run pytest tests/e2e/test_sync_image_progress.py -v --no-cov` in isolation (the `tests/e2e/slow_llm.py` delay proxy — every describe call outlives the 2 s poll, so each position is observed deterministically).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] during a PDF's image span the status reports `images_done`/`images_total` climbing 1→N and the UI shows `· image n/m` in the box (and the button `title`/`aria-label`)
|
||||
- [ ] the label is never frozen for more than one vision call's duration on the fixture PDF (the E2E asserts at least 3 distinct positions observed)
|
||||
- [ ] the image position clears at the overview/summaries transitions and in every terminal; a file with no images never shows a stale position
|
||||
- [ ] `GET /api/sync/status` is additive only — every pre-existing key unchanged (the full-dict pin)
|
||||
- [ ] a hook-less caller (the CLI import path) is byte-identical to pre-phase
|
||||
- [ ] `uv run pytest` green; the dedicated E2E green in isolation; `uv run ruff check . && uv run pyright` clean; `app/` coverage >90%
|
||||
- [ ] one atomic commit per AGENTS.md rule 8 (`--no-gpg-sign`)
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 01 — `SyncStatus` gains `images_done` / `images_total`
|
||||
|
||||
**Phase:** `152_sync_progress_image_pass` · **Source:** `TODO.md:3` — "The sync status on the RAG page gets \"stuck\" when it's embedding images … the status was stuck on `Syncing… … (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The sync status carries the in-flight file's vision-image position, with the same clear/reset discipline the chunk position already has.
|
||||
|
||||
## Work
|
||||
1. `app/api/sync.py` — `SyncStatus` (L276–300): add
|
||||
```python
|
||||
# Phase 152 (task 01): the in-flight file's VISION-image position —
|
||||
# the 1-based index of the image about to be described and the
|
||||
# file's total images for this run (the importer's
|
||||
# on_image_progress hook; the three PDF vision spans share ONE
|
||||
# continuous counter per file). 0/0 before the first image, during
|
||||
# the prelude/overview/summaries phases, and in terminal states.
|
||||
images_done: int = 0
|
||||
images_total: int = 0
|
||||
```
|
||||
and extend the dataclass docstring with a phase-152 paragraph in the file's established voice (it is a SUB-position of a file, like the chunk position: the keep-final-counts convention covers FILES only, so this clears with the file).
|
||||
2. Clear/reset rules — exactly the points `current_chunk`/`chunks_total` clear today (grep `_status.current_chunk = 0` — the `phase = "overview"` transition ~L597, the `phase = "summaries"` transitions ~L650/671, the terminal paths ~L713/717/752/755/770/774, and the run start ~L462): set the two new fields to `0` at the SAME sites. In addition, the per-file `_hook` closure (L553) resets them to `0/0` — the pairing rule is "a `progress` call names the file and clears the image position; every image position until the next `progress` call belongs to it" (unlike the chunk position, a file may have NO images at all, and it must not inherit the previous PDF's position).
|
||||
3. `GET /api/sync/status` (the payload dict, L330–360): add `"images_done"` / `"images_total"` beside `current_chunk`/`chunks_total`, with the same style of inline comment; extend the endpoint docstring with the phase-152 sentence (additive only — every pre-existing key keeps its name, type, and meaning).
|
||||
4. Module docstring: add the phase-152 paragraph after the phase-129 chunk-position paragraph (the file documents each phase's contribution in order).
|
||||
5. ASSUMPTION A5: no new `phase` value — the image spans live inside `phase: "import"`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the full idle-dict pin in `tests/unit/test_sync_button.py` gains the two keys (`0`/`0`); a terminal-clears test and a per-file-reset test (drive the runner's closures directly, the phase-64/129 idiom).
|
||||
- Coverage: **>90%** on the new lines.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/sync/status` returns `images_done` / `images_total` (idle `0`/`0`) and every pre-existing key is unchanged
|
||||
- [ ] the fields clear at the overview/summaries transitions and in success/failed/cancelled, and reset per file
|
||||
- [ ] `uv run pytest tests/unit/test_sync_button.py -q` green; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 02 — the importer's `on_image_progress` hook across the PDF vision spans
|
||||
|
||||
**Phase:** `152_sync_progress_image_pass` · **Source:** `TODO.md:3` — "The sync status on the RAG page gets \"stuck\" when it's embedding images … the status was stuck on `Syncing… … (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Every vision describe call in a file's import reports `(position, total)` before it happens, so the caller can show the span moving.
|
||||
|
||||
## Work
|
||||
1. `app/rag/importer.py` — thread the callback exactly like `on_chunk_progress` (the phase-129 route):
|
||||
- `import_sources(..., on_image_progress: Callable[[int, int], None] | None = None)` (L636–650) — document it in the docstring next to the `on_chunk_progress` paragraph, and CORRECT that paragraph's closing sentence ("the PDF's embedded-image docs … do not report either") — it no longer holds;
|
||||
- `_index_file(..., on_image: Callable[[int, int], None] | None = None)` (L1050) → `_index_pdf_file(..., on_image=…)` (L1372);
|
||||
- the three spans, each called with the file's continuous counter (ASSUMPTION A3 — one counter per file, spans continue each other):
|
||||
* `_index_pdf_embedded_images(..., on_image=…)` (L1924): `total = len(images)` from `extract_pdf_images_with_stats` (phase 151 — post-dedupe, post-blank-gate: the work actually about to happen); call `on_image(base + i + 1, base + total)` **immediately before** each `_describe_or_skip` (so the label moves at the START of each slow call, not after it), where `base` is the count already reported for this file;
|
||||
* `_pictorial_pdf_pass(..., on_image=…)` (L1715): `total = len(candidate_pages)` (the deterministic sparse-page gate's list, already computed before the loop); same call site rule;
|
||||
* `_fallback_page_screenshots(..., on_image=…)` (L1635): `total = len(rendered pages)`; same rule;
|
||||
- the counter handoff: `_index_pdf_file` keeps a small local `reported = 0` and passes it down / adds each span's returned count (each span returns the number of images it reported — reuse its existing return shape or return the count; the executor picks the minimal clean shape). A span that is skipped (toggle off, no candidates, usable text) reports nothing and does not advance the counter;
|
||||
- `None` stays a zero-cost guarded no-op: every call site is `if on_image is not None:` (the phase-129 rule — a hook-less run is byte-identical, so the CLI and chat paths are untouched).
|
||||
2. Update the module docstring's progress paragraph (L287–291) to name the image hook, and the `_index_pdf_embedded_images` docstring's bullet list.
|
||||
3. ASSUMPTION A4: standalone image files (`_index_image_file`, L2989) get NO sub-position — one image per file, `1/1` would be noise; the file count already covers them.
|
||||
4. ASSUMPTION A2: counts only — the hook carries no image identity (the file label already names the PDF).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/test_importer.py` / the image-pass tests' `tests/fakes.py::FakeEmbedder` idiom): a fixture PDF with N embedded images → the hook fires N times with `(1,N)…(N,N)`; a PDF with a pictorial span + an embedded span → ONE continuous counter (no restart, never above the combined total); a text-only PDF → the hook never fires; `on_image_progress=None` → the import result is byte-identical to pre-phase (the regression pin).
|
||||
- The hook never raises into the pipeline: a hook that raises is a caller bug (the phase-64 rule — "the hooks in this repo only assign fields", no try/except), so do NOT add defensive handling; keep the closure assignment-only.
|
||||
- Coverage: **>90%** on the new branches (each span, the skip paths, the `None` path).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the hook fires once per vision describe call, 1-based, with the span's total, before the call
|
||||
- [ ] the three PDF spans share one continuous per-file counter
|
||||
- [ ] a hook-less import is byte-identical to pre-phase
|
||||
- [ ] `uv run pytest tests/unit -q` green; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 03 — wire the hook to the status and render `· image n/m` in the label
|
||||
|
||||
**Phase:** `152_sync_progress_image_pass` · **Source:** `TODO.md:3` — "The sync status on the RAG page gets \"stuck\" when it's embedding images … the status was stuck on `Syncing… … (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The importer's image hook lands on the status object, and the Sources page renders the position — the label advances once per vision call.
|
||||
|
||||
## Work
|
||||
1. `app/api/sync.py` — `_run_sync`: add the third paired closure beside `_hook` (L553) and `_chunk_hook` (L563):
|
||||
```python
|
||||
def _image_hook(done: int, total: int) -> None:
|
||||
_status.images_done = done
|
||||
_status.images_total = total
|
||||
```
|
||||
pass it into the import call (L580): `import_sources(..., on_image_progress=_image_hook)`; comment it in the file's established voice (the three hooks are paired: `progress` names the file and resets the image position, `on_chunk_progress` reports the embed-batch position, `on_image_progress` reports the vision-image position).
|
||||
2. `frontend/assets/sources.js`:
|
||||
- `fmtSyncPhaseLabel(status)` (L896–919): the import branch gains `image` — `status.images_total > 0 ? \`· image ${status.images_done}/${status.images_total}\` : ""` — and the `overview` / `summaries` branches return `image: ""` (the position belongs to the import phase only);
|
||||
- `enterSyncRunningState` (L964): destructure `{ main, count, chunk = "", image = "" }` and pass the new argument to the writer;
|
||||
- `setSyncLabel(mainText, countText = "", chunkText = "", running = false, imageText = "")` (L809, post-phase-150): the composed `full` string becomes `main + (count ? " " + count : "") + (chunk ? " " + chunk : "") + (image ? " " + image : "")` — so the box, the button `title`, and the running `aria-label` all carry `Syncing… <file> (2540/2540) · chunk 125/126 · image 3/15` (the phase-149 box is the surface that shows it; the button's label parts stay as they are — no new span, the running button is the spinner);
|
||||
- update the function's docstring + the phase-129/149 comment blocks that enumerate the label parts (the file cites the "· chunk n/m" contract in ~10 comments — name the image part beside it, do not restate the whole contract).
|
||||
3. No CSS change: the box already wraps mono text with `overflow-wrap: anywhere`; the extra part is just more text.
|
||||
4. ASSUMPTION A2/A3: counts only, one continuous per-file counter (the importer owns the continuity — the UI just renders what it is told).
|
||||
5. ASSUMPTION: no new button label span — while running the parts are hidden (phase 149), so the position rides the composed string only.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the source-level pins in `tests/unit/test_sync_button.py` / `test_frontend_sync_upload.py` gain the image part (the `fmtSyncPhaseLabel` contract: `images_total > 0` → the part present, `0` → absent; the `overview`/`summaries` branches never carry it; the composed `full` string pinned byte-for-byte for a status with both positions).
|
||||
- Integration: a mocked sync run over a fixture PDF shows `images_done` increasing across two polls of `GET /api/sync/status` (the phase-129 mid-run pin idiom).
|
||||
- Coverage: **>90%** on the `app/` change.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] while a PDF's images are described, `GET /api/sync/status` reports the climbing position and `#sync-progress-text` shows `· image n/m`
|
||||
- [ ] the chunk part and the image part can show together and each disappears independently
|
||||
- [ ] the prelude / overview / summaries labels are byte-identical to pre-phase
|
||||
- [ ] `uv run pytest tests/unit -q` green; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 04 — unit + integration tests for the image progress contract
|
||||
|
||||
**Phase:** `152_sync_progress_image_pass` · **Source:** `TODO.md:3` — "The sync status on the RAG page gets \"stuck\" when it's embedding images … the status was stuck on `Syncing… … (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
The new position is pinned at both ends: the importer emits it, the status machine carries and clears it, the label formats it.
|
||||
|
||||
## Work
|
||||
1. `tests/unit/test_sync_button.py` — the status-machine pins (the phase-64/98/129 idiom: drive the runner's closures, assert the endpoint dict):
|
||||
- idle: the full-dict pin includes `images_done: 0` / `images_total: 0` and every pre-existing key unchanged;
|
||||
- mid-run: after `_image_hook(3, 15)` the endpoint reports `3/15` while `current_file` / the chunk position keep their own values (the three positions are independent);
|
||||
- per-file reset: a `_hook(...)` call for the NEXT file resets the image position to `0/0` (the pairing rule — a file with no images never shows a stale position);
|
||||
- every terminal (success / failed / cancelled) clears the image position while the FILE counts keep their final values (the phase-64 convention, restated for the new sub-position);
|
||||
- the `phase = "overview"` and `phase = "summaries"` transitions clear it (the same sites as the chunk position).
|
||||
2. `tests/unit/test_importer.py` (or the image-pass home that already uses `tests/fakes.py::FakeEmbedder`) — the hook contract:
|
||||
- a fixture PDF with N embedded images → the hook fires exactly N times, `(1,N) … (N,N)`, each call BEFORE the corresponding describe call (assert the interleaving by recording events in the fake);
|
||||
- a PDF with a pictorial span AND an embedded span → one continuous counter across both spans, never exceeding the combined total, never restarting;
|
||||
- a text-only PDF → the hook never fires;
|
||||
- `on_image_progress=None` → the `ImportSummary` and the stored rows are identical to the hook-less baseline (the byte-identical regression pin);
|
||||
- the blank-gate interaction (phase 151): images skipped by the dedupe/blank gate are NOT counted in `total` (the total is the work about to happen).
|
||||
3. `tests/unit/test_sync_button.py` (frontend-source pins) — `fmtSyncPhaseLabel`'s image part: present iff `images_total > 0`; the composed `full` string pinned for a status carrying both positions (`… (2540/2540) · chunk 125/126 · image 3/15`); the `overview`/`summaries` labels unchanged.
|
||||
4. Integration (`tests/integration/`): a full sync run over a fixture KB containing one image-bearing PDF, polling `GET /api/sync/status` from the test while the mock import runs — assert the observed `images_done` sequence is monotonically non-decreasing and ends at `images_total` (no backwards jumps, no overshoot).
|
||||
5. Run the gates: `uv run pytest -q`, then `uv run pytest --cov=app --cov-report=term-missing` (>90%).
|
||||
|
||||
## Testing & Quality
|
||||
- Coverage: **>90%** on `app/api/sync.py` + `app/rag/importer.py` new lines (every clear/reset site, each span, the `None` path).
|
||||
- Deterministic: no sleeps, no real LLM (the fakes), no reliance on wall-clock timing.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the status pins cover idle / mid-run / per-file reset / all three terminals / both phase transitions
|
||||
- [ ] the importer hook pins cover the N-image case, the continuous multi-span counter, the no-image case, and the hook-less byte-identical baseline
|
||||
- [ ] the label formatter's image part is pinned byte-for-byte
|
||||
- [ ] `uv run pytest -q` green; coverage >90%; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 05 — the dedicated E2E: the label advances through a PDF's image span
|
||||
|
||||
**Phase:** `152_sync_progress_image_pass` · **Source:** `TODO.md:3` — "The sync status on the RAG page gets \"stuck\" when it's embedding images … the status was stuck on `Syncing… … (2540/2540) · chunk 125/126` for several minutes for some reason"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
In the browser, during a PDF sync whose images take time to describe, the live label shows the image position MOVING — the owner's "stuck for several minutes" is impossible on the fixture.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_sync_image_progress.py` (new; the `tests/e2e/test_sync_chunk_progress.py` recipe — a module-scoped app on its own port, `BOR_IMAGES=true` + the `pdf` import token, the `tests/e2e/slow_llm.py` delay proxy in front of the mock so EVERY LLM call (vision describes included) takes `SLOW_DELAY_S = 1.5 s` ≫ the UI's 2 s poll, the autouse truncate, mock-only — `E2E_REAL_LLM` unsupported):
|
||||
- **fixture:** a source (local dir or `file://` git repo, the suite's existing idiom) with ONE PyMuPDF-built PDF carrying 4 genuinely distinct, non-blank images (post-phase-151 they all survive the gate) and enough text to produce ≥2 embed batches (so the chunk part and the image part are both exercised);
|
||||
- **the observation loop:** press Sync, then poll the page (not the API) and collect the DISTINCT `#sync-progress-text` values seen while the run is in flight;
|
||||
- **assertions:**
|
||||
* at least three distinct labels containing `· image ` are observed (e.g. `· image 1/4`, `· image 2/4`, `· image 3/4`) — the freeze the owner reported is a single repeated label, so "≥3 distinct image positions" IS the bug's negation;
|
||||
* the observed image positions are non-decreasing and never exceed the total;
|
||||
* the chunk part appeared earlier in the run and the image part appears during the image span (the two positions are distinct);
|
||||
* the button `title` / `aria-label` carry the same full string (the A4 untruncated contract);
|
||||
* at settle the box shows the counts line and the image part is gone (no stale position in a terminal state);
|
||||
* a page reload mid-run re-attaches with the current image position visible (the never-stale contract).
|
||||
2. Keep the suite deterministic: the slow proxy makes each position outlive the poll; no `wait_for_timeout` as a barrier — wait on the observed text values.
|
||||
3. Run in isolation until green: `uv run pytest tests/e2e/test_sync_image_progress.py -v --no-cov` (DB up: `podman compose up -d db`).
|
||||
4. Regression run of the neighbouring label suites (they must stay green unedited): `tests/e2e/test_sync_progress_box.py`, `test_sync_chunk_progress.py`, `test_sync_label_mobile.py`, `test_pdf_support.py`, `test_image_documents.py`.
|
||||
5. Commit the phase per AGENTS.md rule 8 — one atomic Conventional Commit with `--no-gpg-sign`, e.g.
|
||||
`feat(sync): report PDF image-describe progress so the sync label never stalls`
|
||||
(moves `.agents/phases/todo/152_sync_progress_image_pass/` to `complete/`).
|
||||
|
||||
## Testing & Quality
|
||||
- The dedicated E2E green IN ISOLATION (the phase gate) + the four neighbouring suites green.
|
||||
- `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
- Mock-only, deterministic, no real vision calls.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] ≥3 distinct `· image n/m` label values observed during one sync (the anti-freeze proof)
|
||||
- [ ] positions are monotonic, cleared at settle, and re-attached on a mid-run reload
|
||||
- [ ] the chunk part's behavior is unchanged
|
||||
- [ ] full suite green, coverage >90%, lint + types clean
|
||||
- [ ] the phase directory moved to `.agents/phases/complete/` and committed with `--no-gpg-sign`
|
||||
Reference in New Issue
Block a user