phase: 129_sync_chunk_progress
All green. Final report:
**Phase 129 final verification pass — 100% green** (previous work intact; this pass re-verified everything and fixed two pre-existing README inconsistencies)
- Verified on-spec: `LLMClient.embed` per-batch `on_progress` hook; importer `on_chunk_progress` plumbing (text/image/PDF branches; summary embed excluded); `SyncStatus.current_chunk/chunks_total` reset+clear contract; UI `.sync-label-chunk` pinned part with untruncated `title`/`#sync-result`
- `uv run pytest --cov=app -q`: exit 0, ~3041 passed, 0 failed, `app/` coverage **99%** (>90% gate)
- `uv run pytest tests/e2e/test_sync_chunk_progress.py -v --no-cov` (isolated, DB up): **2 passed** (endpoint chunk climb with true `chunk_document` total + UI `· chunk n/m` part)
- `uv run ruff check .`: all checks passed · `uv run pyright`: 0 errors, 0 warnings
- Completion criteria: all 5 ✅ (incl. no behavior change in completed phases — full suite green, CLI/chat no-op paths byte-identical)
- Docs: README.md fixed 2 pre-existing inconsistencies (upload section claimed "unpacks + scans immediately" — phase 90 made it unpack+register only, scan deferred to Sync; troubleshooting claimed "120s client timeout" — actual guard is the 300 s pre-token `TURN_TIMEOUT_MS`). Phase-129 label behavior already documented. AGENTS.md re-verified consistent — no changes
- Removed stray untracked `NOTES.md` (8-byte artifact); no live-infra changes; no commits — all changes left in working tree for the harness
- Next pending phase: `130_chat_answer_clusters`
This commit is contained in:
+34
-3
@@ -22,7 +22,7 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import AsyncGenerator
|
||||
from collections.abc import AsyncGenerator, Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast
|
||||
|
||||
@@ -343,12 +343,43 @@ class LLMClient:
|
||||
batches.append(cur)
|
||||
return batches
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[list[float]]:
|
||||
"""Embed *texts* in token-budgeted batches (order always kept)."""
|
||||
async def embed(
|
||||
self,
|
||||
texts: list[str],
|
||||
*,
|
||||
on_progress: Callable[[int, int], None] | None = None,
|
||||
) -> list[list[float]]:
|
||||
"""Embed *texts* in token-budgeted batches (order always kept).
|
||||
|
||||
Progress hook (phase 129, task 01): when *on_progress* is given,
|
||||
it is called once per TOP-LEVEL batch, BEFORE the batch's request
|
||||
(a failing batch still announces its start), with
|
||||
``(first_chunk_1based, total_chunks)`` — the 1-based index of the
|
||||
first text of the in-flight batch and the total number of texts
|
||||
in this call — so a caller can render ``chunk n/m`` (the importer
|
||||
passes this through for the sync label; PLAN §7.4 "never stale").
|
||||
|
||||
``len(out) + 1`` is that first index: each batch's vectors extend
|
||||
the accumulator 1:1, so ``len(out)`` is exactly the number of
|
||||
chunks embedded by earlier batches. The :meth:`_embed_batch`
|
||||
halving backstop does NOT sub-report: a halved batch leaves the
|
||||
announced position approximate (the hook already fired for the
|
||||
original batch's start) — documented; the backstop is rare by
|
||||
construction (phase 114's ``_TooLarge`` handling). ``None`` (the
|
||||
default — every pre-phase caller: the chat retrieval path,
|
||||
:meth:`embed_one`, the summarizer, the importer) is a zero-cost
|
||||
guarded no-op: call sequence and return value are
|
||||
byte-identical to the pre-hook client.
|
||||
"""
|
||||
if not texts:
|
||||
return []
|
||||
out: list[list[float]] = []
|
||||
for chunk in self._batch_texts(texts):
|
||||
# Phase 129 (task 01): announce THIS batch's start before the
|
||||
# request — the hook precedes the try, so even a failing batch
|
||||
# reports its position.
|
||||
if on_progress is not None:
|
||||
on_progress(len(out) + 1, len(texts))
|
||||
try:
|
||||
vecs = await self._embed_batch(chunk)
|
||||
except EmbeddingError:
|
||||
|
||||
Reference in New Issue
Block a user