phase: 194_sync_pending_count
**Phase 194 — final verification pass: all green.**
**Verified (all 5 tasks already complete; nothing re-implemented needed)**
- `app/rag/importer.py`: `_prescan_pending` + `_pending_hash_stmt` (scalar `documents.file_hash`, own session, ONE `asyncio.to_thread` span, REPLACES the stats pre-walk), `on_scan_progress`, `progress` pair = pending position, `0` branch = byte-identical phase-193 walk.
- `app/config.py`/`settings_catalog.py`/`.env.example`: `sync_pending_count=1` + validator; `app/api/sync.py`: `"scan"` phase, `scan_done`/`scan_total` (additive after `files_total`, kept at every terminal), forced/throttled RSS split; `frontend/assets/sources.js`: `fmtSyncPhaseLabel` scan branch first-arm, `fmtSyncLabel` untouched.
**Tests / lint / coverage (exact commands)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4835 passed**, `app/` 99% (7240 stmts / 16 miss), exit 0
- `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → **8 passed in 150.81s** (phase suite, isolated)
- 14 sync-lineage E2E suites, one pytest process each (`test_sync_remaining_count`, `…_chunk_progress`, `…_progress_box`, `…_peak_memory_visible`, `…_label_mobile`, `…_upload_progress`, `test_archive_upload_sources`, `…_image_progress`, `…_summary_visibility`, `…_status_freshness`, `test_smoke`, `test_fast_path_deferred_content`, `test_sync_session_reset`, `test_sync_git_freeze`) → **all exit 0**
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**
**Completion criteria** — all met: `N remaining` = predicted change set (E2E arms 1/2/5, first import counts the walk, no-change → no count part); `scan` phase visible + flips to `import`, scan pair reset/kept at terminals (`test_sync_scan_phase.py`); `BOR_SYNC_PENDING_COUNT=0` walk semantics ("7 remaining" arm) + **negative refuses startup naming the field (verified live by importing `app.main`)**; pre-scan off-loop + cancel lands `cancelled` (integration pins); CLI/upload byte-identical (unedited pins green); payload additive (`_PRE_187_STATUS_KEYS | _MEMORY_KEYS | _SCAN_KEYS`); suite green, coverage >90%, lint/types clean.
**Documentation** — 3 fixes: README sync "States" bullet had an **unclosed parenthesis** (pre-existing, 193/194) and the scan-label "total is 0" clause was ambiguous about a `=0` run (it still counts the walk once the import starts); **AGENTS.md rule 17 had a stray `)`** (pre-existing from phase 191). Also verified consistency both directions for all 87 settings ↔ README ↔ `.env.example` ↔ settings catalog, the pages table ↔ `app/main.py` routes, no stale `BOR_` names, and README/AGENTS claims through phase 194.
**Notes/deviations** — no live-infra changes (prod sync knob ships default-on, no deployment touched); no git index/history commands run (all changes left in the working tree for the harness); commit criterion left to the harness as instructed.
**Next pending phase:** none — `.agents/phases/todo/` holds only `194_sync_pending_count`, which is now ready to move to `complete/`.
This commit is contained in:
@@ -1146,8 +1146,16 @@ def test_no_progress_means_no_prewalk(
|
||||
) -> None:
|
||||
"""``progress=None`` callers pay no extra pass: the real walker is hit
|
||||
exactly once per source root (one pass — as before phase 64), proven
|
||||
with a counting sentinel; with the hook it is hit twice (pre-walk for
|
||||
``total`` + the processing pass)."""
|
||||
with a counting sentinel; with the hook it is hit twice (the phase-194
|
||||
pre-scan for the pending set — the phase-64 stats pre-walk when
|
||||
``BOR_SYNC_PENDING_COUNT=0`` — plus the processing pass).
|
||||
|
||||
Phase 194 (task 01) moved the hook's pair, and this fixture is the
|
||||
second run over an ALREADY-INDEXED file: nothing is pending, so the
|
||||
pair is ``(0, 0)`` — the denominator is the predicted change set and an
|
||||
unchanged KB predicts nothing (the UI then renders no count part). The
|
||||
walk COUNT is what this test owns and it is untouched: one pass without
|
||||
the hook, two with it."""
|
||||
root = tmp_path / "nowalk"
|
||||
root.mkdir()
|
||||
(root / "a.md").write_text("# A\n\na\n")
|
||||
@@ -1186,8 +1194,12 @@ def test_no_progress_means_no_prewalk(
|
||||
)
|
||||
)
|
||||
assert summary2.files == 1
|
||||
assert walk_calls == 2 # pre-walk (total) + processing pass
|
||||
assert events == [(root.name, "a.md", 1, 1)]
|
||||
assert walk_calls == 2 # the count pass (pending set / pre-walk) + walk
|
||||
# phase 194: a.md is unchanged, so the predicted change set is empty
|
||||
# — ``total`` 0 and the position never advances. With the kill switch
|
||||
# at ``0`` this same run reports the phase-193 walk position (1 of 1);
|
||||
# pinned in tests/unit/test_sync_pending_count.py.
|
||||
assert events == [(root.name, "a.md", 0, 0)]
|
||||
finally:
|
||||
_cleanup_source(db, root.name)
|
||||
|
||||
@@ -1195,7 +1207,17 @@ def test_no_progress_means_no_prewalk(
|
||||
def test_progress_hook_counts_unchanged_and_error_files(db, tmp_path: Path) -> None:
|
||||
"""The hook fires *before* ``_index_file``: a file whose embedding
|
||||
fails (and one that re-imports as unchanged) is still reported as the
|
||||
current file — the sequence covers every importable file."""
|
||||
current file — the sequence covers every importable file.
|
||||
|
||||
Phase 194 (task 01) retargeted the pair to the PENDING position (the
|
||||
owner's "1216 remaining" fix), which reads exactly right here. The
|
||||
first run is fresh: both files are predicted, ``1 of 2`` then ``2 of 2``
|
||||
— identical to the pre-phase numbers (A1's first-import parity). The
|
||||
second run is the incremental case: ``bad.md`` rolled back with its
|
||||
embed error so its row is still missing (predicted again, the ONLY
|
||||
work), ``good.md`` is unchanged so it holds the position instead of
|
||||
advancing it — ``1 of 1``, ``1 of 1``: every file is still reported as
|
||||
the current file, and the count is the work."""
|
||||
root = tmp_path / "progress-mixed"
|
||||
root.mkdir()
|
||||
(root / "bad.md").write_text("# Bad\n\npoison content the endpoint refuses\n")
|
||||
@@ -1214,7 +1236,8 @@ def test_progress_hook_counts_unchanged_and_error_files(db, tmp_path: Path) -> N
|
||||
assert events == expected
|
||||
assert first.errors == 1 and first.added == 1
|
||||
# Re-run: good.md is now unchanged, bad.md is retried and fails
|
||||
# again — both still count in the sequence.
|
||||
# again — both still report as the current file, but only bad.md is
|
||||
# pending, so the denominator is 1 and good.md holds the position.
|
||||
events.clear()
|
||||
second = asyncio.run(
|
||||
import_sources(
|
||||
@@ -1222,7 +1245,7 @@ def test_progress_hook_counts_unchanged_and_error_files(db, tmp_path: Path) -> N
|
||||
progress=lambda s, r, d, t: events.append((s, r, d, t)),
|
||||
)
|
||||
)
|
||||
assert events == expected
|
||||
assert events == [(root.name, "bad.md", 1, 1), (root.name, "good.md", 1, 1)]
|
||||
assert second.errors == 1 and second.unchanged == 1
|
||||
finally:
|
||||
_cleanup_source(db, root.name)
|
||||
|
||||
Reference in New Issue
Block a user